RQCconf: Review Quality Collector for conferences

The implementation of this idea is online since September 2016. Read up on it at

https://reviewqualitycollector.org

This page has therefore become mostly obsolete and has accordingly been cut down to a few pieces of possible historical or future interest.

Software implementation status until September 2016

I have started to implement a web-based solution in July 2015. It went online September 1st, 2016. For the development history from that time on, see the RQC past and future page. This is a view of the history before that time:

• Fundamental functionality
  • Reading a review quality definition DONE 2015-07
  • Reading reviewing information from Easychair DONE 2015-09
  • Reading reviewing information from some other conference mgmt system TO DO?
  • Producing a reviewing receipt DONE 2016-04
• Interaction with PC chairs (primary RQC users)
  • Configuring a use of RQC for one conference DONE 2016-03
  • Review quality definition example DONE 2016-07
  • Sending report and notification emails DONE 2016-05
  • Showing reports DONE 2016-07
  • Letting the chair step in as review grader for orphaned reviews DONE 2016-07
• Interaction with reviewers (secondary RQC users)
  • Sending notification emails DONE 2016-05
  • Opting out DONE 2016-06
  • Grading a review DONE 2016-04
  • Emailing a reviewing receipt DONE 2016-07
• Infrastructure
  • Content management DONE 2016-05
  • Time-triggered activity DONE 2016-05
  • Test server & deployment DONE 2016-06
  • Staging and production server & deployment DONE 2016-08
  • Database backup and restore DONE 2016-08
• Miscellaneous
  • Security review DONE 2016-08
  • Filling gaps in the automated tests DONE 2016-08
  • Acceptance testing DONE 2016-08
  • Homepage and editorial content DONE
  • Visual design DONE

List of conference management systems

• Easychair (free cloud-hosted)
• HotCRP (open source PHP, self-hosted or commercial cloud-hosted)
• Linklings (commercial, cloud-hosted)
• CMT (free cloud-hosted)
• OCS (cloud-hosted, free for Springer proceedings)
• PCS Precision Conference System (commercial cloud-hosted)
• softconf.com START V2 (commercial cloud-hosted)
• EDAS (commercial cloud-hosted)
• ConfMaster (commercial? cloud-hosted)
• CyberChairPRO (commercial cloud-hosted, with customization service)
• OpenConf (commercial cloud-hosted or free/commercial user-hosted)
• ConfTool Pro (commercial cloud-hosted, has a list of conference announcement websites)
• MyReview (commercial cloud-hosted or free user-hosted)
• VSIS ConfTool (free user-hosted)
• Confious (commercial(?) cloud-hosted)
• Conference Reviewing System CRS (commercial cloud-hosted, free up to 40 submissions)
• ConfSys (cloud-hosted, used for 2 regular conferences)
• IAPR Commence (free user-hosted, outdated?)

Kalmukov (2011) provides some discussion on several of these.