In the data management community, we have been lucky to have motivated and conscientious leadership over the years which has led to significant improvements in the review process of our conferences, like SIGMOD, VLDB, ICDE, and EDBT.
One of the changes a few years back was the introduction of the feedback phase during which the authors have the opportunity to see the reviews without the scores and provide comments on the main points raised in the reviews. While this is frequently a very useful resource for the review process, it can also become a bottleneck in some cases, especially if the reviewers have clearly decided to accept or reject a paper. In such cases, the feedback phase prolongs the review period by 1-2 months and requires the authors to spend a week on writing a short document (à la “revision document” without the revision) which is not really impacting the review process. And while most reviewers are also conscientious and diligently read the feedback, in some cases the feedback it is not even read.
So, should we get rid of the feedback phase altogether?
No, I agree with the previous leadership of SIGMOD that the feedback is a valuable resource. However, I have a series of small amendments to propose to make it (even) more useful.
First, request from the reviewers to try and agree on a decision as early as possible, and term those “Early Accept”, “Early Reject”, and “Early Revision”. The first attempt in that direction happened in the reviewing process of ICDE 2022 with “Early Accept” and “Early Reject”. The rationale is that it is ideal to allow the process (and the researchers) to focus on the important things (doing research) as soon as possible. If one of those three decisions is made, no feedback would be required.
Second, if the reviewers cannot finalize their decision, then the reviewers can ask specific questions on the submitted paper (for example, up to 7 such questions) for which the authors have to provide concise timely responses (e.g., 500 words for each question) in order to help the reviewers reach a conclusion, and do so quickly. The rationale is that while generic feedback about, potentially, all comments might not be always important or even needed in the review process, targeted feedback to answer questions posed by the reviewers, will significantly accelerate and increase the quality of the review process.
Overall, this proposal aims to (i) help have a faster turnaround in the review process, (ii) reduce review and submission fatigue, and (iii) help the authors focus on doing research.