The effect of publishing peer review reports on referee behavior in five scholarly journals

A new paper from the PEERE group investigating the effect of open peer review reports has been published in Nature Communications.


Authors Giangiacomo Bravo, Francisco Grimaldo, Emilia López-Iñesta, plus EASE Council members Bahar Mehmani and Flaminio Squazzoni, investigated referee behavior in five scholarly journals involved in a pilot study at Elsevier. The study analysed 18,525 reviews from 2010 to 2017, measuring changes both before and during the pilot and found that publishing reports did not significantly compromise referees’ willingness to review, recommendations, or turn-around times. 

Figure 4. Proportion of recommendations by review condition and name disclosure. Error bars represent 95% CI obtained via bootstrap (1000 samples), from Bravo et.al. (2019)

The study also found that younger and non-academic scholars were more willing to agree to open review requests, and provided more positive and objective recommendations, but fewer than 1 in 10  referees agreed to reveal their identity in the published report.

This is promising news for the practicalities of open peer review, indicating a lack of negative impacts on these key aspects, and willingness to contribute transparent comments, provided anonymity is still ensured.

Bravo, G, Grimaldo, F, López-Iñesta, E, Mehmani, B, Squazzoni, F (2019) The effect of publishing peer review reports on referee behavior in five scholarly journals. Nature Communications, 322, 10.1
https://doi.org/10.1038/s41467-018-08250-2