2018 Global State of Peer Review report released

The Global State of Peer Review report, summarising data from a survey of 11,000 international researchers as well as data from Web of Science, ScholarOne, and Publons, presents the state of peer review worldwide, combining quantitative and qualitative data for this very thorough breakdown of the peer review landscape.

The report asked four key questions; Who is doing the reviewing? How efficient is the review process? What do we know about peer review quality? and What does the future hold?

Among many results presented in the report, it looked at established and emerging regions publishing and reviewing research and found researchers from China, Brazil, Turkey, India, Iran, South Korea, Malaysia, and Poland were under represented when compared with reviewers from the USA, Germany, Italy, Spain, France, Netherlands, Sweden, Canada, UK, and Japan.

Editors were also found to invite researchers from their own geographical regions more than chance would predict.

In terms of efficiencies, the report found that reviewer completion rates are decreasing each year, while the total number of review invitations sent is increasing at 9.8% year-on-year. The rate of agreements to invitations in 2013 was around 0.54. In 2017 that had reduced to 0.44, with the number of completed reviews to invitations following the same rate of decline.

Fig.30 — median time to complete a review, by ESI research areas

However, some good news from the report is that peer review process may not necessarily take as long as we think it does. The report found that reviewers took a median of 16.4 days to complete a review after agreeing to the assignment (the mean is 19.1 days). Breaking these results down into 22 subject areas, the report found that review times were longest in Economics & Business and Mathematics at around 30 days average, and shortest in Pharmacology and Chemistry at around 12 days.

For several subject areas, average peer review times have been reducing, and for several others, there has been a small increase.

The report also looked at length of reviews as a method of assessing quality. Although a very imperfect measure, the report provides some interesting results. The average review wordcount
from established regions (e.g. USA, UK, Germany, Spain) was 528, and from emerging regions (e.g. Brazil, China, India, South Korea) was 250. The report also observed longer reviews in higher Impact Factor journals than lower IF ones, and offered the suggestions that editors of journals with higher IFs may solicit more reviews from regions that tend to write longer reviews, regardless of quality, or that higher IFs are associated with subjects that tend to have longer reviews.

The final section of the report looks to the future, and suggests that, should trends continue, that over the next few years the number of invitations required to secure reviewers will increase, review times may continue to decrease, and that open reviewing and reviewing policies may affect the rates at which reviewers agree; 40% of respondents under 26 are likely or highly likely to review for journals that make author and reviewer identities, and review reports public.

We have discussed just a few of many intriguing findings presented in this report, and highly recommend reading the full thing.