The F3-index: an evaluation algorithm for peer reviewers

The latest paper to emerge from the PEERE peer review project has been published, open access, in Journal of Informetrics. Preliminary versions were presented at the PEERE International Conference on Peer Review, in March 2018 and subsequent meetings.

The F3-index. Valuing reviewers for scholarly journals, by Federico Bianchi, Francisco Grimaldo, and EASE Council member Flaminio Squazzoni presents an index that measures reviewer contribution to editorial processes of scholarly journals.


The index (based on a sports metaphor) considers reviewers on different context-specific dimensions, i.e., report delivery time, the length of the report and the alignment of recommendations to editorial decisions. The authors do note that that this tool is not designed to measure any form of quality, which remains an elusive element to capture, but does work effectively in indexing various performance measures.

544 reviewers from 606 submissions over six years were used for testing. The results identified outstanding contributors and weak performing reviewers efficiently.

The authors suggest that the index is a flexible tool, capable of incorporating extensions and could be adopted by journal management software to assist editors in identifying and rewarding high performing reviewers.



Federico Bianchi, Francisco Grimaldo, Flaminio Squazzoni, The F3-index. Valuing reviewers for scholarly journals, Journal of Informetrics, Volume 13, Issue 1, 2019
https://doi.org/10.1016/j.joi.2018.11.007.