Is peer review any good? A quantitative analysis of peer review

Casati, Fabio and Marchese, Maurizio and Ragone, Azzurra and Turrini, Matteo (2009) Is peer review any good? A quantitative analysis of peer review. Technical Report DISI-09-045, Ingegneria e Scienza dell'Informazione, University of Trento.

Abstract

In this paper we focus on the analysis of peer reviews and reviewers behavior in conference review processes. We report on the development, definition and rationale of a theoretical model for peer review processes to support the identification of appropriate metrics to assess the processes main properties. We then apply the proposed model and analysis framework to data sets about reviews of conference papers. We discuss in details results, implications and their eventual use toward improving the analyzed peer review processes. Conclusions and plans for future work close the paper.