Differences between Experts and Novices in the Review of Engineering Journal Papers

Abstract

This paper reports on a controlled experiment carried out to investigate the differences between novices and experts during the review of papers for engineering journals. A total of 17 young faculty members with PhD degrees were given instruction on the peer review process, and then had to review a short paper, for which the actual reviews were available. The quality of the reviews was assessed using criteria developed by editors of medical journals. The evidence obtained from this study helps to identify the critical areas in which reviewers fail to perform a good review, and then to develop strategies to overcome such limitations.