Title
Journal peer review as an information retrieval processJournal peer review as an information retrieval process
Author
Faculty/Department
Faculty of Social Sciences. Instructional and Educational Sciences
Research group
Book, Library and Information
Publication type
article
Publication
London,
Subject
Documentation and information
Computer. Automation
Source (journal)
Journal of documentation / Aslib. - London, 1945, currens
Volume/pages
68(2012):4, p. 527-535
ISSN
0022-0418
1758-7379
ISI
000308836300006
Carrier
E
Target language
English (eng)
Full text (Publishers DOI)
Affiliation
University of Antwerp
Abstract
Purpose - In editorial peer review systems of journals, one does not always accept the best papers. Due to different human perceptions, the evaluation of papers by peer review (for a journal) can be different from the impact that a paper has after its publication (measured by number of citations received) in this or another journal. This system (and corresponding problems) is similar to the information retrieval process in a documentary system. Also there, one retrieves not always the most relevant documents for a certain topic. This is so because the topic is described in the command language of the documentary system and this command does not always completely cover the "real topic" that one wants to describe. This paper aims to address this issue. Design/methodology/approach - Based on this statement classical information retrieval evaluation techniques were applied to the evaluation of peer review systems. Basic in such an information retrieval evaluation are the notions of precision and recall and the precision-recall-curve. Such notions are introduced here for the evaluation of peer review systems. Findings - The analogues of precision and recall are defined and their curve constructed based on peer review data from the journal Angewandte Chemie - International Edition and on citation impact data of accepted papers by this journal or rejected but published elsewhere papers. It is concluded that, clue to the imperfect peer review process (based on human evaluation), if we want to publish a high amount of qualified papers (the ones we seek), several non-qualified papers should also be accepted. Originality/value - The authors conclude that, due to the imperfect peer review process (based on human evaluation), if we want to publish a high amount of qualified papers (the ones we seek), one will also accept several non-qualified papers.
E-info
https://repository.uantwerpen.be/docman/iruaauth/d6f285/ba13458.pdf
http://gateway.webofknowledge.com/gateway/Gateway.cgi?GWVersion=2&SrcApp=PARTNER_APP&SrcAuth=LinksAMR&KeyUT=WOS:000308836300006&DestLinkType=RelatedRecords&DestApp=ALL_WOS&UsrCustomerID=ef845e08c439e550330acc77c7d2d848
http://gateway.webofknowledge.com/gateway/Gateway.cgi?GWVersion=2&SrcApp=PARTNER_APP&SrcAuth=LinksAMR&KeyUT=WOS:000308836300006&DestLinkType=FullRecord&DestApp=ALL_WOS&UsrCustomerID=ef845e08c439e550330acc77c7d2d848
http://gateway.webofknowledge.com/gateway/Gateway.cgi?GWVersion=2&SrcApp=PARTNER_APP&SrcAuth=LinksAMR&KeyUT=WOS:000308836300006&DestLinkType=CitingArticles&DestApp=ALL_WOS&UsrCustomerID=ef845e08c439e550330acc77c7d2d848
Handle