Studies in aspects of peer review

Abstract

The process of peer-review is considered as the sentinel of science in scholarly communications. However, with the deluge of research articles and the overload of scholarly information, it is increasingly becoming difficult for humans to keep up with the pace of the latest research. This thesis uses Machine Learning (ML) and Natural Language Processing (NLP) to investigate critical issues relevant to scholarly communications and peer review. Detecting whether a document contains sufficient new information to be deemed as novel is of immense significance in this age of data duplication. We investigate if we can automatically identify the newness in a document based on its information content against a set of relevant documents. We create two benchmark datasets: TAP-DLND 1.0 and TAP-DLND 2.0, and devise several methods for novelty detection at the document level. One particular challenge in peer review is that the journal editors and conference program chairs are overwhelmed with the ever-increasing rise in article submissions. Studies show that many submissions are not well-informed and do not fit within the scope of the intended journal or conference. We investigate how an AI could assist the editors and program chairs in identifying potential out-of-scope submissions based on the past accepted papers of the particular journal conference. We experiment with several methods, including feature-based machine learning, deep multimodal architectures, to multiview clustering. Again, the recent unprecedented growth in paper submissions in major Machine Learning and AI conferences is a grand challenge to the community to maintain high-quality reviewing practices while ensuring fair evaluation of the manuscripts. We investigate if we can identify which reviews were significant enough to be considered in the decision-making. We design an end-to-end deep network that leverages on exhaustiveness and strength of the peer review in a multi-task fashion to comprehend it's informativeness. Our DeepSentiPeer model is one of the first attempts to predict decisions in peer review automatically. Finally, finding the relevant literature or identifying the works that have inspired the current work under scrutiny is central to an evaluation in peer review. We analyze the context of citations within paper full-text to identify a given research lineage. The idea is to leverage citation significance classification to build a research lineage via a significant citation graph. We argue that finding the lineage of given research would help identify the true academic impact of a paper beyond quantitative citation counts. Awarded by : Indian Institute of Technology Patna, India on 28 August, 2021. Supervised by : Asif Ekbal and Pushpak Bhattacharyya. Available at : https://www.iitp.ac.in/~ai-nlp-ml/resources/dissertations/PhD/Tirthankar_Ghosal_PhD_Thesis_IIT_Patna.pdf.