Abstract
Ranking models are the main components of information retrieval systems. Several approaches to ranking are based on traditional machine learning algorithms using a set of hand-crafted features. Recently, researchers have leveraged deep learning models in information retrieval. These models are trained end-to-end to extract features from the raw data for ranking tasks, so that they overcome the limitations of hand-crafted features. A variety of deep learning models have been proposed, and each model presents a set of neural network components to extract features that are used for ranking. In this paper, we compare the proposed models in the literature along different dimensions in order to understand the major contributions and limitations of each model. In our discussion of the literature, we analyze the promising neural components, and propose future research directions. We also show the analogy between document retrieval and other retrieval tasks where the items to be ranked are structured documents, answers, images and videos.
Highlights
Recent advances in neural networks enable the improvement in the performance of multiple fields including computer vision, natural language processing, machine translation, speech recognition, etc
These features are used for an in-depth overview of several neural ranking models that are proposed in the literature. – In Sect. 8, we show that neural ranking models can generalize beyond document retrieval
An end-to-end learning to rank (LTR) architecture for ad-hoc document retrieval consists of extracting features from a query-document pair using a feature extractor, and mapping the extracted features to a real-valued relevance score using a ranking function
Summary
Recent advances in neural networks enable the improvement in the performance of multiple fields including computer vision, natural language processing, machine translation, speech recognition, etc. In order to return a useful set of documents to the user, the retrieval model should be able to rank documents based on the given query This means that the model ranks the documents using features from both the query and documents. We briefly introduce the deep learning terminology and techniques most commonly used in ad-hoc retrieval This includes Convolutional Neural Networks (CNN) (LeCun & Bengio, 1998), Recurrent Neural Networks (RNN) (Elman, 1990), Long Short-Term Memory (LSTM) (Hochreiter & Schmidhuber, 1997), Gated Recurrent Units (GRU) (Cho et al, 2014), word embedding techniques (Wang et al, 2020), attention mechanism (Bahdanau et al, 2015), deep contextualized language models (Peters et al, 2018; Devlin et al, 2019), and knowledge graphs (Wang et al, 2017). CNNs were first introduced to solve image-related tasks such as image classification (Krizhevsky et al, 2017; Simonyan & Zisserman, 2015; Szegedy et al, 2015; Ioffe & Szegedy, 2015; Szegedy et al, 2016), and were later adapted to solve textrelated tasks such as NLP and information retrieval (Collobert et al, 2011; Dai et al, 2018; Hui et al, 2017; Jaech et al, 2017; Lan & Xu, 2018; McDonald et al, 2018; Pang et al, 2016b; Tang & Yang, 2019; Kalchbrenner et al, 2014)
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
