Terms Of NDCG Research Articles

BCE4ZSR: Bi-encoder empowered by teacher cross-encoder for zero-shot cold-start news recommendation

In the realm of news recommendations, the persistent challenge of the cold-start problem continues to impede progress. Existing approaches rely heavily on information exchange between news articles and users to personalize news recommendations and have struggled to adapt to users and news articles without historical interaction data. In this paper, we introduce BCE4ZSR, a novel framework that leverages a rarely utilized zero-shot approach to effectively tackle the cold-start problem in news recommendations. The proposed approach consists of two main steps: First, we generate embeddings for inference with a sentence transformer (a bi-encoder). In the second step, a fine-tuned transformer model is augmented during the training phase to distil the bi-encoder with knowledge from the cross-encoder model using a student–teacher framework. As a result of this synergy, the cross-encoder serves as a teacher, imparting its knowledge to the bi-encoder. The proposed technique can be applied to any neural news recommender system and is empirically evaluated in both cold-start and regular user-news interaction situations. Experiments on real-world benchmark datasets (MIND-small and MIND-large) indicated that BCE4ZSR outperformed the baseline methods in terms of nDCG@k, AUC, and MRR. Specifically, AUC improvement of 1.5%–6% & 2.2%–7.93%; MRR improved by 1%–13.85% & 0.9%–14.72%; nDCG@5 improved by 4.9%–18.79% & 2.1%–16.44%; and nDCG@10 improved by 1.7%–15.18% & 1.9%–14.24% in the cold start and warm user scenarios respectively, proving the superiority of our model compared to baseline methods.

Effectiveness Perspectives and a Deep Relevance Model for Spatial Keyword Queries

Geo-textual objects with both geographical location and textual description are gaining in prevalence. Over the past decades, substantial research has been conducted on spatial keyword queries, which integrate location into keyword-based querying of geo-textual content. However, existing proposals mostly focus on efficiency for processing spatial keyword queries, and little effort was made to address the effectiveness perspectives. In this work, using two datasets with ground truth query results, we evaluate the effectiveness of standard spatial keyword queries. Our evaluation results show that the TkQ query that ranks objects by a weighted combination of spatial proximity and text relevance is the most effective. Motivated by the finding, we propose a Deep relevance with Weight learning (DrW) model to further improve the effectiveness of the retrieval ranking. DrW is featured with two novel ideas: First, we propose a neural network architecture to learn the text relevance matching over the local interaction between the query and geo-textual objects. Second, we find that a query-dependent weight to balance text relevance and spatial proximity in ranking can improve effectiveness, and we develop a learning-based method to learn the query-dependent weight. Experimental results reveal that our model outperforms state-of-the-art methods on effectiveness, with improvements up to 32.15%, 32.34%, and 33.00% in terms of NDCG@3, NDCG@5, and MRR.

Addressing the Cold-Start Problem in Collaborative Filtering Through Positive-Unlabeled Learning and Multi-Target Prediction

The cold-start problem is one of the main challenges in recommender systems and specifically in collaborative filtering methods. Such methods, albeit effective, typically can not handle new items or users that do not have any prior interaction activity in the system. In this paper, we propose a novel two-step approach to address the cold-start problem. First, we view the user-item interactions in a positive unlabeled (PU) learning setting and reconstruct the interaction matrix between users and warm items, detecting missing links and recommending warm items to existing users. Second, an inductive multi-target regressor is trained on this reconstructed interaction matrix and subsequently predicts interactions for new items that enter the system. To the best of our knowledge, this is the first time that such a two-step PU learning method is proposed to address the cold-start problem in recommender systems. To evaluate the proposed approach, we employed four benchmark datasets from movie and news recommendation domains with explicit and implicit feedback. We compared our method against three other competitor approaches that address the cold-start problem and showed that our proposed method significantly outperforms them, achieving in a case an increase of 16.9% in terms of NDCG.

A GA-based algorithm meets the fair ranking problem

Ranking items is a vital component in almost every application dealing with selecting the most suitable items among a pool of candidates. Yet, specific individuals or groups may be systematically disadvantaged in getting the opportunity of appearing on the ranking list. The fair ranking problem aims at mitigating the bias imposed on protected groups (i.e., disadvantaged groups) while preserving the total quality of the ranking list as high as possible. FA*IR is one of the existing algorithms, which finds the exact solutions for only one protected group, considering a given minimum number of protected items at every prefix of a ranking list. However, when an item belongs to more than two protected groups achieving optimal solutions gets more difficult. This paper proposes an algorithm called FARGO, a fair ranking algorithm based on the genetic algorithm (GA) enhanced by the simulated annealing (SA) that is able to handle any number of protected groups. A new objective function is also proposed by incorporating the main goals of the problem, which is utilized as FARGO's fitness function. Furthermore, a novel evaluation metric named Expected Gain Ratio (EGR) is introduced to assess a fair ranking algorithm's output. Experimental results on real-world datasets demonstrate that FARGO attains comparative performance with FA*IR for one protected group and finds near-optimal solutions for more than one protected group in terms of NDCG and EGR. Note that involving other concepts such as exposure is not a matter of this paper and can be an interesting subject for further studies.

Multi-Stage Conversational Passage Retrieval: An Approach to Fusing Term Importance Estimation and Neural Query Rewriting

Conversational search plays a vital role in conversational information seeking. As queries in information seeking dialogues are ambiguous for traditional ad hoc information retrieval (IR) systems due to the coreference and omission resolution problems inherent in natural language dialogue, resolving these ambiguities is crucial. In this article, we tackle conversational passage retrieval, an important component of conversational search, by addressing query ambiguities with query reformulation integrated into a multi-stage ad hoc IR system. Specifically, we propose two conversational query reformulation (CQR) methods: (1) term importance estimation and (2) neural query rewriting. For the former, we expand conversational queries using important terms extracted from the conversational context with frequency-based signals. For the latter, we reformulate conversational queries into natural, stand-alone, human-understandable queries with a pretrained sequence-to-sequence model. Detailed analyses of the two CQR methods are provided quantitatively and qualitatively, explaining their advantages, disadvantages, and distinct behaviors. Moreover, to leverage the strengths of both CQR methods, we propose combining their output with reciprocal rank fusion, yielding state-of-the-art retrieval effectiveness, 30% improvement in terms of NDCG@3 compared to the best submission of Text REtrieval Conference (TREC) Conversational Assistant Track (CAsT) 2019.

Knowledge-Enhanced Top-K Recommendation in Poincaré Ball

Personalized recommender systems are increasingly important as more content and services become available and users struggle to identify what might interest them. Thanks to the ability for providing rich information, knowledge graphs (KGs) are being incorporated to enhance the recommendation performance and interpretability. To effectively make use of the knowledge graph, we propose a recommendation model in the hyperbolic space, which facilitates the learning of the hierarchical structure of knowledge graphs. Furthermore, a hyperbolic attention network is employed to determine the relative importances of neighboring entities of a certain item. In addition, we propose an adaptive and fine-grained regularization mechanism to adaptively regularize items and their neighboring representations. Via a comparison using three real-world datasets with state-of-the-art methods, we show that the proposed model outperforms the best existing models by 2-16% in terms of NDCG@K on Top-K recommendation.

RTEX: A novel framework for ranking, tagging, and explanatory diagnostic captioning of radiography exams.

ObjectiveThe study sought to assist practitioners in identifying and prioritizing radiography exams that are more likely to contain abnormalities, and provide them with a diagnosis in order to manage heavy workload more efficiently (eg, during a pandemic) or avoid mistakes due to tiredness.Materials and MethodsThis article introduces RTEx, a novel framework for (1) ranking radiography exams based on their probability to be abnormal, (2) generating abnormality tags for abnormal exams, and (3) providing a diagnostic explanation in natural language for each abnormal exam. Our framework consists of deep learning and retrieval methods and is assessed on 2 publicly available datasets.ResultsFor ranking, RTEx outperforms its competitors in terms of nDCG@k. The tagging component outperforms 2 strong competitor methods in terms of F1. Moreover, the diagnostic captioning component, which exploits the predicted tags to constrain the captioning process, outperforms 4 captioning competitors with respect to clinical precision and recall.DiscussionRTEx prioritizes abnormal exams toward the improvement of the healthcare workflow by introducing a ranking method. Also, for each abnormal radiography exam RTEx generates a set of abnormality tags alongside a diagnostic text to explain the tags and guide the medical expert. Human evaluation of the produced text shows that employing the generated tags offers consistency to the clinical correctness and that the sentences of each text have high clinical accuracy.ConclusionsThis is the first framework that successfully combines 3 tasks: ranking, tagging, and diagnostic captioning with focus on radiography exams that contain abnormalities.

Mining shape of expertise: A novel approach based on convolutional neural network

Expert finding addresses the task of retrieving and ranking talented people on the subject of user query. It is a practical issue in the Community Question Answering networks. Recruiters looking for knowledgeable people for their job positions are the most important clients of expert finding systems. In addition to employee expertise, the cost of hiring new staff is another significant concern for organizations. An efficient solution to cope with this concern is to hire T-shaped experts that are cost-effective. In this study, we have proposed a new deep model for T-shaped experts finding based on Convolutional Neural Networks. The proposed model tries to match queries and users by extracting local and position-invariant features from their corresponding documents. In other words, it detects users’ shape of expertise by learning patterns from documents of users and queries simultaneously. The proposed model contains two parallel CNN’s that extract latent vectors of users and queries based on their corresponding documents and join them together in the last layer to match queries with users. Experiments on a large subset of Stack Overflow documents indicate the effectiveness of the proposed method against baselines in terms of NDCG, MRR, and ERR evaluation metrics.

Automatic test report augmentation to assist crowdsourced testing

In crowdsourced mobile application testing, workers are often inexperienced in and unfamiliar with software testing. Meanwhile, workers edit test reports in descriptive natural language on mobile devices. Thus, these test reports generally lack important details and challenge developers in understanding the bugs. To improve the quality of inspected test reports, we issue a new problem of test report augmentation by leveraging the additional useful information contained in duplicate test reports. In this paper, we propose a new framework named test report augmentation framework (TRAF) towards resolving the problem. First, natural language processing (NLP) techniques are adopted to preprocess the crowdsourced test reports. Then, three strategies are proposed to augment the environments, inputs, and descriptions of the inspected test reports, respectively. Finally, we visualize the augmented test reports to help developers distinguish the added information. To evaluate TRAF, we conduct experiments over five industrial datasets with 757 crowdsourced test reports. Experimental results show that TRAF can recommend relevant inputs to augment the inspected test reports with 98.49% in terms of NDCG and 88.65% in terms of precision on average, and identify valuable sentences from the descriptions of duplicates to augment the inspected test reports with 83.58% in terms of precision, 77.76% in terms of recall, and 78.72% in terms of F-measure on average. Meanwhile, empirical evaluation also demonstrates that augmented test reports can help developers understand and fix bugs better.

Entity representation for pairwise collaborative ranking using restricted Boltzmann machine

Pairwise preference data is one of several kinds of feedback data that many modern intelligent systems gather and process in order to help people make better decisions. Recommender systems are of main categories of such intelligent systems. An emerging recommendation approach in this domain is Pairwise Collaborative Ranking (PCR) which seeks to learn pairwise preferences of users over items. Current PCR algorithms generally learn latent factors for users and items that are consistent to the ground truth pairwise preferences, instead of directly extracting latent factors for pairwise preferences. Although this is the typical approach for rating oriented recommendation, it does not seem to be the best option for pairwise collaborative ranking. In this paper RBMRank, a novel model-based PCR approach, is introduced, that is designed for recommendation based on pairwise preference data. RBMRank uses the power of the restricted Boltzmann machines to map the entities of the system into dense feature spaces. Its embedding system is able to generate useful representations even when dealing with sparse data sets, that is a very important issue in recommender systems domain. The other important feature of RBMRank is that it is able to extract latent factors for pairwise preferences, instead of items, that allows it to directly predict the unknown preferences of each user. Experimental results on MovieLens datasets show up to 5.9% improvement of accuracy in terms of NDCG@5 and NDCG@10 compared to other state-of-the-art algorithms in the field of collaborative ranking.

GeoMF++

Location recommendation is an important means to help people discover attractive locations. However, extreme sparsity of user-location matrices leads to a severe challenge, so it is necessary to take implicit feedback characteristics of user mobility data into account and leverage the location’s spatial information. To this end, based on previously developed GeoMF, we propose a scalable and flexible framework, dubbed GeoMF++, for joint geographical modeling and implicit feedback-based matrix factorization. We then develop an efficient optimization algorithm for parameter learning, which scales linearly with data size and the total number of neighbor grids of all locations. GeoMF++ can be well explained from two perspectives. First, it subsumes two-dimensional kernel density estimation so that it captures spatial clustering phenomenon in user mobility data; Second, it is strongly connected with widely used neighbor additive models, graph Laplacian regularized models, and collective matrix factorization. Finally, we extensively evaluate GeoMF++ on two large-scale LBSN datasets. The experimental results show that GeoMF++ consistently outperforms the state-of-the-art and other competing baselines on both datasets in terms of NDCG and Recall. Besides, the efficiency studies show that GeoMF++ is much more scalable with the increase of data size and the dimension of latent space.

MiRiaD: A Text Mining Tool for Detecting Associations of microRNAs with Diseases.

BackgroundMicroRNAs are increasingly being appreciated as critical players in human diseases, and questions concerning the role of microRNAs arise in many areas of biomedical research. There are several manually curated databases of microRNA-disease associations gathered from the biomedical literature; however, it is difficult for curators of these databases to keep up with the explosion of publications in the microRNA-disease field. Moreover, automated literature mining tools that assist manual curation of microRNA-disease associations currently capture only one microRNA property (expression) in the context of one disease (cancer). Thus, there is a clear need to develop more sophisticated automated literature mining tools that capture a variety of microRNA properties and relations in the context of multiple diseases to provide researchers with fast access to the most recent published information and to streamline and accelerate manual curation.MethodsWe have developed miRiaD (microRNAs in association with Disease), a text-mining tool that automatically extracts associations between microRNAs and diseases from the literature. These associations are often not directly linked, and the intermediate relations are often highly informative for the biomedical researcher. Thus, miRiaD extracts the miR-disease pairs together with an explanation for their association. We also developed a procedure that assigns scores to sentences, marking their informativeness, based on the microRNA-disease relation observed within the sentence.ResultsmiRiaD was applied to the entire Medline corpus, identifying 8301 PMIDs with miR-disease associations. These abstracts and the miR-disease associations are available for browsing at http://biotm.cis.udel.edu/miRiaD. We evaluated the recall and precision of miRiaD with respect to information of high interest to public microRNA-disease database curators (expression and target gene associations), obtaining a recall of 88.46–90.78. When we expanded the evaluation to include sentences with a wide range of microRNA-disease information that may be of interest to biomedical researchers, miRiaD also performed very well with a F-score of 89.4. The informativeness ranking of sentences was evaluated in terms of nDCG (0.977) and correlation metrics (0.678-0.727) when compared to an annotator’s ranked list.ConclusionsmiRiaD, a high performance system that can capture a wide variety of microRNA-disease related information, extends beyond the scope of existing microRNA-disease resources. It can be incorporated into manual curation pipelines and serve as a resource for biomedical researchers interested in the role of microRNAs in disease. In our ongoing work we are developing an improved miRiaD web interface that will facilitate complex queries about microRNA-disease relationships, such as “In what diseases does microRNA regulation of apoptosis play a role?” or “Is there overlap in the sets of genes targeted by microRNAs in different types of dementia?”.”Electronic supplementary materialThe online version of this article (doi:10.1186/s13326-015-0044-y) contains supplementary material, which is available to authorized users.

Using Implicit Preference Relations to Improve Recommender Systems

Our work is generally focused on making recommendations for small or medium-sized e-commerce portals, where we are facing scarcity of explicit feedback, low user loyalty, short visit durations or a low number of visited objects. In this paper, we present a novel approach to use a specific user behavior pattern as implicit feedback, forming binary relations between objects. Our hypothesis is that if a user selects a specific object from the list of displayed objects, it is an expression of his/her binary preference between the selected object and others that are visible, but ignored. We expand this relation with content-based similarity of objects. We define implicit preference relation (IPR) a partial ordering of objects based on similarity expansion of ignored-selected preference relation. We propose a merging algorithm utilizing the synergic effect of two approaches this IPR partial ordering and a list of recommended objects based on any/another algorithm. We report on a series of offline experiments with various recommending algorithms on two real-world e-commerce datasets. The merging algorithm could improve the ranked list of most of the evaluated algorithms in terms of nDCG. Furthermore, we also provide access to the relevant datasets and source codes for further research.