Listwise Approach Research Articles

Listwise Generative Retrieval Models via a Sequential Learning Process

Recently, a novel generative retrieval (GR) paradigm has been proposed, where a single sequence-to-sequence model is learned to directly generate a list of relevant document identifiers (docids) given a query. Existing GR models commonly employ maximum likelihood estimation (MLE) for optimization: This involves maximizing the likelihood of a single relevant docid given an input query, with the assumption that the likelihood for each docid is independent of the other docids in the list. We refer to these models as the pointwise approach in this article. While the pointwise approach has been shown to be effective in the context of GR, it is considered sub-optimal due to its disregard for the fundamental principle that ranking involves making predictions about lists. In this article, we address this limitation by introducing an alternative listwise approach, which empowers the GR model to optimize the relevance at the docid list level. Specifically, we view the generation of a ranked docid list as a sequence learning process: At each step, we learn a subset of parameters that maximizes the corresponding generation likelihood of the i th docid given the (preceding) top i -1 docids. To formalize the sequence learning process, we design a positional conditional probability for GR. To alleviate the potential impact of beam search on the generation quality during inference, we perform relevance calibration on the generation likelihood of model-generated docids according to relevance grades. We conduct extensive experiments on representative binary and multi-graded relevance datasets. Our empirical results demonstrate that our method outperforms state-of-the-art GR baselines in terms of retrieval performance.

Supervised ranking approach to identify infLuential websites in the darknet

The anonymity and high security of the Tor network allow it to host a significant amount of criminal activities. Some Tor domains attract more traffic than others, as they offer better products or services to their customers. Detecting the most influential domains in Tor can help detect serious criminal activities. Therefore, in this paper, we present a novel supervised ranking framework for detecting the most influential domains. Our approach represents each domain with 40 features extracted from five sources: text, named entities, HTML markup, network topology, and visual content to train the learning-to-rank (LtR) scheme to sort the domains based on user-defined criteria. We experimented on a subset of 290 manually ranked drug-related websites from Tor and obtained the following results. First, among the explored LtR schemes, the listwise approach outperforms the benchmarked methods with an NDCG of 0.93 for the top-10 ranked domains. Second, we quantitatively proved that our framework surpasses the link-based ranking techniques. Third, we observed that using the user-visible text feature can obtain comparable performance to all the features with a decrease of 0.02 at NDCG@5. The proposed framework might support law enforcement agencies in detecting the most influential domains related to possible suspicious activities.

Evaluation of factors associated with HIV self-testing Acceptability and Uptake among the MSM community in Nairobi, Kenya: A cross sectional study.

Human Immunodeficiency Virus self-test (HIVST) refers to a process where a person collects his or her own specimen (blood or oral), performs a test and interprets the results. The interpretation of results can either be done in private or through support of a trusted partner. Self-test should be seen as screening and confirmatory tests are typically strongly encouraged. To determine facilitating factors for HIVST acceptability and uptake among men who have sex with men (MSM). A cross-sectional exploratory study design, targeting MSM in Nairobi was used. Adult men (aged 18-60 years) who reported to be actively engaging in anal or oral sex with men were eligible for the study. Purposive sampling was used to identify the sites where data was collected, snowballing technique was then employed to reach the respondents. Data was collected between July 2018 and June 2019. A total of 391 MSM respondent were recruited of whom 345 MSM completed the questionnaires. The missing data was handled through the listwise approach that omits those cases with the missing data and analyze the remaining data. We also excluded responses with inconsistencies in all confirmatory questions in the questionnaire. Two-thirds (64.0%) of the participants were aged 18-24 years with 13.4% being married to women and 40.2% having tertiary level of education. Majority, 72.7% were unemployed and two-thirds (64.0%) of participants were young (18-24 years) and self reported as male sex workers (58.8). There were significant associations between willingness to undertake HIV self-tests and frequency of HIV testing as well as with previous knowledge about self-testing. Habitual HIV testers were more likely to have used the HIVST kit than the non-habitual testers. Willingness to undertake confirmatory test within one month of self-testing was associated with acceptability of HIVST. Most of the MSM preferred blood sample self-test kits as compared to oral self-test kits, believing that blood test will be more accurate than oral self-test. Other factors associated with HIVST included consistent use of protection regardless of HIV status, preference of "treatment buddies". High costs of the self-test kits and inadequate knowledge on the use of HIV self-test kits were the main hindrances to HIVST uptake. This study has showed that age, habitual testing, self-care/partner care, as well as confirmatory testing and immediate introduction into care if found sero-positive were associated with the use of HIVST kit. This study contributes to the pool of knowledge of the characteristics of MSM that would adopt and embrace HIVST, and demonstrates that these MSM are self and partner care aware and conscious. The challenge however remains on how to encourage those that are not self/partner care aware to embrace HIV testing and particularly HIVST as routine practices. Future studies may need to explore potential motivators to self-testing among the young, elder MSM generations and the MSM with higher economic status in Kenya.

Design, implementation, and evaluation of the computer-aided clinical decision support system based on learning-to-rank: collaboration between physicians and machine learning in the differential diagnosis process

BackgroundWe are researching, developing, and publishing the clinical decision support system based on learning-to-rank. The main objectives are (1) To support for differential diagnoses performed by internists and general practitioners and (2) To prevent diagnostic errors made by physicians. The main features are that “A physician inputs a patient's symptoms, findings, and test results to the system, and the system outputs a ranking list of possible diseases”.MethodThe software libraries for machine learning and artificial intelligence are TensorFlow and TensorFlow Ranking. The prediction algorithm is Learning-to-Rank with the listwise approach. The ranking metric is normalized discounted cumulative gain (NDCG). The loss functions are Approximate NDCG (A-NDCG). We evaluated the machine learning performance on k-fold cross-validation. We evaluated the differential diagnosis performance with validated cases.ResultsThe machine learning performance of our system was much higher than that of the conventional system. The differential diagnosis performance of our system was much higher than that of the conventional system. We have shown that the clinical decision support system prevents physicians' diagnostic errors due to confirmation bias.ConclusionsWe have demonstrated that the clinical decision support system is useful for supporting differential diagnoses and preventing diagnostic errors. We propose that differential diagnosis by physicians and learning-to-rank by machine has a high affinity. We found that information retrieval and clinical decision support systems have much in common (Target data, learning-to-rank, etc.). We propose that Clinical Decision Support Systems have the potential to support: (1) recall of rare diseases, (2) differential diagnoses for difficult-to-diagnoses cases, and (3) prevention of diagnostic errors. Our system can potentially evolve into an explainable clinical decision support system.

List-Wise Rank Learning for Stereoscopic Image Retargeting Quality Assessment

Stereoscopic image retargeting (SIR) techniques attempt to display stereoscopic images on stereoscopic devices of various resolutions and aspect ratios to provide the users with better viewing experience. However, new quality perceptual problems emerge in the retargeted stereoscopic images generated by current SIR operators are quite different from those in the retargeted 2D images. In this paper, we dedicate to exploring the perceptual quality-related factors (e.g., shape preservation, object preservation and visual comfort.) of retargeted stereoscopic images, and propose a novel quality evaluation metric for SIR to achieve a more consistent evaluation with 3D perception and image degradation mechanism in the SIR process. Moreover, image quality features and 3D perceptual features are integrated into one representation for an overall perceptual quality prediction using a list-wise ranking approach, which gives priority to the ranking among the SIR results generated from the same stereoscopic source. Experimental results demonstrate that the proposed method outperforms most quality models developed for retargeted 2D/stereoscopic images.

Smart Phishing Detection in Web Pages using Supervised Deep Learning Classification and Optimization Technique ADAM

Phishing is the process of portraying malignant web pages in the place of genuine web pages to obtain important and delicate information from the end-user. Nowadays phishing is considered as one of the most serious threats to web security. Most of the existing techniques for phishing detection use Bayesian classification for differentiating malignant web pages from genuine web pages. These methods work well if a dataset contains less no of web pages and they provide accuracy up to 90 percent. In recent years the size of the web is increasing tremendously and the existing methods have not provideda good enough accuracy for large datasets. So this paper proffers an innovative approach to identify phishing websites using hyperlinks available in the source code of the HTML page in the corresponding website. The proposed method uses a feature vector with 30 parameters to detect malignant web pages. These features are used in training the supervised Deep Neural Network model with Adam optimizer for differentiating fraudulent websites from genuine websites. The proposed deep learning model with Adam Optimizer uses a Listwise approach to classify phishing websites and genuine websites. The performance of the proposed approach is decent when compared to other traditional machine learning approaches like SVM, Adaboost, AdaRank. The results show that the proposed approach provides more accurate results in detecting phishing websites.

Semantic Service Search in IT Crowdsourcing Platform: A Knowledge Graph-Based Approach

Understanding user’s search intent in vertical websites like IT service crowdsourcing platform relies heavily on domain knowledge. Meanwhile, searching for services accurately on crowdsourcing platforms is still difficult, because these platforms do not contain enough information to support high-performance search. To solve these problems, we build and leverage a knowledge graph named ITServiceKG to enhance search performance of crowdsourcing IT services. The main ideas are to (1) build an IT service knowledge graph from Wikipedia, Baidupedia, CN-DBpedia, StuQ and data in IT service crowdsourcing platforms, (2) use properties and relations of entities in the knowledge graph to expand user query and service information, and (3) apply a listwise approach with relevance features and topic features to re-rank the search results. The results of our experiments indicate that our approach outperforms the traditional search approaches.

Listwise learning to rank with extreme order sensitive constraint via cross‐correntropy

SummaryIn information retrieval, learning to rank was originally proposed for ranking retrieved documents according to their relevance by machine learning techniques. In general, it can effectively improve the performance of critical ranking tasks, where the listwise approach is one of the most popular approaches adopted for ranking. However, considering that documents with more relevance receive greater attention, current ranking function based listwise approaches discriminate correctly ranked sequences with incorrect sequences; however, they are incapable of recognizing which incorrect sequences are relatively more satisfied. In this study, we propose a listwise approach based on an extreme order sensitive constraint to overcome the aforementioned drawback. We first define this constraint and then design a novel listwise loss function, ListXOS, based on the constraint via cross‐correntropy to improve the performance of ranking tasks. Experimental results on three public datasets present improved performance of learning to rank by 6% compared with conventional methods, which demonstrate the superiority of the proposed approach over related state‐of‐the‐art approaches.

Clustering-Based Transductive Semi-Supervised Learning for Learning-to-Rank

Learning-to-rank (LTR) is a very hot topic of research for information retrieval (IR). LTR framework usually learns the ranking function using available training data that are very cost-effective, time-consuming and biased. When sufficient amount of training data is not available, semi-supervised learning is one of the machine learning paradigms that can be applied to get pseudo label from unlabeled data. Cluster and label is a basic approach for semi-supervised learning to identify the high-density region in data space which is mainly used to support the supervised learning. However, clustering with conventional method may lead to prediction performance which is worse than supervised learning algorithms for application of LTR. Thus, we propose rank preserving clustering (RPC) with PLocalSearch and get pseudo label for unlabeled data. We present semi-supervised learning that adopts clustering-based transductive method and combine it with nonmeasure specific listwise approach to learn the LTR model. Moreover, each cluster follows the multi-task learning to avoid optimization of multiple loss functions. It reduces the training complexity of adopted listwise approach from an exponential order to a polynomial order. Empirical analysis on the standard datasets (LETOR) shows that the proposed model gives better results as compared to other state-of-the-arts.

Listwise approach based on the cross‐correntropy for learning to rank

The problem of learning to rank is addressed and a novel listwise approach by taking document retrieval as an example is proposed. It first introduces the concept of cross-correntropy into learning to rank and then proposes the listwise loss function based on the cross-correntropy between the ranking list given by the label and the one predicted by training model. The use of the cross-correntropy loss leads to the development of the listwise approach called ListCCE, which employs the gradient descent algorithm to train a neural network model. Experimental results tested on publicly available data sets show that the proposed approach performs better than some existing approaches.

Kernelized cost-sensitive listwise ranking

This thesis research aims to conduct a study on a cost-sensitive listwise approach to learning to rank. Learning to Rank is an area of application in machine learning, typically supervised, to build ranking models for Information Retrieval systems. The training data consists of lists of items with some partial order specified induced by an ordinal score or a binary judgment (relevant/not relevant). The model purpose is to produce a permutation of the items in this list in a way which is close to the rankings in the training data. This technique has been successfully applied to ranking, and several approaches have been proposed since then, including the listwise approach. A cost-sensitive version of that is an adaptation of this framework which treats the documents within a list with different probabilities, i.e. attempt to impose weights for the documents with higher cost. We then take this algorithm to the next level by kernelizing the loss and exploring the optimization in different spaces. Among the different existing likelihood algorithms, we choose ListMLE as primary focus of experimentation, since it has been shown to be the approach with the best empirical performance. The theoretical framework is given along with its mathematical properties. Experimentation is done on the benchmark LETOR dataset. They contain queries and some characteristics of the retrieved documents and its human judgments on the relevance of the documents on the queries. Based on that we will show how the Kernel Cost-Sensitive ListMLE performs compared to the baseline Plain Cost-Sensitive ListMLE, ListNet, and RankSVM and show different aspects of the proposed loss function within different families of kernels.

Comparing Pointwise and Listwise Objective Functions for Random-Forest-Based Learning-to-Rank

Current random-forest (RF)-based learning-to-rank (LtR) algorithms use a classification or regression framework to solve the ranking problem in a pointwise manner. The success of this simple yet effective approach coupled with the inherent parallelizability of the learning algorithm makes it a strong candidate for widespread adoption. In this article, we aim to better understand the effectiveness of RF-based rank-learning algorithms with a focus on the comparison between pointwise and listwise approaches. We introduce what we believe to be the first listwise version of an RF-based LtR algorithm. The algorithm directly optimizes an information retrieval metric of choice (in our case, NDCG) in a greedy manner. Direct optimization of the listwise objective functions is computationally prohibitive for most learning algorithms, but possible in RF since each tree maximizes the objective in a coordinate-wise fashion. Computational complexity of the listwise approach is higher than the pointwise counterpart; hence for larger datasets, we design a hybrid algorithm that combines a listwise objective in the early stages of tree construction and a pointwise objective in the latter stages. We also study the effect of the discount function of NDCG on the listwise algorithm. Experimental results on several publicly available LtR datasets reveal that the listwise/hybrid algorithm outperforms the pointwise approach on the majority (but not all) of the datasets. We then investigate several aspects of the two algorithms to better understand the inevitable performance tradeoffs. The aspects include examining an RF-based unsupervised LtR algorithm and comparing individual tree strength. Finally, we compare the the investigated RF-based algorithms with several other LtR algorithms.