Cold-start Settings Research Articles

Multimodal Pre-training for Sequential Recommendation via Contrastive Learning

Sequential recommendation systems often suffer from data sparsity, leading to suboptimal performance. While multimodal content, such as images and text, has been utilized to mitigate this issue, its integration within sequential recommendation frameworks remains challenging. Current multimodal sequential recommendation models are often unable to effectively explore and capture correlations among behavior sequences of users and items across different modalities, either neglecting correlations among sequence representations or inadequately capturing associations between multimodal data and sequence data in their representations. To address this problem, we explore multimodal pre-training in the context of sequential recommendation, with the aim of enhancing fusion and utilization of multimodal information. We propose a novel Multimodal Pre-training for Sequential Recommendation (MP4SR) framework, which utilizes contrastive losses to capture the correlation among different modality sequences of users, as well as the correlation among different modality sequences of users and items. MP4SR consists of three key components: (1) multimodal feature extraction; (2) a backbone network, Multimodal Mixup Sequence Encoder (M 2 SE); and (3) pre-training tasks. After utilizing pre-trained encoders to generate initial multimodal features of items, M 2 SE adopts a complementary sequence mixup strategy to fuse different modality sequences, and leverages contrastive learning to capture modality interactions at the sequence-to-sequence and sequence-to-item levels. Extensive experiments on four real-world datasets demonstrate that MP4SR outperforms state-of-the-art approaches in both normal and cold-start settings. We further highlight the efficacy of incorporating multimodal pre-training in sequential recommendation representation learning, serving as an effective regularizer and optimizing the parameter space for the recommendation task.

DTA-GTOmega: Enhancing Drug-Target Binding Affinity Prediction with Graph Transformers Using OmegaFold Protein Structures

Understanding drug-protein interactions is crucial for elucidating drug mechanisms and optimizing drug development. However, existing methods have limitations in representing the three-dimensional structure of targets and capturing the complex relationships between drugs and targets. This study proposes a new method, DTA-GTOmega, for predicting drug-target binding affinity. DTA-GTOmega utilizes OmegaFold to predict protein three-dimensional structure and construct target graphs, while processing drug SMILES sequences with RDKit to generate drug graphs. By employing multi-layer graph transformer modules and co-attention modules, this method effectively integrates atomic-level features of drugs and residue-level features of targets, accurately modeling the complex interactions between drugs and targets, thereby significantly improving the accuracy of binding affinity predictions. Our method outperforms existing techniques on benchmark datasets such as KIBA, Davis, and BindingDB_Kd under cold-start setting. Moreover, DTA-GTOmega demonstrates competitive performance in real-world DTI scenarios involving DrugBank data and drug-target interactions related to cardiovascular and nervous system-related diseases, highlighting its robust generalization capabilities. Additionally, the introduced DTI evaluation metrics further validate DTA-GTOmega’s potential in handling imbalanced data.

ParaCPI: A Parallel Graph Convolutional Network for Compound-Protein Interaction Prediction.

Identifying compound-protein interactions (CPIs) is critical in drug discovery, as accurate prediction of CPIs can remarkably reduce the time and cost of new drug development. The rapid growth of existing biological knowledge has opened up possibilities for leveraging known biological knowledge to predict unknown CPIs. However, existing CPI prediction models still fall short of meeting the needs of practical drug discovery applications. A novel parallel graph convolutional network model for CPI prediction (ParaCPI) is proposed in this study. This model constructs feature representation of compounds using a unique approach to predict unknown CPIs from known CPI data more effectively. Experiments are conducted on five public datasets, and the results are compared with current state-of-the-art (SOTA) models under three different experimental settings to evaluate the model's performance. In the three cold-start settings, ParaCPI achieves an average performance gain of 26.75%, 23.84%, and 14.68% in terms of area under the curve compared with the other SOTA models. In addition, the results of the experiments in the case study show ParaCPI's superior ability to predict unknown CPIs based on known data, with higher accuracy and stronger generalization compared with the SOTA models. Researchers can leverage ParaCPI to accelerate the drug discovery process.

Self-Supervised Bot Play for Transcript-Free Conversational Critiquing with Rationales

Conversational critiquing in recommender systems offers a way for users to engage in multi-turn conversations to find items they enjoy. For users to trust an agent and give effective feedback, the recommender system must be able to explain its suggestions and rationales. We develop a two-part framework for training multi-turn conversational critiquing in recommender systems that provide recommendation rationales that users can effectively interact with to receive better recommendations. First, we train a recommender system to jointly suggest items and explain its reasoning via subjective rationales. We then fine-tune this model to incorporate iterative user feedback via self-supervised bot-play. Experiments on three real-world datasets demonstrate that our system can be applied to different recommendation models across diverse domains to achieve state-of-the-art performance in multi-turn recommendation. Human studies show that systems trained with our framework provide more useful, helpful, and knowledgeable suggestions in warm- and cold-start settings. Our framework allows us to use only product reviews during training, avoiding the need for expensive dialog transcript datasets that limit the applicability of previous conversational recommender agents.

3D graph neural network with few-shot learning for predicting drug–drug interactions in scaffold-based cold start scenario

Understanding drug–drug interactions (DDI) of new drugs is critical for minimizing unexpected adverse drug reactions. The modeling of new drugs is called a cold start scenario. In this scenario, Only a few structural information or physicochemical information about new drug is available. The 3D conformation of drug molecules usually plays a crucial role in chemical properties compared to the 2D structure. 3D graph network with few-shot learning is a promising solution. However, the 3D heterogeneity of drug molecules and the discretization of atomic distributions lead to spatial confusion in few-shot learning. Here, we propose a 3D graph neural network with few-shot learning, Meta3D-DDI, to predict DDI events in cold start scenario. The 3DGNN ensures rotation and translation invariance by calculating atomic pairwise distances, and incorporates 3D structure and distance information in the information aggregation stage. The continuous filter interaction module can continuously simulate the filter to obtain the interaction between the target atom and other atoms. Meta3D-DDI further develops a FSL strategy based on bilevel optimization to transfer meta-knowledge for DDI prediction tasks from existing drugs to new drugs. In addition, the existing cold start setting may cause the scaffold structure information in the training set to leak into the test set. We design scaffold-based cold start scenario to ensure that the drug scaffolds in the training set and test set do not overlap. The extensive experiments demonstrate that our architecture achieves the SOTA performance for DDI prediction under scaffold-based cold start scenario on two real-world datasets. The visual experiment shows that Meta3D-DDI significantly improves the learning for DDI prediction of new drugs. We also demonstrate how Meta3D-DDI can reduce the amount of data required to make meaningful DDI predictions.

Adversarial Auto-encoder Domain Adaptation for Cold-start Recommendation with Positive and Negative Hypergraphs

This article presents a novel model named Adversarial Auto-encoder Domain Adaptation to handle the recommendation problem under cold-start settings. Specifically, we divide the hypergraph into two hypergraphs, i.e., a positive hypergraph and a negative one. Below, we adopt the cold-start user recommendation for illustration. After achieving positive and negative hypergraphs, we apply hypergraph auto-encoders to them to obtain positive and negative embeddings of warm users and items. Additionally, we employ a multi-layer perceptron to get warm and cold-start user embeddings called regular embeddings. Subsequently, for warm users, we assign positive and negative pseudo-labels to their positive and negative embeddings, respectively, and treat their positive and regular embeddings as the source and target domain data, respectively. Then, we develop a matching discriminator to jointly minimize the classification loss of the positive and negative warm user embeddings and the distribution gap between the positive and regular warm user embeddings. In this way, warm users’ positive and regular embeddings are connected. Since the positive hypergraph maintains the relations between positive warm user and item embeddings, and the regular warm and cold-start user embeddings follow a similar distribution, the regular cold-start user embedding and positive item embedding are bridged to discover their relationship. The proposed model can be easily extended to handle the cold-start item recommendation by changing inputs. We perform extensive experiments on real-world datasets for both cold-start user and cold-start item recommendations. Promising results in terms of precision, recall, normalized discounted cumulative gain, and hit rate verify the effectiveness of the proposed method.

Cold-start active learning for image classification

Active learning (AL) aims to select valuable samples for labeling from an unlabeled sample pool to build a training dataset with minimal annotation cost. Traditional methods always require partially and initially labeled samples to start active selection and then query annotations of samples incrementally through several iterations. However, this scheme is not effective in the deep learning scenario. On the one hand, initially labeled sample sets are not always available in the beginning. On the other hand, the performance of the traditional model is usually poor in the early iterations due to limited training feedback. For the first time, we propose a cold-start AL model based on representative (CALR) sampling, which selects valuable samples without the need for an initial labeled set or the iterative feedback of the target models. Experiments on three image classification datasets, CIFAR-10, CIFAR-100 and Caltech-256, showed that CALR achieved a new state-of-the-art performance of AL in cold-start settings. Especially in low annotation budget conditions, our method can achieve up to a 10% performance increase compared to traditional methods. Furthermore, CALR can be combined with warm-start methods to improve the start-up efficiency while further breaking the performance ceiling of AL, which makes CALR have a broader application scenario.

MetaCAR: Cross-Domain Meta-Augmentation for Content-Aware Recommendation

Cold-start has become critical for recommendations, especially for sparse user-item interactions. Recent approaches based on meta-learning succeed in alleviating the issue, owing to the fact that these methods have strong generalization, so they can fast adapt to new tasks under cold-start settings. However, these meta-learning-based recommendation models learned with single and spase ratings are easily falling into the meta-overfitting, since the one and only rating <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$r_{ui}$</tex-math></inline-formula> to a specific item <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$i$</tex-math></inline-formula> cannot reflect a user's diverse interests under various circumstances(e.g., time, mood, age, etc), i.e. if <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$r_{ui}$</tex-math></inline-formula> equals to 1 in the historical dataset, but <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$r_{ui}$</tex-math></inline-formula> could be 0 in some circumstance. In meta-learning, tasks with these single ratings are called Non-Mutually-Exclusive(Non-ME) tasks, and tasks with diverse ratings are called Mutually-Exclusive(ME) tasks. Fortunately, a meta-augmentation technique is proposed to relief the meta-overfitting for meta-learning methods by transferring Non-ME tasks into ME tasks by adding noises to labels without changing inputs. Motivated by the meta-augmentation method, in this paper, we propose a cross-domain meta-augmentation technique for content-aware recommendation systems (MetaCAR) to construct ME tasks in the recommendation scenario. Our proposed method consists of two stages: meta-augmentation and meta-learning. In the meta-augmentation stage, we first conduct domain adaptation by a dual conditional variational autoencoder (CVAE) with a multi-view information bottleneck constraint, and then apply the learned CVAE to generate ratings for users in the target domain. In the meta-learning stage, we introduce both the true and generated ratings to construct ME tasks that enables the meta-learning recommendations to avoid meta-overfitting. Experiments evaluated in real-world datasets show the significant superiority of MetaCAR for coping with the cold-start user issue over competing baselines including cross-domain, content-aware, and meta-learning-based recommendations.

Personality-Aware Product Recommendation System Based on User Interests Mining and Metapath Discovery

A recommendation system is an integral part of any modern online shopping or social network platform. The product recommendation system as a typical example of the legacy recommendation systems suffers from two major drawbacks: recommendation redundancy and unpredictability concerning new items (cold start). These limitations take place because the legacy recommendation systems rely only on the user's previous buying behavior to recommend new items. Incorporating the user's social features, such as personality traits and topical interest, might help alleviate the cold start and remove recommendation redundancy. Therefore, in this article, we propose Meta-Interest, a personality-aware product recommendation system based on user interest mining and metapath discovery. Meta-Interest predicts the user's interest and the items associated with these interests, even if the user's history does not contain these items or similar ones. This is done by analyzing the user's topical interests and, eventually, recommending the items associated with the user's interest. The proposed system is personality-aware from two aspects; it incorporates the user's personality traits to predict his/her topics of interest and to match the user's personality facets with the associated items. The proposed system was compared against recent recommendation methods, such as deep-learning-based recommendation system and session-based recommendation systems. Experimental results show that the proposed method can increase the precision and recall of the recommendation system, especially in cold-start settings.

Simple and effective neural-free soft-cluster embeddings for item cold-start recommendations

Recommender systems are widely used in online platforms for easy exploration of personalized content. The best available recommendation algorithms are based on using the observed preference information among collaborating entities. A significant challenge in recommender system continues to be item cold-start recommendation: how to effectively recommend items with no observed or past preference information. Here we propose a two-stage algorithm based on soft clustering to provide an efficient solution to this problem. The crux of our approach lies in representing the items as soft-cluster embeddings in the space spanned by the side-information associated with the items. Though many item embedding approaches have been proposed for item cold-start recommendations in the past—and simple as they might appear—to the best of our knowledge, the approach based on soft-cluster embeddings has not been proposed in the research literature. Our experimental results on four benchmark datasets conclusively demonstrate that the proposed algorithm makes accurate recommendations in item cold-start settings compared to the state-of-the-art algorithms according to commonly used ranking metrics like Normalized Discounted Cumulative Gain (NDCG) and Mean Average Precision (MAP). The performance of our proposed algorithm on the MovieLens 20M dataset clearly demonstrates the scalability aspect of our algorithm compared to other popular algorithms. We also propose the metric Cold Items Precision (CIP) to quantify the ability of a system to recommend cold-start items. CIP can be used in conjunction with relevance ranking metrics like NDCG and MAP to measure the effectiveness of the cold-start recommendation algorithm.

Social Boosted Recommendation With Folded Bipartite Network Embedding

With the prevalence of online social platforms, social recommendation has emerged as a promising direction that leverages the social network among users to enhance recommendation performance. However, the available social relations among users are usually extremely sparse and noisy, which may lead to inferior recommendation performance. To alleviate this problem, this paper novelly exploits the implicit higher-order social influence and dependencies among users to enhance social recommendation. In this paper, we propose a novel embedding method for general bipartite graphs, which defines inter-class message passing between explicit relations and intra-class message passing between implicit higher-order relations via a novel sequential modelling paradigm. Inspired by recent advances in self-attention-based sequential modelling, the proposed model features a self-attentive representation learning mechanism for implicit user-user relations. Moreover, this paper also explores the inductive embedding learning for social recommendation problems to improve the recommendation performance in cold-start settings. The proposed inductive learning paradigm for social recommendation enables embedding inference for those cold-start users and items (unseen during training) as long as they are linked to existing nodes in the original network. Extensive experiments on real-world datasets demonstrate the superiority of our method and suggest that higher-order implicit relationship among users is beneficial to improving social recommendation.

ML2E: Meta-Learning Embedding Ensemble for Cold-Start Recommendation

Cold-start problem has been recognized as the most crucial challenge in recommender systems. Many recommendation algorithms work well when lots of preference information is available but start to degrade in cold-start settings. Inspired by the spirit of meta-learning, we identified that the appeal of cold-start problems and the superiority of meta-learning are congruous where meta-learning aims to learn a model from a small set of labeled examples (users' consuming history), and then this model can be quickly generalized to new tasks (recommendation for new users or new items). Therefore, faced with the extreme cold-start scenario, we proposed a meta-learning embedding ensemble (ML2E) recommendation algorithm to forecast new users' preference and generate desirable initial embedding for new items. It's worth pointing out that the training process of ML2E only uses few first-order gradient information, and ML2E not only has the incremental mining ability for different mini-batches on the same task but also the generalization ability for different tasks. Finally, we validated ML2E on two benchmark datasets, experimental results showed that our algorithm has significantly improved recommendation metric in comparison with three existing baselines.

Personalizing recommendation diversity based on user personality

In recent years, diversity has attracted increasing attention in the field of recommender systems because of its ability of catching users’ various interests by providing a set of dissimilar items. There are few endeavors to personalize the recommendation diversity being tailored to individual users’ diversity needs. However, they mainly depend on users’ behavior history such as ratings to customize diversity, which has two limitations: (1) They neglect taking into account a user’s needs that are inherently caused by some personal factors such as personality; (2) they fail to work well for new users who have little behavior history. In order to address these issues, this paper proposes a generalized, dynamic personality-based greedy re-ranking approach to generating the recommendation list. On one hand, personality is used to estimate each user’s diversity preference. On the other hand, personality is leveraged to alleviate the cold-start problem of collaborative filtering recommendations. The experimental results demonstrate that our approach significantly outperforms related methods (including both non-diversity-oriented and diversity-oriented methods) in terms of metrics measuring recommendation accuracy and personalized diversity degree, especially in the cold-start setting.

OPML: A one-pass closed-form solution for online metric learning

To achieve a low computational cost when performing online metric learning for large-scale data, we present a one-pass closed-form solution namely OPML in this paper. Typically, the proposed OPML first adopts a one-pass triplet construction strategy, which aims to use only a very small number of triplets to approximate the representation ability of whole original triplets obtained by batch-manner methods. Then, OPML employs a closed-form solution to update the metric for new coming samples, which leads to a low space (i.e., O(d)) and time (i.e., O(d2)) complexity, where d is the feature dimensionality. In addition, an extension of OPML (namely COPML) is further proposed to enhance the robustness when in real case the first several samples come from the same class (i.e., cold start problem). In the experiments, we have systematically evaluated our methods (OPML and COPML) on three typical tasks, including UCI data classification, face verification, and abnormal event detection in videos, which aims to fully evaluate the proposed methods on different sample number, different feature dimensionalities and different feature extraction ways (i.e., hand-crafted and deeply-learned). The results show that OPML and COPML can obtain the promising performance with a very low computational cost. Also, the effectiveness of COPML under the cold start setting is experimentally verified.

A dual-perspective latent factor model for group-aware social event recommendation

Event-based social networks (EBSNs) have experienced increased popularity and rapid growth. Due to the huge volume of events available in EBSNs, event recommendation becomes essential for users to find suitable events to attend. Different from classic recommendation scenarios (e.g., movies and books), a large majority of EBSN users join groups unified by a common interest, and events are organized by groups. In this paper, we propose a dual-perspective latent factor model for group-aware event recommendation by using two kinds of latent factors to model the dual effect of groups: one from the user-oriented perspective (e.g., topics of interest) and another from the event-oriented perspective (e.g., event planning and organization). Pairwise learning is used to exploit unobserved RSVPs by modeling the individual probability of preference via Logistic and Probit sigmoid functions. These latent group factors alleviate the cold-start problems, which are pervasive in event recommendation because events published in EBSNs are always in the future and many of them have little or no trace of historical attendance. The proposed model is flexible and we further incorporate additional contextual information such as event venue, event popularity, temporal influence and geographical distance. We conduct a comprehensive set of experiments on four datasets from Meetup in both regular and cold-start settings. The results demonstrate that the proposed approach yields substantial improvement over the state-of-the-art baselines by utilizing the dual latent factors of groups.

Factorizing LambdaMART for cold start recommendations

Recommendation systems often rely on point-wise loss metrics such as the mean squared error. However, in real recommendation settings only few items are presented to a user. This observation has recently encouraged the use of rank-based metrics. LambdaMART is the state-of-the-art algorithm in learning to rank which relies on such a metric. Motivated by the fact that very often the users' and items' descriptions as well as the preference behavior can be well summarized by a small number of hidden factors, we propose a novel algorithm, LambdaMART matrix factorization (LambdaMART-MF), that learns latent representations of users and items using gradient boosted trees. The algorithm factorizes LambdaMART by defining relevance scores as the inner product of the learned representations of the users and items. We regularise the learned latent representations so that they reflect the user and item manifolds as these are defined by their original feature based descriptors and the preference behavior. We also propose to use a weighted variant of NDCG to reduce the penalty for similar items with large rating discrepancy. We experiment on two very different recommendation datasets, meta-mining and movies-users, and evaluate the performance of LambdaMART-MF, with and without regularization, in the cold start setting as well as in the simpler matrix completion setting. The experiments show that the factorization of LambdaMart brings significant performance improvements both in the cold start and the matrix completion settings. The incorporation of regularisation seems to have a smaller performance impact.

Location-aware online learning for top-k recommendation

We address the problem of recommending highly volatile items for users, both with potentially ambiguous location that may change in time. The three main ingredients of our method include (1) using online machine learning for the highly volatile items; (2) learning the personalized importance of hierarchical geolocation (for example, town, region, country, continent); finally (3) modeling temporal relevance by counting recent items with an exponential decay in recency.For (1), we consider a time-aware setting, where evaluation is cumbersome by traditional measures since we have different top recommendations at different times. We describe a time-aware framework based on individual item discounted gain. For (2), we observe that trends and geolocation turns out to be more important than personalized user preferences: user–item and content-item matrix factorization improves in combination with our geo-trend learning methods, but in itself, they are greatly inferior to our location based models. In fact, since our best performing methods are based on spatiotemporal data, they are applicable in the user cold start setting as well and perform even better than content based cold start methods. Finally for (3), we estimate the probability that the item will be viewed by its previous views to obtain a powerful model that combines item popularity and recency.To generate realistic data for measuring our new methods, we rely on Twitter messages with known GPS location and consider hashtags as items that we recommend the users to be included in their next message.

A Neighborhood-based Matrix Factorization Technique for Recommendation

The data sparsity and prediction quality are recognized as the key challenges in the existing recommender Systems. Most of the existing recommender systems depend on collaborating flitering (CF) method which mainly leverages the user-item rating matrix representing the relationship between users and items. However, the CF-based method sometimes fails to provide accurate information for predicting recommendations as there is an assumption that the relationship between attributes of items is independent and identically distributed. In real applications, there are often several kinds of coupling relationships or connections existed among users or items. In this paper, we incorporate the coupling relationship analysis to capture the under-discovered relationships between items and aim to make the ratings more reasonable. Next, we propose a neighborhood-based matrix factorization model, which considers both the explicit and implicit correlations between items, to suggest the more reasonable items to user. The experimental evaluations demonstrate that the proposed algorithms outperform the state-of-the-art algorithms in the warm- and cold-start settings.

Combining content with user preferences for non-fiction multimedia recommendation: a study on TED lectures

This paper introduces a new dataset and compares several methods for the recommendation of non-fiction audio visual material, namely lectures from the TED website. The TED dataset contains 1,149 talks and 69,023 profiles of users, who have made more than 100,000 ratings and 200,000 comments. The corresponding metadata, which we make available, can be used for training and testing generic or personalized recommender systems. We define content-based, collaborative, and combined recommendation methods for TED lectures and use cross-validation to select the best parameters of keyword-based (TFIDF) and semantic vector space-based methods (LSI, LDA, RP, and ESA). We compare these methods on a personalized recommendation task in two settings, a cold-start and a non-cold-start one. In the cold-start setting, semantic vector spaces perform better than keywords. In the non-cold-start setting, where collaborative information can be exploited, content-based methods are outperformed by collaborative filtering ones, but the proposed combined method shows acceptable performances, and can be used in both settings. For the generic recommendation task, LSI and RP again outperform TF-IDF.

Bayesian Probabilistic Matrix Factorization with Social Relations and Item Contents for recommendation

Recommendation systems have received great attention for their commercial value in today's online business world. However, most recommendation systems encounter the data sparsity problem and the cold-start problem. To improve recommendation accuracy in this circumstance, additional sources of information about the users and items should be incorporated in recommendation systems. In this paper, we modify the model in Bayesian Probabilistic Matrix Factorization, and propose two recommendation approaches fusing social relations and item contents with user ratings in a novel way. The proposed approach is computationally efficient and can be applied to trust-aware or content-aware recommendation systems with very large dataset. Experimental results on three real world datasets show that our method gets more accurate recommendation results with faster converging speed than other matrix factorization based methods. We also verify our method in cold-start settings, and our method gets more accurate recommendation results than the compared approaches.