Sparsity Problem Research Articles

Pre-Training Across Different Cities for Next POI Recommendation

The Point-of-Interest (POI) transition behaviors could hold absolute sparsity and relative sparsity very differently for different cities. Hence, it is intuitive to transfer knowledge across cities to alleviate those data sparsity and imbalance problems for next POI recommendation. Recently, pre-training over a large-scale dataset has achieved great success in many relevant fields, like computer vision and natural language processing. By devising various self-supervised objectives, pre-training models can produce more robust representations for downstream tasks. However, it is not trivial to directly adopt such existing pre-training techniques for next POI recommendation, due to the lacking of common semantic objects (users or items) across different cities . Thus in this paper, we tackle such a new research problem of pre-training across different cities for next POI recommendation. Specifically, to overcome the key challenge that different cities do not share any common object, we propose a novel pre-training model named CATUS , by transferring the cat egory-level u niversal tran s ition knowledge over different cities. Firstly, we build two self-supervised objectives in CATUS : next category prediction and next POI prediction , to obtain the universal transition-knowledge across different cities and POIs. Then, we design a category-transition oriented sampler on the data level and an implicit and explicit transfer strategy on the encoder level to enhance this transfer process. At the fine-tuning stage, we propose a distance oriented sampler to better align the POI representations into the local context of each city. Extensive experiments on two large datasets consisting of four cities demonstrate the superiority of our proposed CATUS over the state-of-the-art alternatives. The code and datasets are available at https://github.com/NLPWM-WHU/CATUS.

WES-BTM: A Short Text-Based Topic Clustering Model

User comments often contain their most practical requirements. Using topic modeling of user comments, it is possible to classify and downscale text data, mine the information in user comments, and understand users’ requirements and preferences. However, user comment texts are usually short and lack rich word frequency and contextual information with sparsity. The traditional topic model cannot model and analyze these short texts well. The biterm topic model (BTM), while solving the sparsity problem, suffers from accuracy and noise problems. In order to eliminate information barriers and further ensure information symmetry, a new topic clustering model, termed the word-embedding similarity-based BTM (WES-BTM), is proposed in this paper. The WES-BTM builds on the BTM by converting word pairs into word vectors and calculating their similarity to perform word pair filtering, which in turn improves clustering accuracy. Based on the experimental results using actual data, the WES-BTM outperforms the BTM, LDA, and NMF models in terms of topic coherence, perplexity, and Jensen–Shannon divergence. It is verified that the WES-BTM can effectively reduce noise and improve the quality of topic clustering. In this way, the information in user comments can be better mined.

A Recommendation System Involving a Hybrid Approach of Student Review and Rating for an Educational Video

Video recommendation systems in e-learning platforms are a specific type of recommendation system that uses algorithms to suggest educational videos to students based on their interests and preferences. Student’s written feedback or reviews can provide more details about the educational video, including its strengths and weaknesses. In this paper, we build an education video recommender system based on learners’ reviews. we use LDA topic model on textual data extracted from educational videos to train language models as an input to supervised CNN model. Additionally, we used latent factor model to extract the educational videos' features and learner preferences from learners’ historical data (ratings and reviews) as an output CNN model. In our proposed technique, we use hybrid user ratings and reviews to tackle sparsity and cold start problem in the recommender system. Our recommender uses user review to suggest new recommended videos, but in case there is no review (empty cell in matrix factorization) or unclear comment then we will take user rating on that educational video. We worked on real-world big and diverse learning courses and video content datasets from Coursera. Results show that new prediction ratings from learners' reviews can be used to make good new recommendations about videos that have not been previously seen and reduce cold start and sparsity problem effects.

Do Trust based Social Recommendation Algorithms Work as Intended?

Recommender systems are powerful tools that filter and recommend content/information relevant to a given user. Collaborative filtering is the most popular technique used in building recommender systems and it has been successfully incorporated in many applications. These conventional recommendation systems require a minimum number of users, items, and ratings in order to provide effective recommendations. This results in the infamous cold-start problem where the system is not able to produce effective recommendations for new users. Recently, there has been an escalation in the popularity and usage of social networks, which persuades people to share their experiences in the form of reviews and ratings on social media. The components of social media such as the influence of friends, interests, and enjoyment create the opportunities to develop solutions for sparsity and cold start problems of recommendation systems. This paper aims to observe these patterns and analyze three of the existing social recommendation systems, SocialMF, SocialFD, and GraphRec. SocialMF and SocialFD algorithms are based on matrix factorization and distance metric learning respectively whereas GraphRec is an attention based deep learning model. Through extensive experimentation with the datasets that these algorithms were tested on and one new dataset, we compared the results based on evaluation metrics including Root Mean Squared Error (RMSE) and Mean Absolute Error (MAE). To investigate how trust impacts the performance of these models, we evaluated them by modifying the trust and social component. Experimental results show that there is no conclusive evidence that trust propagation plays a major part in these models. Moreover, these models show a slightly improved performance in the absence of trust statements.

BayesSentiRS: Bayesian sentiment analysis for addressing cold start and sparsity in ranking-based recommender systems

Recommendation systems are widely used to filter massive information. However, they often face the challenges of cold start and sparsity problems, limiting their effectiveness. Bayesian Personalized Ranking (BPR), which focuses on predicting the relative order of user items, has been conventionally proposed to address these challenges for ranking-based recommendation systems. However, conventional BPR approaches rely solely on implicit feedback and lack semantic and visual sentiment information. In this study, we propose a novel multi-modal BPR method that incorporates both semantic and visual sentiment to capture more nuanced user preferences and provide more accurate recommendations. Experimental evaluations show that the performance of our proposed method across seven metrics outperforms conventional BPR in both the review-rich and image-rich scenarios, indicating the potential and significance of considering sentiment for improving the performance of BPR recommendation systems.

A novel approach to enhance the quality of health care recommender system using fuzzy-genetic approach

In recent generations of the digital world medical data in Recommender Systems. Health Care Recommender System (HCRS) analyses the medical data and then predicts the user’s or patient’s illness. Nowadays, healthcare data is used by various users or patients in recommendation systems which are useful for everyone. Analysing and predicting medical data provides awareness to users and these data predictions may be enriched using various techniques of RS. Machine learning techniques are used to make sure that health data is reliable and of high quality. In every RS the issues are targeted such as scalability, sparsity and cold start problems. In many social networking applications, these issues are resolved using ML algorithms. However, there is a significant gap between IT systems and medical diagnosis. The fuzzy genetic method is used in HCRS in order to bridge the gap between IT and healthcare applications. Through the use of the mutation and crossover operators, a real-value genetic method is used in this to compute similarity. With the user’s extra personalized information, fuzzy rules are later generated for the database. The Hybrid fuzzy-genetic method, also known as this situation, combines both techniques to improve recommendation quality. Utilizing this method will improve the quality of the recommendation process by discovering the most precise similarity measures among different users. Six factors are subjected to fuzzification, including age, gender, employment, height, weight, and region. Genre-interesting measure weights are then used, including Very Light, Light, Average, Heavy, and Very Heavy. Finally, the evaluation metrics used MAE and RMSE to evaluate the recommendation accuracy which showed the best results in comparison with baseline approaches such as Convolutional Neural Networks and Restricted Boltzman Machine.

Short‐term origin‐destination demand forecasting in bus rapid transit based on dual attentive multi‐scale convolutional network

AbstractThe short‐term origin‐destination (OD) forecast in bus rapid transit (BRT) can provide future OD demand, which means that transport plan personnel can take measures in advance, such as adjusting the departure schedule, to avoid congestion. This emphasizes the importance of accurate and stable OD forecasting in real‐time. Deep learning methods have been widely applied due to their superior performance in capturing dynamic spatiotemporal and nonlinear features. However, current methods based on deep learning and data‐driven cannot simultaneously address the problems of data availability and sparsity, and their ability to capture dynamic spatiotemporal patterns is insufficient, which cannot provide satisfactory accuracy and stability. To address these problems, this paper proposes a method for forecasting the OD demand in BRT based on dual attention multi‐scale convolutional network (DAMSCN). Considering the three temporal properties of OD, that is, closeness, period and trend dependencies. DAMSCN uses multi‐scale convolution (MSC), dual attention mechanism (DAt) and stacked convolutional long short‐term memory networks (ConvLSTM) to model the three kinds of properties. The MSC module converts sparse OD information into dense useful features to solve the problem of data sparsity. The DAt and ConvLSTM module model the dependencies relationship in the spatial and temporal dimensions, which can improve the spatiotemporal features extraction ability. DAMSCN also combine inflow information to enhance data availability. The three properties are dynamically aggregated and further integrated with external influences (e.g. weather conditions etc.) to obtain the final forecasting result. The accuracy and stability are evaluated on the Xiamen BRT (XMBRT) and Shanghai Metro datasets (SHMetro). The experimental results show that the DAMSCN is superior to other method. Compared to the best baseline, on the XMBRT, MAE decreased by 8.60%, and RMSE decreased by 4.81%. On the SHMetro, MAE decreased by 7.22%, and RMSE decreased by 8.96%.

Predicting Student Performance in Future Exams via Neutrosophic Cognitive Diagnosis in Personalized E-Learning Environment

To provide intelligent learning guidance for students in e-learning systems, it is necessary to accurately predict their performance in future exams by analyzing score data in past exams. However, existing research has not addressed the uncertain and dynamic features of students' cognitive status, whereas these features are essential for improving the accuracy of performance prediction. A novel approach via neutrosophic cognitive diagnosis is proposed to predict student performance in a personalized e-learning environment. This approach innovatively employs the neutrosophic set (NS) theory to comprehensively measure the students' cognitive status on knowledge concepts from three features, i.e., understanding level, degree of misunderstanding, and uncertainty. A new measurement method of NS similarity is used to identify top- <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">k</i> neighboring students for one student. Then, we predict a student's performance on exercises in future exams by integrating a neighborhood-based collaborative filtering algorithm with a probabilistic matrix factorization method to address the data sparsity problem. Finally, the experiments based on real-world datasets demonstrate that the proposed approach provides higher prediction accuracy than the existing ones with a low execution cost.

TAG: Joint Triple-Hierarchical Attention and GCN for Review-Based Social Recommender System

Recommender systems across many Internet services have become a critical part of online businesses, as consumers would refer to them before making decisions. However, the lack of explicit ratings for items on many services makes it challenging to capture user preferences and item characteristics. Both academia and the industry have drawn attention to rating predications as a fundamental problem in recommendation systems. With the emergence of social networks, social recommender systems have been proposed to utilize the relationship between users and items to alleviate the data sparsity problem for rating predictions. However, they either concentrate on the opinion mining for each user and item, or consider the connections between users only. In this paper, we present an effective framework, Triple-hierarchical Attention Graph-based social rating prediction (TAG), to exploit the social relationships between users, the user-item interest relationships, the correlation relationships between items, and reviews for rating predictions. In order to consider opinions from reviews and these complex relationships, we first employ two triple-hierarchical attention to extract user and item features from reviews. We then design an inductive GNN, which generates effective embedding for users and items. Experiments over Yelp show that TAG outperforms state-of-the-art methods across RMSE, MAE, and NDCG metrics.

An improved heterogeneous graph convolutional network for job recommendation

Job recommendation is crucial in online recruitment platforms due to the overwhelming number of job postings. Job seekers spend considerable time and effort searching for suitable employment. With millions of job seekers browsing job postings daily, the demand for accurate and effective job recommendations is more pressing than ever. To address this challenge, we propose IHGCN, an improved semi-supervised heterogeneous graph convolutional network model for job recommendation. IHGCN aims to provide job recommendations for early job seekers based on their resumes. Firstly, we introduce a novel labeling classification standard specifically tailored to early job seeker resumes. Secondly, we construct a heterogeneous resume graph where each resume is represented as a node. Job recommendation is treated as a multi-classification problem. Thirdly, our IHGCN model learns a node representation from the graph to perform effective job recommendations. To evaluate our model, we conduct experiments using a real-world resume dataset obtained from LinkedIn. The results demonstrate that IHGCN outperforms the baselines by around 10%. This study highlights the benefits of leveraging meta-paths within the Graph Convolutional Network model to address the sparsity problem caused by the one-hot representation of nodes.

Enhanced Collaborative Filtering Algorithm for Movies Recommendation using Big Five Personality Traits

Recommender System suffers from data sparsity and cold start problems which arises when there is no sufficient rating history for user who has recently log into the system and no proper recommendations can be made. This paper develops an Enhanced collaborative filtering algorithm for Movie recommender system Using genre and Big five Personality traits (EMUBP) as system’s input. The experimental result shows that the EMUBP system improved recommendation quality and accuracy by 8.33% compared with the existing state-of-the-art using precision, recall and MAE.

A multi-modal vehicle trajectory prediction framework via conditional diffusion model: A coarse-to-fine approach

Accurate prediction of the future motion of surrounding vehicles is crucial for ensuring the safety of motion planning in autonomous vehicles. However, it is challenging to perform because of the complex interactions between vehicles. In this study, we propose a novel multi-modal vehicle trajectory prediction framework that utilizes a coarse-to-fine approach by first generating initial trajectory proposals with a conditional variational autoencoder (CVAE) and then refining them using the conditional diffusion model. We first address the problems of data sparsity and irregularity by converting the trajectory coordinates to the Frenet coordinate system. To enable the model to better distinguish between different features, we employ a temporal encoder to extract trajectory features and a long short-term memory (LSTM) network to extract lane features. The target lane evaluator is utilized to calculate the attention weights for each lane candidate, thereby generating more deterministic future trajectories. We then use the CVAE to generate initial multiple trajectory proposals based on the surrounding scene context and the trajectory features of the target vehicle. Finally, we formulate the trajectory refinement task as a reverse process of the conditional diffusion model, which effectively enhances the multi-modal trajectory proposals. Experiments conducted on Argoverse and nuScenes demonstrate that our method outperforms state-of-the-art methods in some evaluation metrics while achieving the optimal trade-off between predictive accuracy and efficiency. Field experiments conducted in urban environments further validate the effectiveness of the proposed approach.

A survey of topic models: From a whole-cycle perspective

With the rapid development of information science and social networks, the Internet has accumulated various data containing valuable information and topics. The topic model has become one of the primary semantic modeling and classification methods. It has been widely studied in academia and industry. However, most topic models only focus on long texts and often suffer from semantic sparsity problems. The sparse, short text content and irregular data have brought major challenges to the application of topic models in semantic modeling and topic discovery. To overcome these challenges, researchers have explored topic models and achieved excellent results. However, most of the current topic models are applicable to a specific model task. The majority of current reviews ignore the whole-cycle perspective and framework. It brings great challenges for novices to learn topic models. To deal with the above challenges, we investigate more than a hundred papers on topic models and summarize the research progress on the entire topic model process, including theory, method, datasets, and evaluation indicator. In addition, we also analyzed the statistical data results of the topic model through experiments and introduced its applications in different fields. The paper provides a whole-cycle learning path for novices. It encourages researchers to give more attention to the topic model algorithm and the theory itself without paying extra attention to understanding the relevant datasets, evaluation methods and latest progress.

Weighted automata extraction and explanation of recurrent neural networks for natural language tasks

Recurrent Neural Networks (RNNs) have achieved tremendous success in processing sequential data, yet understanding and analyzing their behaviours remains a significant challenge. To this end, many efforts have been made to extract finite automata from RNNs, which are more amenable for analysis and explanation. However, existing approaches like exact learning and compositional approaches for model extraction have limitations in either scalability or precision. In this paper, we propose a novel framework of Weighted Finite Automata (WFA) extraction and explanation to tackle the limitations for natural language tasks. First, to address the transition sparsity and context loss problems we identified in WFA extraction for natural language tasks, we propose an empirical method to complement missing rules in the transition diagram, and adjust transition matrices to enhance the context-awareness of the WFA. We also propose two data augmentation tactics to track more dynamic behaviours of RNN, which further allows us to improve the extraction precision. Based on the extracted model, we propose an explanation method for RNNs including a word embedding method – Transition Matrix Embeddings (TME) and TME-based task oriented explanation for the target RNN. Our evaluation demonstrates the advantage of our method in extraction precision than existing approaches, and the effectiveness of TME-based explanation method in applications to pretraining and adversarial example generation.

Feature augmentation and semi-supervised conditional transfer learning for early detection of sepsis

Early detection of Sepsis is crucial for improving patient outcomes, as it is a significant public health concern that results in substantial morbidity and mortality. However, despite the widespread use of the Sequential Organ Failure Assessment (SOFA) in clinical settings to identify sepsis, obtaining sufficient physiological data before onset remains challenging, limiting early detection of sepsis. To address this challenge, we propose an interpretable machine learning model, ITFG (Interpretable Tree-based Feature Generation), that leverages potential correlations between features based on existing knowledge to identify sepsis within six hours of onset using valuable and continuous physiological measures. Furthermore, we introduce a Semi-supervised Attention-based Conditional Transfer Learning (SAC-TL) framework to enhance the model’s generality and enable it to be used for early warning of sepsis in the target domain with less information from the source domain. Our proposed approaches effectively address the problem of systematic feature sparsity and missing data, while also being practical for different degrees of generalizability. We evaluated our proposed approaches on open datasets, MIMIC and PhysioNet, obtaining AUC of 97.98% and 86.21%, respectively, demonstrating their effectiveness in different data environments and achieving the best early detection results.

Data Augmented Sequential Recommendation Based on Counterfactual Thinking

Sequential recommendation has recently attracted increasing attention from the industry and academic communities. While previous models have achieved remarkable successes, an important problem may still hinder their performances, that is, the sparsity of the real-world data. In this paper, we propose a novel counterfactual data augmentation framework to alleviate the problem of data sparsity. In specific, our framework contains a sampler model and an anchor model. The sampler model aims to generate high-quality user behavior sequences, while the anchor model is trained based on the original and new generated samples, and leveraged to provide the final recommendation list. To implement the sampler model, we firstly design four types of heuristic methods based on either random or frequency-based strategies. And then, to improve the quality of the generated sequences, we propose two learning-based samplers by discovering the decision boundaries or increasing the sample informativeness. At last, we build an RL based model to automatically determine where to edit the history behaviors and how many items should be replaced. Considering that the sampler model can be imperfect, we, at last, analyze the influence of the noisy information contained in the generated sequences on the anchor model in theory, and design a simple but effective method to better serve the anchor model. We conduct extensive experiments to demonstrate the effectiveness of our model.

Combining Graph Contrastive Embedding and Multi-head Cross-Attention Transfer for Cross-Domain Recommendation

Cross-domain recommendation (CDR) has become an important research direction in the field of recommender systems due to the increasing demand for personalized recommendations across different domains. However, CDR faces multiple challenges, including data sparsity, popularity bias, and long-tail problems. To address these challenges, we propose a novel framework that combines graph contrastive embedding and multi-head cross-attention transfer for cross-domain recommendation, called GCE-MCAT. Specifically, in the pre-training process, we generate more uniform user and item embeddings through contrastive learning, effectively solving the problem of inconsistent data embedding space distribution and recommendation popularity bias. Moreover, we propose a multi-head cross-attention transfer mechanism that allows the model to extract user common and specific domain features from multiple perspectives and perform cross-domain bidirectional knowledge transfer. Finally, we propose a cross-domain feature fusion mechanism that dynamically assigns weights to common user features and specific domain features. This enables the model to more effectively learn common user interests. We evaluate the proposed framework on three real-world CDR datasets and show that GCE-MCAT consistently and significantly improves recommendation performance compared to state-of-the-art methods. In particular, the proposed framework has demonstrated remarkable effectiveness in addressing long-tail distribution and enhancing recommendation novelty, providing users with more diversified recommendations and reducing popularity bias.

Improving Collaborative Filter Using BERT

With the increasing number of books published and the difficulty of obtaining appropriate research attention, the recommendation systems can increase the affordability and availability of these books. In this work, we expand our work to enhance the accuracy of book collaborative filtering by applying semantic similarity to book summaries, in addition to that addressing major problems of the current work by applying effective techniques to handle the scalability and sparsity problems. The proposed approach consists of three stages: preprocessing, building the system, and evaluation. The technologies used in the pre-processing stage included reduction and normalization. The construction system is divided into two phases: semantic similarity and recommendation. The semantic similarity is done by using BERT for sentence embedding and cosine similarity to calculate the similarity between sentences. During the recommendation phase by using CF based on KNN. In the evaluation stage, classification accuracy metrics had been used. The proposed approach improved the accuracy of the book recommendation system and increased the accuracy to 0.89 compared to previous works on a dataset of 271,000 book summaries. The proposed approach yielded better results due to avoiding problems in previous work, such as scalability and sparsity, by using BERT with CF based KNN. Filtering the data using BERT and the KNN algorithm in the CF added strength to the recommendation, which led to an increase in the accuracy rate.

SSTP: Social and Spatial-Temporal Aware Next Point-of-Interest Recommendation

The expansion of available information in location-based social networks (LBSNs) has led to information overload, making it urgent to discover users’ next point-of-interest (POI). Some existing works only consider certain modal information in LBSNs and do not transform them into high-dimensional structures, which hinders the alleviation of the data sparsity problem. Moreover, many approaches rely solely on social relationships, making it difficult to recommend POIs to new users without association information. To tackle these challenges, we propose a social- and spatial–temporal-aware next point-of-Interest (SSTP) recommendation model. SSTP uses two feature encoders based on self-attention mechanism and gate recurrent unit to model users’ check-in enhancement sequence hierarchically. We also design a random neighborhood sampling approach to mine user social relationships, thus alleviating the user cold start problem. Finally, we propose a geographical-aware graph attention network to learn the sensitivity of users to distance. Extensive experiments on two real-world datasets show that SSTP outperforms state-of-the-art models, improving Hit@k by 2.26–6.55% and MAP@k by 3.49–6.55%. Moreover, SSTP has better performance on sparse data, with an average improvement of 6.09% on the Hit@k. The code can be downloaded at https://github.com/Rih0/sstp.

Mutual Domain Adaptation

To solve the label sparsity problem, domain adaptation has been well-established, suggesting various methods such as finding a common feature space of different domains using projection matrices or neural networks. Despite recent advances, domain adaptation is still limited and is not yet practical. The most pronouncing problem is that the existing approaches assume source-target relationship between domains, which implies one domain supplies label information to another domain. However, the amount of label is only marginal in real-world domains, so it is unrealistic to find source domains having sufficient labels. Motivated by this, we propose a method that allows domains to mutually share label information. The proposed method finds a projection matrix that matches the respective distributions of different domains, preserves their respective geometries, and aligns their respective class boundaries. The experiments on benchmark datasets show that the proposed method outperforms relevant baselines. In particular, the results on varying proportions of labels present that the fewer labels the better improvement.