A news service recommendation model for enterprise employees integrating behavioural analysis and sequential patterns

Combining item feature information helps extract comprehensive sequential patterns, thereby improving the accuracy of sequential recommendations. However, existing methods usually combine features of each item using a vanilla attention mechanism. We argue that such a combination ignores the interactions between features and does not model integrated feature representations. In this study, we propose a novel Feature Interaction Dual Self-attention network (FIDS) model for sequential recommendation, which utilizes dual self-attention to capture both feature interactions and sequential transition patterns. Specifically, we first model the feature interactions for each item to form meaningful higher-order feature representations using a multi-head attention mechanism. Then, we adopt two independent self-attention networks to capture the transition patterns in both the item sequence and the integrated feature sequence, respectively. Moreover, we stack multiple self-attention blocks and add residual connections at each block for all self-attention networks. Finally, we combine the feature-wise and item-wise sequential patterns into a fully connected layer for the next item recommendation. We conduct experiments on two real-world datasets, and our experimental results show that the proposed FIDS method outperforms state-of-the-art recommendation models.

Data mining for client behavior analysis has become increasingly important in business, however further analysis on transactions and sequential behaviors would be of even greater value, especially in the financial service industry, such as banking and insurance, government and so on. In a real-world business application of taxation debt collection, in order to understand the internal relationship between taxpayers' sequential behaviors (payment, lodgment and actions) and compliance to their debt, we need to find the contrast sequential behavior patterns between compliant and non-compliant taxpayers. Contrast Patterns (CP) are defined as the itemsets showing the difference/discrimination between two classes/datasets (Dong and Li, 1999). However, the existing CP mining methods which can only mine itemset patterns, are not suitable for mining sequential patterns, such as time-ordered transactions in taxpayer sequential behaviors. Little work has been conducted on Contrast Sequential Pattern (CSP) mining so far. Therefore, to address this issue, we develop a CSP mining approach, eCSP, by using an effective CSP-tree structure, which improves the PrefixSpan tree (Pei et al., 2001) for mining contrast patterns. We propose some heuristics and interestingness filtering criteria, and integrate them into the CSP-tree seamlessly to reduce the search space and to find business-interesting patterns as well. The performance of the proposed approach is evaluated on three real-world datasets. In addition, we use a case study to show how to implement the approach to analyse taxpayer behaviour. The results show a very promising performance and convincing business value.

On mining multi-time-interval sequential patterns

With the rapid development of Internet, online survey becomes an emerging industry. It is a very challenging task to get interesting knowledge from the large-scale behavioral data of respondents. This paper firstly makes reduction of user properties and behavior data from an online survey company, and based on which we construct an online survey user model, then, an improved generalized sequential pattern (GSP) algorithm is proposed to mine frequent sequential patterns, finally, we give an in-depth user behavior analysis of online survey, which is from conventional sequential patterns of user behavior, sequential patterns based on specific behavior and time window, and user behavior prediction. The experimental results show that it is effective to analyze the sequence of user behavior thorough improved GSP algorithm. Compared with the classical GSP algorithm, user behavior prediction accuracy rate increases 19% via our proposed sequential pattern analysis approach.

Top-$N$ sequential recommendation models each user as a sequence of items interacted in the past and aims to predict top-$N$ ranked items that a user will likely interact in a `near future'. The order of interaction implies that sequential patterns play an important role where more recent items in a sequence have a larger impact on the next item. In this paper, we propose a Convolutional Sequence Embedding Recommendation Model (\emph{Caser}) as a solution to address this requirement. The idea is to embed a sequence of recent items into an `image' in the time and latent spaces and learn sequential patterns as local features of the image using convolutional filters. This approach provides a unified and flexible network structure for capturing both general preferences and sequential patterns. The experiments on public datasets demonstrated that Caser consistently outperforms state-of-the-art sequential recommendation methods on a variety of common evaluation metrics.

Recommending a limited number of Point-of-Interests (POIs) a user will visit next has become increasingly important to both users and POI holders for Location-Based Social Networks (LBSNs). However, POI recommendation is a challenging task since complex sequential patterns and rich contexts are contained in extremely sparse user check-in data. Recent studies show that embedding techniques effectively incorporate POI contextual information to alleviate the data sparsity issue, and Recurrent Neural Network (RNN) has been successfully employed for sequential prediction. Nevertheless, existing POI recommendation approaches are still limited in capturing user personalized preference due to separate embedding learning or network modeling. To this end, we propose a novel unified spatio-temporal neural network framework, named PPR, which leverages users’ check-in records and social ties to recommend personalized POIs for querying users by joint embedding and sequential modeling. Specifically, PPR first learns user and POI representations by joint modeling User-POI relation, sequential patterns, geographical influence, and social ties in a heterogeneous graph and then models user personalized sequential patterns using the designed spatio-temporal neural network based on LSTM model for the personalized POI recommendation. Furthermore, we extend PPR to an end-to-end recommendation model by jointly learning node representations and modeling user personalized sequential preference. Extensive experiments on three real-world datasets demonstrate that our model significantly outperforms state-of-the-art baselines for successive POI recommendation in terms of Accuracy, Precision, Recall and NDCG. The source code is available at: https://www.anonymous.4open.science/r/DSE-1BEC.

Customers usually change their purchase interests in the short product life cycle of the e-commerce environment. Therefore, recent transaction patterns should have a greater effect on the customer preferences. From the seller's point of view, an e-commerce recommender system should focus on the profit of recommendation. This study proposes a new sequential pattern mining algorithm that incorporates the concepts of frequency, recency, and profit to discover frequent, recent, and profitable sequential patterns, called FRP-sequences. Based on the discovered sequential patterns, this study develops a collaborative recommender system to improve recommendation accuracy for customers and the profit of recommendation from the seller's perspective. The proposed recommender system clusters customers, discovers FRP-sequences for each cluster, and then recommends items to the target customers based on their frequent, recent, and profitable FRP-sequences. In the stage of discovering FRP-sequences, the transaction patterns near the current time period and profitable items are weighted more heavily to improve profit. This study uses a public food mart database to determine the performance of the proposed approach, and compares it with traditional recommendation models. The proposed system performs better than traditional recommendation models in both recommendation accuracy and profit.

Mining contrast sequential patterns, which are sequential patterns that characterize a given sequence class and distinguish that class from another given sequence class, has a wide range of applications including medical informatics, computational finance and consumer behavior analysis. In previous studies on contrast sequential pattern mining, each element in a sequence is a single item or symbol. This paper considers a more general case where each element in a sequence is a set of items. The associated contrast sequential patterns will be called itemset-based distinguishing sequential patterns (itemset-DSP). After discussing the challenges on mining itemset-DSP, we present iDSP-Miner, a mining method with various pruning techniques, for mining itemset-DSPs that satisfy given support and gap constraint. In this study, we also propose a concise border-like representation (with exclusive bounds) for sets of similar itemset-DSPs and use that representation to improve efficiency of our proposed algorithm. Our empirical study using both real data and synthetic data demonstrates that iDSP-Miner is effective and efficient.

Behavior analysis is a widely used non-invasive tool in the practical production routine, as the animal acts as a biosensor capable of reflecting its degree of adaptation and discomfort to some environmental challenge. Conventional statistics use occurrence data for behavioral evaluation and well-being estimation, disregarding the temporal sequence of events. The Generalized Sequential Pattern (GSP) algorithm is a data mining method that identifies recurrent sequences that exceed a user-specified support threshold, the potential of which has not yet been investigated for broiler chickens in enriched environments. Enrichment aims to increase environmental complexity with promising effects on animal welfare, stimulating priority behaviors and potentially reducing the deleterious effects of heat stress. The objective here was to validate the application of the GSP algorithm to identify temporal correlations between heat stress and the behavior of broiler chickens in enriched environments through a proof of concept. Video image collection was carried out automatically for 48 continuous hours, analyzing a continuous period of seven hours, from 12:00 PM to 6:00 PM, during two consecutive days of tests for chickens housed in enriched and non-enriched environments under comfort and stress temperatures. Chickens at the comfort temperature showed high motivation to perform the behaviors of preening (P), foraging (F), lying down (Ld), eating (E), and walking (W); the sequences <{Ld,P}>; <{Ld,F}>; <{P,F,P}>; <{Ld,P,F}>; and <{E,W,F}> were the only ones observed in both treatments. All other sequential patterns (comfort and stress) were distinct, suggesting that environmental enrichment alters the behavioral pattern of broiler chickens. Heat stress drastically reduced the sequential patterns found at the 20% threshold level in the tested environments. The behavior of lying laterally "Ll" is a strong indicator of heat stress in broilers and was only frequent in the non-enriched environment, which may suggest that environmental enrichment provides the animal with better opportunities to adapt to stress-inducing challenges, such as heat.

This full paper of the research-to-practice category addresses the problem of discovering patterns in students' learning paths in an introductory programming course. Student interactions in courses delivered through learning management systems (LMS) are stored in event logs. Each event corresponds to some action performed by students on course content. Process Mining (PM) techniques allow extracting knowledge from the event logs generating the trace, that is, the ordered set of events of a given student in a course. PM can also obtain the process models generated from the traces. Process models show where students started and ended their paths, enabling them to characterize possible behaviours. It shows that each student can behave differently, generating different sequences. Applying Sequential Pattern Mining (SPM) techniques make it possible to extract sequential patterns and discover sequences of similar events. Thus, this article aims to present the results of applying PM and SPM techniques to verify and analyze the students' learning paths in an introductory programming course. For this, a log of Moodle events was collected, which went through data selection and cleaning, and segmentation. The Moodle event log stores data that allows exploring information on three levels: (1) activity-type, (2) activity, and (3) action in the activity. Thus, the study aimed to investigate which activity-types, activities, and actions were performed and in what order. Among the applied techniques are those to obtain: event log statistics, process models with the Directly-Follows Graph algorithm and event sequences with the Generalized Sequential Patterns algorithm. The results showed that the students had distinct behaviors when accessing and performed the activities. Overall, we highlight the following two main findings that enabled a better understanding of the different behaviors: (i) verifying process models of Levels 2 and 3 attributes allows to deepen the analysis in the flow of the events of the activities; (ii) The subsequences of interest extracted via SPM can complement the analysis of the students' behaviors.

Next Point-of-Interest (POI) recommendation aims to predict the next POI for users from their historical activities. Existing methods typically rely on location-level POI check-in trajectories to explore user sequential transition patterns, which suffer from the severe check-in data sparsity issue. However, taking into account region-level and category-level POI sequences can help address this issue. Moreover, collaborative information between different granularities of POI sequences is not well utilized, which can facilitate mutual enhancement and benefit to augment user preference learning. To address these challenges, we propose multi-granularity contrastive learning (MGCL) for next POI recommendation, which utilizes multi-granularity representation and contrastive learning to improve the next POI recommendation performance. Specifically, location-level POI graph, category-level, and region-level sequences are first constructed. Then, we use graph convolutional networks on POI graph to extract cross-user sequential transition patterns. Furthermore, self-attention networks are used to learn individual user sequential transition patterns for each granularity level. To capture the collaborative signals between multi-granularity, we apply the contrastive learning approach. Finally, we jointly train the recommendation and contrastive learning tasks. Extensive experiments demonstrate that MGCL is more effective than state-of-the-art methods.

In e-commerce platforms, a user's next behavior will be affected by his long-term constant interests and short-term temporal needs. Such information is usually hidden in the users' historical online behavior data, so how to capture long-term and short-term patterns becomes the key to design better recommendation models or algorithms. Current mainstream methods such as Markov chain, convolutional neural network, and recurrent neural network cannot well express the mixed dynamic characteristics. In this paper, we propose an attention-based deep neural network (ADNNet) to solve the problem. In ADNNet, a convolutional neural network is used to extract the short-term patterns in the behavior sequences, and a gated recurrent unit is used to mine the long-term patterns in the behavior sequences. The attention mechanism is adopted to help the network automatically learn the best fusion coefficient of these two patterns. Our experimental result on four real public datasets (+0.69% in Hit Ratio and +3.49% in MRR) shows the superiority of our proposed ADNNet compared with other state-of-the-art methods.

Nowadays, due to escalating the demands of the direct purchase of e-commerce with many kinds of item, the demand for e-commerce is exploding. The recommender system finds items for customer easily and targets in customers for the e-commerce firms easily by an automated recommending process. And also, association rules are often used to facilitate product sales in marketing pattern analysis through recommender systems in e-commerce. This paper takes aim at a new recommending service in e-commerce using periodicity analysis of weighted sequential pattern. In e-commerce, we need the FRAT segmentation method based on the various items purchased by the customer, to reflect the weights and perform the pre-processing tasks using sequential patterns for period analysis. We apply an effective incremental sequential data mining method that adds incremental purchasing data as a change of the four season changes. As a result, we suggest a recommendation service for micro-marketing of e-commerce based on big data analysis for promoting e-commerce purchasing, which satisfies customer’s taste with various products of e-commerce. To check the performance of the proposal, we tested the data set with the same conditions as before. As a result, the proposed system is more efficient than the other systems in the results of system evaluation.

Hybrid collaborative filtering model for consumer dynamic service recommendation based on mobile cloud information system

Recommender systems play an important role in online platforms to provide personalized high-quality service for users. Sequential Recommendation (SR) captures users' dynamic preferences by modeling historical sequences, but most SR models suffer performance degradation due to data sparsity problem. To address this issue, we focus on Cross-Domain Sequential Recommendation (CDSR) in this paper, which aims to transfer sequential patterns across domains to promote accuracy of single-domain recommendation services. Challenges arise when tackling CDSR, i.e., (1) how to extract intra- and inter-sequence collaborations within domains, (2) how to transfer stable interaction patterns across domains, and (3) how to alleviate data sparsity after integrating cross-domain knowledge. To this end, we propose <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">CHFMG</b>, a contrastive hypergraph flow model with multifaceted gates, which contains two modules, i.e., dual hypergraph flow modeling and multi-view contrastive learning. The first module develops a dual hypergraph flow network to explore dynamic intra- and inter-domain sequential patterns. Innovative multifaceted attentive transfer gates connect local and global hypergraph flow, realizing internal feature fusion and external feature alignment in transfer. The second module achieves contrastive learning from two aspects, i.e., short-term contrasting for retaining sequential pattern consistency and long-term contrasting for enhancing distribution uniformity. Empirical studies on benchmark datasets demonstrate the effectiveness of <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">CHFMG</b>.