Long-term Dependencies Research Articles

Short-term photovoltaic power prediction model based on hierarchical clustering of K-means++ algorithm and deep learning hybrid model

In order to further improve the accuracy of photovoltaic (PV) power prediction and the stability of power system, a short-term PV power prediction model based on hierarchical clustering of K-means++ algorithm and deep learning hybrid model is proposed in this paper. First, hierarchical clustering of the K-means++ algorithm is used to cluster historical data into different weather scenes according to different seasons. Second, a hybrid model combining convolutional neural network (CNN), squeeze-and-excitation attention mechanism (SEAM), and bidirectional long short-term memory (BILSTM) neural network is constructed to capture long-term dependencies in time series, and the improved pelican optimization algorithm (IPOA) is used to optimize the hyperparameters in the prediction model. Finally, an example for modeling analysis is conducted by using the actual output and meteorological data of a PV power station in the Ili region of Xinjiang, China. The effectiveness and accuracy of the proposed model are verified by comparing with LSTM, BILSTM, CNN-BILSTM, and POA-CNN-SEAM-BILSTM models, and the superiority of IPOA is verified by comparing with particle swarm optimization and whale optimization algorithm. The results show that the proposed model can obtain better results under different weather scenes in different seasons, and the prediction accuracy of the model optimized by IPOA is further improved.

State Estimation of Lithium-ion Battery for Shipboard Applications: Key Challenges and Future Trends

With the aggravation of environmental problems caused by the long-term dependence of shipping traffic on heavy fossil fuels, it is an irreversible development trend for electrified ships to integrate large-capacity battery energy storage systems (ESSs). As the main component, the shipboard lithium-ion battery (LIB) plays an important role in the operation of ship power system to balance the source and load sides. By analyzing the effects of temperature, vibration, humidity and salt spray on battery characteristics in the shipping environment, this paper points out that the characteristics of shipboard LIB have certain differences on the state changes with the land-based batteries. Then, this paper systematically reviews the most commonly used LIB modeling and state estimation methods and their applicability to the shipping environment, including the empirical models, electrochemical models, equivalent circuit models (ECMs) and data-driven models. On this basis, the state estimation methods of state of charge (SOC), state of power (SOP), state of health (SOH), state of energy (SOE) and state of temperature (SOT) are reviewed. Finally, the challenges and prospects of shipboard LIB research are prospected, in the hope of providing inspiration for the development and design of efficient and safe electric ships.

Research and implementation of multi-disease diagnosis on chest X-ray based on vision transformer.

Disease diagnosis in chest X-ray images has predominantly relied on convolutional neural networks (CNNs). However, Vision Transformer (ViT) offers several advantages over CNNs, as it excels at capturing long-term dependencies, exploring correlations, and extracting features with richer semantic information. We adapted ViT for chest X-ray image analysis by making the following three key improvements: (I) employing a sliding window approach in the image sequence feature extraction module to divide the input image into blocks to identify small and difficult-to-detect lesion areas; (II) introducing an attention region selection module in the encoder layer of the ViT model to enhance the model's ability to focus on relevant regions; and (III) constructing a parallel patient metadata feature extraction network on top of the image feature extraction network to integrate multi-modal input data, enabling the model to synergistically learn and expand image-semantic information. The experimental results showed the effectiveness of our proposed model, which had an average area under the curve value of 0.831 in diagnosing 14 common chest diseases. The metadata feature network module effectively integrated patient metadata, further enhancing the model's accuracy in diagnosis. Our ViT-based model had a sensitivity of 0.863, a specificity of 0.821, and an accuracy of 0.834 in diagnosing these common chest diseases. Our model has good general applicability and shows promise in chest X-ray image analysis, effectively integrating patient metadata and enhancing diagnostic capabilities.

Enhanced prediction of stock markets using a novel deep learning model PLSTM-TAL in urbanized smart cities

Accurate predictions of stock markets are important for investors and other stakeholders of the equity markets to formulate profitable investment strategies. The improved accuracy of a prediction model even with a slight margin can translate into considerable monetary returns. However, the stock markets' prediction is regarded as an intricate research problem for the noise, complexity and volatility of the stocks' data. In recent years, the deep learning models have been successful in providing robust forecasts for sequential data. We propose a novel deep learning-based hybrid classification model by combining peephole LSTM with temporal attention layer (TAL) to accurately predict the direction of stock markets. The daily data of four world indices including those of U.S., U.K., China and India, from 2005 to 2022, are examined. We present a comprehensive evaluation with preliminary data analysis, feature extraction and hyperparameters' optimization for the problem of stock market prediction. TAL is introduced post peephole LSTM to select the relevant information with respect to time and enhance the performance of the proposed model. The prediction performance of the proposed model is compared with that of the benchmark models CNN, LSTM, SVM and RF using evaluation metrics of accuracy, precision, recall, F1-score, AUC-ROC, PR-AUC and MCC. The experimental results show the superior performance of our proposed model achieving better scores than the benchmark models for most evaluation metrics and for all datasets. The accuracy of the proposed model is 96% and 88% for U.K. and Chinese stock markets respectively and it is 85% for both U.S. and Indian markets. Hence, the stock markets of U.K. and China are found to be more predictable than those of U.S. and India. Significant findings of our work include that the attention layer enables peephole LSTM to better identify the long-term dependencies and temporal patterns in the stock markets' data. Profitable and timely trading strategies can be formulated based on our proposed prediction model.

Emergency sign language recognition from variant of convolutional neural network (CNN) and long short term memory (LSTM) models

Sign language is the primary communication tool used by the deaf community and people with speaking difficulties, especially during emergencies. Numerous deep learning models have been proposed to solve the sign language recognition problem. Recently. Bidirectional LSTM (BLSTM) has been proposed and used in replacement of Long Short-Term Memory (LSTM) as it may improve learning long-team dependencies as well as increase the accuracy of the model. However, there needs to be more comparison for the performance of LSTM and BLSTM in LRCN model architecture in sign language interpretation applications. Therefore, this study focused on the dense analysis of the LRCN model, including 1) training the CNN from scratch and 2) modeling with pre-trained CNN, VGG-19, and ResNet50. Other than that, the ConvLSTM model, a special variant of LSTM designed for video input, has also been modeled and compared with the LRCN in representing emergency sign language recognition. Within LRCN variants, the performance of a small CNN network was compared with pre-trained VGG-19 and ResNet50V2. A dataset of emergency Indian Sign Language with eight classes is used to train the models. The model with the best performance is the VGG-19 + LSTM model, with a testing accuracy of 96.39%. Small LRCN networks, which are 5 CNN subunits + LSTM and 4 CNN subunits + BLSTM, have 95.18% testing accuracy. This performance is on par with our best-proposed model, VGG + LSTM. By incorporating bidirectional LSTM (BLSTM) into deep learning models, the ability to understand long-term dependencies can be improved. This can enhance accuracy in reading sign language, leading to more effective communication during emergencies.

MILET: multimodal integration and linear enhanced transformer for electricity price forecasting

The electricity market is a complex and dynamic environment characterized by a multitude of factors that influence electricity prices. Accurate and reliable electricity price forecasting (EPF) is crucial for market participants, including power generators, consumers, and policymakers. Electricity prices are influenced by temporal dependencies and electricity consumption patterns. Therefore, dependencies across different feature dimensions (cross-dimensional dependencies) and temporal trend information are essential. To address the aforementioned issues, we propose Multimodal Integration and Linear Enhanced Transformer (MILET), which combines cross-dimensional dependencies with single-dimensional modal features. First, we decompose electricity price data into three regular modals using Variational Mode Decomposition and Sample Entropy. This approach enables us to uncover the intrinsic patterns within the variable, thereby simplifying the complexity of the data series. Then integrate these three modals and the original dataset into a five-channel encoder (Modal Integration Encoder, MIE) with both single and multi-dimensional information. MIE is composed of Overall Two-Stage Attention (OTSA) and Long Short-Term Memory (LSTM), where OTSA handles cross-dimensional dependencies, and LSTM addresses long-term dependencies. Additionally, we capture trend information in electricity consumption features through linear layers and linearly integrate the data to obtain the forecasting results. Extensive experimental results on five electricity price datasets demonstrate the effectiveness of MILET compared to state-of-the-art techniques. Our code is available at https://github.com/Lisen-Zhao/MILET/tree/master.

Enhancing Disease Prediction with a Hybrid CNN-LSTM Framework in EHRs

This study developed a novel hybrid deep learning framework aimed at enhancing the accuracy of disease prediction using temporal data from Electronic Health Records (EHRs). The framework integrates Convolutional Neural Networks (CNN) and Long Short-Term Memory (LSTM) networks, leveraging the strength of CNNs in extracting hierarchical feature representations from complex data and the capability of LSTMs in capturing long-term dependencies in temporal information. An empirical investigation on real-world EHR datasets revealed that, compared to Support Vector Machine (SVM) models, standalone CNNs, and LSTMs, this hybrid deep learning network demonstrated significantly higher prediction accuracy in disease prediction tasks. This research not only advances the performance of predictive models in the health data analytics domain but also underscores the importance of adopting and further developing advanced deep learning technologies to address the complexity of modern medical data. Our findings advocate for a shift towards integrating complex neural network architectures in developing predictive models, potentially offering avenues for more personalized and proactive disease management and care, thereby setting new standards for future health management practices.

Subseasonal Prediction of Summer Temperature in West Africa Using Artificial Intelligence: A Case Study of Senegal

Despite the rapid growth of machine learning (ML) and its far-reaching applications in various fields such as healthcare, finance, and urban heat management, there are still some unresolved challenges in the field of climate change. Reliable subseasonal forecasts of summer temperatures would be a great benefit to society. Although numerical weather prediction (NWP) models are better at capturing relevant sources of predictability, such as temperatures, land, and sea surface conditions, the subseasonal potential is not fully exploited. One such challenge is accurate subseasonal temperature forecasting using cutting-edge ML technology. This study aims to assess and predict the changes in subseasonal temperature during the summer season (from March to June) in Senegal on 2-weeks time scales. Six ML techniques, including linear regression (LR), decision tree (DT), support vector machine (SVM), artificial neural network (ANN), long short-term memory (LSTM), and gated recurrent units (GRU), are used. The experiments utilize a multivariate approach by incorporating variables of the ERA-5 dataset from 1981 to 2022. The results compared all the performances of the methods to assess their overall effectiveness in forecasting air temperature (t2m) values over 2 weeks. Our analysis demonstrates that the GRU model outperforms the other ML models, achieving a Nash–Sutcliffe efficiency (NSE) score of 74.68% and a mean absolute percentage error (MAPE) of 2.51%. The GRU model effectively captures long-term dependencies and exhibits superior performance in temperature forecasting. Furthermore, a comparison between the observed and predicted values confirms the accuracy of the GRU model in aligning with actual temperature trends. Overall, this study contributes an impactful deep learning model to the field of subseasonal temperature forecasting in West Africa (Senegal), which offers local authorities the capability to anticipate climatic events and enact preventive measures accordingly.

ANN-LSTM-A Water Consumption Prediction Based on Attention Mechanism Enhancement

To reduce the energy consumption of domestic hot water (DHW) production, it is necessary to reasonably select a water supply plan through early predictions of DHW consumption to optimize energy consumption. However, the fluctuations and intermittence of DHW consumption bring great challenges to the prediction of water consumption. In this paper, an ANN-LSTM-A water quantity prediction model based on attention mechanism (AM) enhancement is improved. The model includes an input layer, an AM layer, a hidden layer, and an output layer. Based on the combination of artificial neural network (ANN) and long short-term memory (LSTM) models, an AM is incorporated to address the issue of the traditional ANN model having difficulty capturing the long-term dependencies, such as lags and trends in time series, to improve the accuracy of the DHW consumption prediction. Through comparative experiments, it was found that the root mean square error of the ANN-LSTM-A model was 15.4%, 13.2%, and 13.2% lower than those of the ANN, LSTM, and ANN-LSTM models, respectively. The corresponding mean absolute error was 17.9%, 11.5%, and 8% lower than those of the ANN, LSTM, and ANN-LSTM models, respectively. The results showed that the proposed ANN-LSTM-A model yielded better performances in predicting DHW consumption than the ANN, LSTM, and ANN-LSTM models. This work provides an effective reference for the reasonable selection of the water supply plan and optimization of energy consumption.

Automatic pipeline fault detection using one-dimensional convolutional bidirectional long short-term memory networks with wide first-layer kernels

Pipeline networks are crucial components of modern infrastructure, and ensuring their reliable operation is essential for sustainable development. The percussion-based methods are considered promising for detecting pipeline faults due to their avoidance of constant-contact sensors and ease of implementation. However, the majority of existing percussion-based methods suffer from limitations such as the requirement for manual feature extraction, as well as subpar noise resilience and adaptability. This paper introduces a one-dimensional convolutional bidirectional long short-term memory network with wide first-layer kernels for the classification of percussion-induced acoustic signals, thus achieving automatic identification of pipeline leakage and water deposit conditions. This approach directly extracts features from audio signals using wide first-layer convolutional kernels, eliminating the need for manual feature extraction. Additionally, it employs bidirectional long short-term memory to effectively capture long-term signal dependencies from both past and future contexts. To validate the effectiveness of the method, two case studies were conducted on three groups of pipes. The results show that the proposed method demonstrates superior noise resistance and adaptability compared to other methods, and it also exhibits strong applicability to other percussion signal datasets. Additionally, the impact of different first convolutional kernel sizes on the noise resistance and adaptive performance of the model was investigated, which provides robust guidance for the effective processing of percussion-induced acoustic signals.

Wind power deviation charge reduction using long short term memory network

High penetration of variable generation like wind in modern power systems results in frequent load-generation imbalances. Additional spinning reserves and balancing services are deployed to reduce or avoid such imbalances. However, these additional balancing services result in increased total operation costs and it is generally shifted to Wind Power Generation Companies (WPGC) as Deviation Charges (DCs). The high DCs diminish the profitability of WPGCs. DCs are directly proportional to forecasting errors and can be reduced by enhancing forecasting accuracy. This paper proposes a Long Short-Term Memory (LSTM) network to reduce DCs. LSTM is a special kind of recurrent neural network and it is capable of learning long-term and short-term dependencies effectively. Therefore, the LSTM network can produce minimal error wind power forecasts and this helps to reduce DCs. The performance of the proposed LSTM network is realized using the real system data. Results obtained from the proposed LSTM model are compared with various forecasting models like ARIMA, ANN, and SVR. The comparison shows that the proposed LSTM model shows the least total DCs on most of the days followed by ANN, SVR, and ARIMA. The proposed LSTM, ANN, SVR, and ARIMA models show an average total DC of 6194.25$, 8459.32$, 15822.22$, and 38269.89$ respectively. Thus, the proposed LSTM model is capable of reducing 517.82%, 155.43%, and 36.56% DCs from ARIMA, SVR, and ANN models respectively, and helps the WPGCs to enhance their profitability.

Using Machine Learning and Finite Element Analysis to Extract Traction-Separation Relations at Bonding Wire Interfaces of Insulated Gate Bipolar Transistor Modules.

For insulated gate bipolar transistor (IGBT) modules using wire bonding as the interconnection method, the main failure mechanism is cracking of the bonded interface. Studying the mechanical properties of the bonded interface is crucial for assessing the reliability of IGBT modules. In this paper, first, shear tests are conducted on the bonded interface to test the bonded interface's strength. Then, finite element-cohesive zone modeling (FE-CZM) is established to describe the mechanical behavior of the bonded interface. A novel machine learning (ML) architecture integrating a convolutional neural network (CNN) and a long short-term memory (LSTM) network is used to identify the shape and parameters of the traction separation law (TSL) of the FE-CZM model accurately and efficiently. The CNN-LSTM architecture not only has excellent feature extraction and sequence-data-processing abilities but can also effectively address the long-term dependency problem. A total of 1800 sets of datasets are obtained based on numerical computations, and the CNN-LSTM architecture is trained with load-displacement (F-δ) curves as input parameters and TSL shapes and parameters as output parameters. The results show that the error rate of the model for TSL shape prediction is only 0.186%. The performance metric's mean absolute percentage error (MAPE) is less than 3.5044% for all the predictions of the TSL parameters. Compared with separate CNN and LSTM architectures, the proposed CNN-LSTM-architecture approach exhibits obvious advantages in recognizing TSL shapes and parameters. A combination of the FE-CZM and ML methods in this paper provides a promising and effective solution for identifying the mechanical parameters of the bonded interfaces of IGBT modules.

Urban rail transit passenger flow prediction with ResCNN-GRU based on self-attention mechanism

With the development of modern cities, urban rail transit has become an indispensable part of residents' travelling mode, and accurate prediction of urban rail transit passenger flow is particularly important. However, due to the non-linearity and non-stability of passenger flow, the low quality of big data and the lack of data make it more and more difficult to predict the passenger flow of urban rail transit. In this paper, we propose a unique model structure that ingeniously integrates Gated Recurrent Units (GRU), Convolutional Neural Networks (CNN) with fused residual network connections, and a self-attention mechanism. The design of this model is primarily to effectively handle complex and low-quality data, which is often prevalent in passenger flow prediction scenarios. To validate the effectiveness of the proposed model, we used data from the Beijing urban rail transit in 2016 for prediction. In the experiments, we performed comparison study and ablation study. The experimental results show that the model proposed in this paper has significantly improved prediction accuracy in both horizontal and vertical comparisons. This outcome substantiates that the proposed model can not only effectively handle complex and low-quality data but also extract short-term features and long-term dependency features of passenger flow well, thereby achieving more accurate predictions.

Non-probability sampling network based on anomaly pedestrian trajectory discrimination for pedestrian trajectory prediction

Pedestrian trajectory prediction in first-person view is an important support for achieving fully automated driving in cities. However, existing pedestrian trajectory prediction methods still have significant shortcomings in terms of pedestrian trajectory diversity, dynamic scene constraints, and dependence on long-term trajectory prediction. We proposes a non-probability sampling network based on pedestrian trajectory anomaly recognition (ADsampler) to predict multiple possible future pedestrian trajectories. First, by incorporating pose and optical flow information, ADsampler models the multi-dimensional motion characteristics of pedestrians based on observed trajectory information and discriminates trajectory states. The sampling range in the Gaussian latent space is determined based on the recognition results. Next, velocity and yaw information of the car are introduced to model the car's motion state. A subtraction fusion network is employed to remove redundant image feature constraints in highly dynamic scenes. Finally, ADsampler utilizes a novel trajectory decoding network that combines the position encoding capability of GRU with the long-term dependency capturing ability of Transformer to decode and predict the fused features. we evaluate our model on crowded videos in the public datasets JAAD, PIE, ETH and UCY. Experiments demonstrate that the proposed method outperforms state-of-the-art approaches in prediction accuracy.

Predicting circRNA-RBP Binding Sites Using a Hybrid Deep Neural Network.

Circular RNAs (circRNAs) are non-coding RNAs generated by reverse splicing. They are involved in biological process and human diseases by interacting with specific RNA-binding proteins (RBPs). Due to traditional biological experiments being costly, computational methods have been proposed to predict the circRNA-RBP interaction. However, these methods have problems of single feature extraction. Therefore, we propose a novel model called circ-FHN, which utilizes only circRNA sequences to predict circRNA-RBP interactions. The circ-FHN approach involves feature coding and a hybrid deep learning model. Feature coding takes into account the physicochemical properties of circRNA sequences and employs four coding methods to extract sequence features. The hybrid deep structure comprises a convolutional neural network (CNN) and a bidirectional gated recurrent unit (BiGRU). The CNN learns high-level abstract features, while the BiGRU captures long-term dependencies in the sequence. To assess the effectiveness of circ-FHN, we compared it to other computational methods on 16 datasets and conducted ablation experiments. Additionally, we conducted motif analysis. The results demonstrate that circ-FHN exhibits exceptional performance and surpasses other methods. circ-FHN is freely available at https://github.com/zhaoqi106/circ-FHN .

Integrating LSTM and NLP techniques for essay generation

This research explores the potential of Long Short-Term Memory (LSTM) and Natural Language Processing (NLP) for automated essay generation. The goal of the study is to create a model that can produce high-quality essays that are not only grammatically correct but also semantically meaningful and contextually relevant. The rise of NLP and Deep Learning has made it possible to generate text that is coherent and semantically sound. In this research, there is an advantage by leveraging the ability of LSTMs to capture long-term dependencies and context within the text, and combining it with NLP techniques, such as word embeddings, to process and encode textual data. The results of experiments show that the proposed model can effectively generate essays that are coherent, contextually relevant, and semantically meaningful. This is a significant advancement in the field of text generation and has potential applications in areas such as education, content creation, and language translation. In education, for example, the model could be used to generate essays for language proficiency tests or as a writing aid for students. In content creation, it could be used to generate articles, blog posts, and other written content. In language translation, the model could be used to generate essays in the target language that are semantically and contextually equivalent to the source language essay. The findings of this study contribute to the advancement of NLP and deep learning techniques in the area of text generation and open up new avenues for future research. Short after the proposed model was deployed, it was discovered that it outscored Multi-Topic Aware LSTM (MTA-LSTM), Topic-Attention LSTM (TAT-LSTM), and Topic-Averaged LSTM (TAV-LSTM) in human evaluation by 6.84 percent, 25.40 percent, and 34.94 percent, respectively. Furthermore, it enhanced automatic BLEU score evaluation scores by 11.68 percent, 26.23 percent, and 54.11 percent in MTA-LSTM, TAT-LSTM, and TAV- LSTM, respectively.

GA-Dueling DQN Jamming Decision-Making Method for Intra-Pulse Frequency Agile Radar.

Optimizing jamming strategies is crucial for enhancing the performance of cognitive jamming systems in dynamic electromagnetic environments. The emergence of frequency-agile radars, capable of changing the carrier frequency within or between pulses, poses significant challenges for the jammer to make intelligent decisions and adapt to the dynamic environment. This paper focuses on researching intelligent jamming decision-making algorithms for Intra-Pulse Frequency Agile Radar using deep reinforcement learning. Intra-Pulse Frequency Agile Radar achieves frequency agility at the sub-pulse level, creating a significant frequency agility space. This presents challenges for traditional jamming decision-making methods to rapidly learn its changing patterns through interactions. By employing Gated Recurrent Units (GRU) to capture long-term dependencies in sequence data, together with the attention mechanism, this paper proposes a GA-Dueling DQN (GRU-Attention-based Dueling Deep Q Network) method for jamming frequency selection. Simulation results indicate that the proposed method outperforms traditional Q-learning, DQN, and Dueling DQN methods in terms of jamming effectiveness. It exhibits the fastest convergence speed and reduced reliance on prior knowledge, highlighting its significant advantages in jamming the subpulse-level frequency-agile radar.

Multi-scale spatial features and temporal attention mechanisms: advancing the accuracy of ENSO prediction

The exponential progression in oceanic observational technology has fostered the accumulation of substantial time series data pivotal for predictions in ocean meteorology. Foremost among the phenomena observed is El Niño-Southern Oscillation (ENSO), a critical determinant in the interplay of global ocean atmosphere interactions, with its severe manifestations inducing extreme meteorological conditions. Therefore, precisely predicting ENSO events carries immense gravitas. Historically, predictions hinged primarily on dynamic models and statistical approaches; however, the intricate and multifaceted spatiotemporal dynamics of ENSO events have often impeded the accuracy of these traditional methodologies. A notable lacuna in contemporary research is the insufficient exploration of long-term dependencies within oceanic data and the suboptimal integration of spatial information derived from spatiotemporal data. To address these limitations, this study introduces a forward-thinking ENSO prediction framework synergizing multiscale spatial features with temporal attention mechanisms. This innovation facilitates a more profound exploration of temporal and spatial domains, enhancing the retention of extensive-period data while optimizing the use of spatial information. Preliminary analyses executed on the global ocean data assimilation system dataset attest to the superior efficacy of the proposed method, underscoring a substantial improvement over established methods including SA-convolutional long short-term memory, particularly in facilitating long-term predictions.The source code and datasets are provided. The code is available at https://github.com/tse1998/ENSO-prediction.

Predicting customer abandonment in recurrent neural networks using short-term memory

Customer retention, a critical business priority, has become a growing concern, especially in the telecommunications industry. This study addresses the need to anticipate and understand customer churn through the application of Deep Learning models. The central focus of the research was the development and evaluation of a short-term memory model (LSTM) specifically designed to predict customer leakage. The choice of LSTM as the mainstay of the research is based on its proven ability to model long-term dependencies in sequences, its resilience to recurrent challenges in neural networks, and its success in various sequence prediction tasks. The model implementation, configured sequentially with Keras, comprised of an initial LSTM layer of 64 units, followed by a 20% removal layer to mitigate overfitting. The second LSTM layer, with 32 units, was supplemented with another elimination layer. Model training was conducted using a dataset consisting of 20 attributes and 4250 records. The model evaluation was based on crucial measures such as precision, accuracy, sensitivity and F1 count, revealing exceptional results with 95% performance on all metrics. This study, therefore, highlights the effectiveness of the LSTM model in predicting customer churn, providing companies with a valuable tool to improve retention and mitigate associated losses.

DLAN:Modeling user long- and short-term preferences based on double-layer attention network for next point-of-interest recommendation

The next Point-of-Interest (POI) recommendation, in recent years, has attracted an extensive amount of attention from the academic community. RNN-based methods cannot establish effective long-term dependencies among the input sequences when capturing the user’s motion patterns, resulting in inadequate exploitation of user preferences. Besides, the majority of prior studies often neglect high-order neighborhood information in users’ check-in trajectory and their social relationships, yielding suboptimal recommendation efficacy. To address these issues, this paper proposes a novel Double-Layer Attention Network model, named DLAN. Firstly, DLAN incorporates a multi-head attention module that can combine first-order and high-order neighborhood information in user check-in trajectories, thereby effectively and parallelly capturing both long- and short-term preferences of users and overcoming the problem that RNN-based methods cannot establish long-term dependencies between sequences. Secondly, this paper designs a user similarity weighting layer to measure the influence of other users on the target users leverage the social relationships among them. Finally, comprehensive experiments are conducted on user check-in data from two cities, New York (NYC) and Tokyo (TKY), and the results demonstrate that DLAN achieves a performance in Accuracy and Mean Reverse Rank enhancement by 8.07% -36.67% compared to the state-of-the-art method. Moreover, to investigate the effect of dimensionality and the number of heads of the multi-head attention mechanism on the performance of the DLAN model, we have done sufficient sensitivity experiments.