Long Short-term Memory Neural Network Research Articles

Forecasting stock prices changes using long-short term memory neural network with symbolic genetic programming

This study introduces an augmented Long-Short Term Memory (LSTM) neural network architecture, integrating Symbolic Genetic Programming (SGP), with the objective of forecasting cross-sectional price returns across a comprehensive dataset comprising 4500 listed stocks in the Chinese market over the period from 2014 to 2022. Using the S&P Alpha Pool Dataset for China as basic input, this architecture incorporates data augmentation and feature extraction techniques. The result of this study demonstrates significant improvements in Rank Information coefficient (Rank IC) and IC information ratio (ICIR) by 1128% and 5360% respectively when it is applied to fundamental indicators. For technical indicators, the hybrid model achieves a 206% increase in Rank IC and an impressive surge of 2752% in ICIR. Furthermore, the proposed hybrid SGP-LSTM model outperforms major Chinese stock indexes, generating average annualized excess returns of 31.00%, 24.48%, and 16.38% compared to the CSI 300 index, CSI 500 index, and the average portfolio, respectively. These findings highlight the effectiveness of SGP-LSTM model in improving the accuracy of cross-sectional stock return predictions and provide valuable insights for fund managers, traders, and financial analysts.

Secure key generation and distribution scheme based on historical fiber channel state information with LSTM.

In this paper, a scheme to realize unclonable physical-layer security key generation and distribution (PL-SKGD) based on historical fiber channel state information (HFCSI) is proposed. PL-SKGD schemes based on channel characteristics for enhancing the physical-layer security of optical networks have been proposed in recent years. However, there are potential disadvantages in these schemes, such as 1) low key generation rate (KGR): the slow frequency of the analog waveform change of the channel characteristic leading to low KGR; 2) incompatibility with existing infrastructure: active scrambling to increase the frequency of channel characteristic changes, or tracking changes of channel characteristics requires additional devices; 3)easy to be cloned: all of the optical channel state information is reflected in the signal transmitted inside the fiber, which makes it easy to reproduce by illegal eavesdropper through features analysis and other methods. In order to solve the above problems, a PL-SKGD scheme is designed which uses the chain structure composed of long short-term memory neural network (LSTM-NN) units to learn and store the unique mapping relationship between historical channel time series and provides unclonability based on the fundamental fact that the eavesdropper Eve can never obtain the full HFCSI. The simulation conducted in a quadrature phase shift keying point-to-point optical link system verified successfully that KGR = 0.82 Gbit/s error-free SKGD. The loss function of LSTM-NN drops sharply in the early stages of training and remains a small value. The security of the SKGD system is analyzed, which effectively improves the unclonability of the system. Finally, it is verified that the optimal fiber channel length for error-free SKGD of the proposed scheme is 150 km considering the error correction capability of information reconciliation and weighing key sequence error rate and valid bit generation rate.

Prediction and classification of IoT sensor faults using hybrid deep learning model

The quality and reliability of internet of thing (IoT) ecosystems heavily rely on accurate and dependable sensor data. However, resource limited sensors are prone to failure due to various factors like environmental disturbances and electrical noise in which they can produce erroneous and faulty measurements. These can have significant consequences across different domains, including a threat to safety in critical systems. Though many researches have been conducted, the existing literature primarily focuses on fault detection in the sensor data, while fault detection is useful, it is still a reactive approach that identifies the faults after they have occurred, meaning that actions are taken after the fault has already impacted the system, potentially leading to negative consequences. In this study, a proactive approach has been proposed by developing a two-stage solution. In the first stage, a hybrid convolutional neural network-long short term memory (CNN-LSTM) model was trained to forecast sensor measurements based on historical data, while in the second stage, the forecasted measurements were passed to a hybrid convolutional neural network-multi layer perceptron (CNN-MLP) model that has been trained to recognize different types of sensor faults and classify the new measurements accordingly. By passing the forecasted sensor values as input to the classification model and categorizing them as normal, bias, drift, random or poly-drift, anticipated the potential faults before they manifest. The publicly available Intel Lab data raw dataset is used, which has been annotated and fault-injected. For regression, gated recurrent unit (GRU), Long short term memory (LSTM), bidirectional long short term memory (BiLSTM), convolutional neural network-gated recurrent unit (CNN-GRU), convolutional neural network-long short term memory (CNN-LSTM), and convolutional neural network-bidirectional long short term memory (CNN-BiLSTM), were evaluated and compared their performance using root mean squared error (RMSE), mean squared error (MSE) and mean absolute error (MAE) with 2-split time series cross-validation. CNN-LSTM outperformed the other models with a Mean Absolute Error of 2.0957 for a 45 time steps forecast. For the classification task, convolutional neural network (CNN), multi-layer perceptron (MLP), and convolutional neural network-multi layer perceptron (CNN-MLP) evaluated using the metrics accuracy, precision, recall, and F1-score with 5 and tenfold cross-validations. CNN-MLP outperformed the others with accuracy of 96.11% for bias, 99.33% for drift, and 98.61% for random and 98.81% for poly-drift. The average accuracy across the 4 faults is 98.21%, which is a 0.3% increase from the baseline work 97.91%. By adopting a proactive approach to sensor fault prediction and classification, this research aims to enhance the reliability and efficiency of IoT systems, allowing for preventive measures to be taken before faults have a detrimental impact.

An innovative approach to detecting the freshness of fruits and vegetables through the integration of convolutional neural networks and bidirectional long short-term memory network

Fruit and vegetable freshness testing can improve the efficiency of agricultural product management, reduce resource waste and economic losses, and plays a vital role in increasing the added value of fruit and vegetable agricultural products. At present, the detection of fruit and vegetable freshness mainly relies on manual feature extraction combined with machine learning. However, manual extraction of features has the problem of poor adaptability, resulting in low efficiency in fruit and vegetable freshness detection. Although exist some studies that have introduced deep learning methods to automatically learn deep features that characterize the freshness of fruits and vegetables to cope with the diversity and variability in complex scenes. However, the performance of these studies on fruit and vegetable freshness detection needs to be further improved. Based on this, this paper proposes a novel method that fusion of different deep learning models to extract the features of fruit and vegetable images and the correlation between various areas in the image, so as to detect the freshness of fruits and vegetables more objectively and accurately. First, the image size in the dataset is resized to meet the input requirements of the deep learning model. Then, deep features characterizing the freshness of fruits and vegetables are extracted by the fused deep learning model. Finally, the parameters of the fusion model were optimized based on the detection performance of the fused deep learning model, and the performance of fruit and vegetable freshness detection was evaluated. Experimental results show that the CNN_BiLSTM deep learning model, which fusion convolutional neural network (CNN) and bidirectional long-short term memory neural network (BiLSTM), is combined with parameter optimization processing to achieve an accuracy of 97.76% in detecting the freshness of fruits and vegetables. The research results show that this method is promising to improve the performance of fruit and vegetable freshness detection.

Sampled-Data State Estimation for LSTM.

This article first introduces a sampled-data state estimator design method for continuous-time long short-term memory (LSTM) neural networks with irregularly sampled output. To this end, the structure of the LSTM is addressed to obtain its dynamic equation. As a result, the LSTM neural network is modeled as a continuous-time linear parameter-varying system that is dependent on the gate units. For this system, the sampled-data Luenberger-and Arcak-type state estimator design methods are presented in terms of linear matrix inequalities (LMIs) by using the properties of the gate units. Lastly, the proposed method not only provides a numerical example for analyzing absolute stability but also demonstrates it in practice by applying a pre-trained behavior generation model of a robot manipulator.

F10.7 Daily Forecast Using LSTM Combined With VMD Method

AbstractThe F10.7 solar radiation flux is a well‐known parameter that is closely linked to solar activity, serving as a key index for measuring the level of solar activity. In this study, the Variational Mode Decomposition (VMD) and Long Short‐term Memory (LSTM) network are combined to construct a VMD‐LSTM model for predicting F10.7 values. The F10.7 sequence is decomposed into several intrinsic mode functions (IMF) by VMD, then the LSTM neural network is utilized to forecast each IMF. All IMF prediction results are aggregated to obtain the final F10.7 value. The data sets from 1957 to 2008 are used for training and the data sets from 2009 to 2019 are used for testing. The results show that the VMD‐LSTM model achieves an annual average root mean square error of only 4.47 sfu and an annual average correlation coefficient (R) of 0.99 during solar cycle 24, which is significantly better than the accuracy of the LSTM model (W. Zhang et al., 2022, https://doi.org/10.3390/universe8010030), the AR model (Du, 2020, https://doi.org/10.1007/s11207-020-01689-x), and the BP model (Xiao et al., 2017, https://doi.org/10.11728/cjss2017.01.001). The VMD‐LSTM model exhibits strong predictive capability for the F10.7 index during solar cycle 24.

Mental Health Status Detection Model Based on LSTM Neural Network

In today's society where mental health is increasingly concerned, how to effectively monitor and evaluate an individual's mental health status has become a research focus. However, traditional mental health evaluation methods have disadvantages such as strong subjectivity and strong dependence on professionals, and there is an urgent need to establish an automatic evaluation method based on objective data. This article is based on the LSTM (Long Short-Term Memory) mental health state detection model, which inputs the text sequence data to be detected into the LSTM model, and obtains the corresponding mental health state prediction results through forward transmission. A threshold can be set to determine whether the mental health status is normal, or the predicted results can be compared with real markers to evaluate the precision and performance of the model. Multiple features such as emotional analysis with textual features were integrated to construct a new model for evaluating the role of integrated features in mental health state recognition. Only text features had an accuracy rate of 0.87, an precision rate of 0.89, and a recall rate of 0.86. The accuracy rate under text feature+sentiment analysis was 0.90; the precision rate was 0.92; the recall rate was 0.89. This article provided reference and tool support for the detection of mental health status.

Evaluation of Neoadjuvant Chemoradiotherapy Response in Rectal Cancer Using MR Images and Deep Learning Neural Networks.

The aim of the study was to develop deep-learning neural networks to guide treatment decisions and for the accurate evaluation of tumor response to neoadjuvant chemoradiotherapy (nCRT) in rectal cancer using magnetic resonance (MR) images. Fifty-nine tumors with stage 2 or 3 rectal cancer that received nCRT were retrospectively evaluated. Pathological tumor regression grading was carried out using the Dworak (Dw-TRG) guidelines and served as the ground truth for response predictions. Imaging-based tumor regression grading was performed according to the MERCURY group guidelines from pre-treatment and post-treatment para-axial T2-weighted MR images (MR-TRG). Tumor signal intensity signatures were extracted by segmenting the tumors volumetrically on the images. Normalized histograms of the signatures were used as input to a deep neural network (DNN) housing long short-term memory (LSTM) units. The output of the network was the tumor regression grading prediction, DNN-TRG. In predicting complete or good response, DNN-TRG demonstrated modest agreement with Dw-TRG (Cohen's kappa= 0.79) and achieved 84.6% sensitivity, 93.9% specificity, and 89.8% accuracy. MR-TRG revealed 46.2% sensitivity, 100% specificity, and 76.3% accuracy. In predicting a complete response, DNN-TRG showed slight agreement with Dw-TRG (Cohen's kappa= 0.75) with 71.4% sensitivity, 97.8% specificity, and 91.5% accuracy. MR-TRG provided 42.9% sensitivity, 100% specificity, and 86.4% accuracy. DNN-TRG benefited from higher sensitivity but lower specificity, leading to higher accuracy than MR-TRG in predicting tumor response. The use of deep LSTM neural networks is a promising approach for evaluating the tumor response to nCRT in rectal cancer.</P>.

Assessing Consciousness in Patients With Disorders of Consciousness Using a Musical Stimulation Paradigm and Verifiable Criteria.

Numerous studies have shown that musical stimulation can activate corresponding functional brain areas. Electroencephalogram (EEG) activity during musical stimulation can be used to assess the consciousness states of patients with disorders of consciousness (DOC). In this study, a musical stimulation paradigm and verifiable criteria were used for consciousness assessment. Twenty-nine participants (13 healthy subjects, 6 patients in a minimally conscious state (MCS) and 10 patients in a vegetative state (VS)) were recruited, and EEG signals were collected while participants listened to preferred and relaxing music. Fusion features based on differential entropy (DE), common spatial pattern (CSP), and EEG-based network pattern (ENP) features were extracted from EEG signals, and a convolutional neural network-long short-term memory (CNN-LSTM) model was employed to classify preferred and relaxing music.The results showed that the average classification accuracy for healthy subjects reached 85.58%. For two of the patients in the MCS group, the classification accuracies reached 78.18% and 66.14%, and they were diagnosed with emergence from MCS (EMCS) two months later. The accuracies of three patients in the VS group were 58.18%, 64.32% and 62.05%, with two patients showing slight increases in scale scores. Our study suggests that musical stimulation could be an effective method for consciousness detection, with significant diagnostic implications for patients with DOC.

CNN-LSTM Hybrid Model for Enhanced Malware Analysis and Detection

The ever-evolving tactics employed by malware authors to avoid detection pose challenges to the conventional static analysis method, which entails examining the malware code. These challenges arise from the authors’ capacity to obfuscate their code. To address this matter and enhance the identification of malware, integrating dynamic detection and machine learning has emerged as a highly promising approach. This methodology has demonstrated efficacy in identifying malware specifically engineered to circumvent established detection techniques. Behavioural analysis is a crucial component in ensuring endpoints’ security, with the CNN-LSTM algorithm being particularly notable for its effectiveness in identifying Zero-Day malware. This type of malware poses a substantial obstacle to conventional signature-based approaches. This paper aims to assess the efficacy of the Convolutional Neural Network-Long Short-Term Memory (CNN-LSTM) model, emphasising its significance in tackling the continuously evolving realm of cybersecurity obstacles. The research highlights the significance of transitioning from traditional signature-based detection methods to behavioural analysis techniques. It suggests utilising deep learning approaches such as Long Short-Term Memory (LSTM) and Convolutional Neural Networks (CNN) to improve the ability to detect malware in an environment where threats constantly evolve. The malware detection system that has been developed encompasses a log parser analyser, API monitoring, and an extension checker module. The CNN-LSTM model demonstrates a commendable ability to accurately identify malicious behaviour, achieving a validation accuracy of 96%. This study demonstrates the efficacy of employing behavioural analysis and deep learning techniques to enhance cybersecurity, particularly in addressing sophisticated, evasive, and previously unknown malware risks.

Forecasting electricity consumption by LSTM neural network

Relevance. The need to enhance the precision of electricity consumption forecasting for improving energy efficiency and, consequently, enhancing the competitiveness of manufactured products by reducing the proportion of electricity costs in their total cost. When determining forecast indicators of electricity consumption by industrial enterprises, it is important to apply contemporary high-precision forecasting methods. Only 20–30 forecasting methods of the 150 existing ones are actively implemented in practice. An examination of prevailing forecasting methodologies used by industrial enterprises reveals that they are mainly based either on expert assessments of electricity volumes or on accounting for specific electricity consumption (per unit of product manufactured). Aim. To elevate the accuracy of electricity consumption forecasting at industrial enterprises by using artificial intelligence methods, specifically, artificial neural network techniques, including the Long-Short Term Memory (LSTM) approach. Methods. When developing the forecasting model, artificial neural network techniques were adopted, with a particular emphasis on the Long-Short Term Memory (LSTM) method. For primary data processing, Gaussian distribution principles and normalization/scaling techniques were applied. Results. Substantiated computationally by applying the proposed model based on the artificial neural network technique for forecasting electricity consumption of industrial enterprises. A significant advantage of this method is its capability for learning and adaptability to forecasting. Real-time computations demonstrate its successful implementation, attributed primarily to appropriate selection of input layers and mitigation of random variables.

Multiple decomposition‐aided long short‐term memory network for enhanced short‐term wind power forecasting

AbstractWith the increasing penetration of grid‐scale wind energy systems, accurate wind power forecasting is critical to optimizing their integration into the power system, ensuring operational reliability, and enabling efficient system asset utilization. Addressing this challenge, this study proposes a novel forecasting model that combines the long‐short‐term memory (LSTM) neural network with two signal decomposition techniques. The EMD technique effectively extracts stable, stationary, and regular patterns from the original wind power signal, while the VMD technique tackles the most challenging high‐frequency component. A deep learning‐based forecasting model, i.e. the LSTM neural network, is used to take advantage of its ability to learn from longer sequences of data and its robustness to noise and outliers. The developed model is evaluated against LSTM models employing various decomposition methods using real wind power data from three distinct offshore wind farms. It is shown that the two‐stage decomposition significantly enhances forecasting accuracy, with the proposed model achieving values up to 9.5% higher than those obtained using standard LSTM models.

Improving wave height prediction accuracy with deep learning

A novel convolutional neural network-long short-term memory (CNN-LSTM) model is proposed for wave height prediction. The model effectively extracts relevant features such as wind speed, wind direction, wave height, latitude, and longitude. The proposed model outperforms traditional machine learning algorithms such as multi-layer perceptron (MLP), support vector machine (SVM), random forest and LSTM, especially for extreme values and fluctuations. The model has a significantly lower average root mean square error (RMSE) of 71.1%, 72.8%, 71.9% and 72.2% for MLP, SVM, random forest and LSTM, respectively. Our model is computationally more efficient than traditional numerical simulations, making it suitable for real-time applications. Moreover, it has better long-term robustness compared to traditional models. The integration of CNN and LSTM techniques improves wave height prediction accuracy while enhancing its efficiency and robustness. The proposed CNN-LSTM model provides a promising tool for effective wave height prediction, making a valuable contribution to coastal disaster prevention and mitigation. Future research should aim to improve long-term prediction accuracy, and we believe that the CNN-LSTM model plays a crucial role in developing real-time coastal disaster prevention and mitigation measures. Overall, our study represents a significant step towards achieving more accurate and efficient wave height prediction using machine learning techniques.

Monitoring the quality of ground water in pipelines using deep neural network model

Monitoring the water quality in pipelines is crucial to guaranteeing that consumers receive safe and hygienic water. Conventional monitoring techniques rely on labor-intensive, time-consuming laboratory testing and physical sampling. This study aims to introduce an integrated LSTM-BP neural network model for real-time water quality monitoring in pipelines, showcasing improved predictions for parameters like dissolved oxygen and pH. This paper investigated the results of real-time Water Quality (WQ) monitoring in pipelines using several Deep Neural Network (DNN) models and pre-processing techniques for the information being provided. This paper proposes an integrated long-short-term memory (LSTM) and backpropagation (BP) neural network to offer an I-LSTM-BP model for Monitoring Water Quality in Pipelines (MWQP). The performance of predictions has been enhanced through the integration of LSTM and BP neural networks, particularly for time-series forecasting applications like MWQP. The model may use the LSTM network’s memory and adaptability while still keeping the BP network’s capacity to simulate non-linear interactions and enhance performance during generalization. Dissolved oxygen (DO; mg/L) and hydrogen (pH) potential are the parameters considered during the MWQP process. Information collected from China’s autonomous water quality monitoring units has been used to examine the water quality characteristics thoroughly. The forecasting model has been trained and evaluated using data collected from monitoring stations from June 2018 to December 2022. Statistical measures such as the correlation coefficient, root mean square error (RMSE), mean absolute error (MAE), and their normalized equivalents have been used to evaluate the DNN model’s performance throughout the training and testing stages. The results of this work demonstrate the viability and efficiency of using the proposed I-LSTM-BP-based DNN for MWQP applications. These results show the I-LSTM-BP model’s significant performance benefit, which makes it the model of choice for precise MWQP parameter prediction and WQ evaluation.

Pattern recognition system for rapid detection of gases using microfluidic olfaction detector: A case study using methane and ethane

Two significant shortcomings of metal oxide semiconductor (MOS) sensors are their low selectivity and prolonged recovery time after exposure to target gases. Microfluidic olfaction detectors (MFOD), having an embedded MOS at the distal end of a microchannel, improve MOS selectivity but further prolong the recovery time due to the delay in the gas movement as the result of diffusion through the microchannel. This paper aims to reduce the sampling time of a single MFOD. Such reduction is achieved here by developing a pattern recognition system involving: (i) an ensemble of four heterogeneous base machine learning models, (ii) a Long-Short Term Memory Neural Network (LSTM) to construct a portion of the transient response, (iii) features from an additional fast response sensor, and (iv) detachment of MOS from the microchannel for accelerated recovery. Using data collected from methane and ethane as a case study, the results of classification and regression demonstrate the need for only 17.5% of the previously used response time without any loss in accuracy. This reduces the total testing cycle of an MFOD by 60%, further enhancing the ability of selective MFODs by enabling faster detection of gases.

Fault detection of high-speed train axle bearings based on a hybridized physical and data-driven temperature model

As a key component connecting axle and frame, axle bearings of the high-speed train are essential for the safe operation of modern high-speed trains. The current on-board fault detection system, which compares the axle box temperature measured from the sensor to a predefined threshold value, faces problems such as poor adaptability to varied service conditions, undesired high rate of false alarms and missing alarms, and lack of potential to be optimized. To solve these problems, in the present study, a novel hybrid model is proposed which predicts the temperature of a healthy bearing under the given service condition, and based on that detects the bearing fault. The hybrid model includes a physical model reflecting the thermal behavior of the axle box and two data-driven models adopting the same inputs but representing the relationship by means of back-propagation (BP) neural network and long-short term memory neural network (LSTM). After calibrating these models by real-life operation data collected from 2.25 million kilometers and using all results as inputs, a fault discriminant mechanism is established. The performance of this new mechanism is superior than the current on-board system and all stand-alone models in terms of indicators such as precision, recall and F1 score. This way, the study shows that similar hybrid models can be a viable solution for the future on-board fault detection system.

Semantic speech analysis using machine learning and deep learning techniques: a comprehensive review

Human cognitive functions such as perception, attention, learning, memory, reasoning, and problem-solving are all significantly influenced by emotion. Emotion has a particularly potent impact on attention, modifying its selectivity in particular and influencing behavior and action motivation. Artificial Emotional Intelligence (AEI) technologies enable computers to understand a user's emotional state and respond appropriately. These systems enable a realistic dialogue between people and machines. The current generation of adaptive user interference technologies is built on techniques from data analytics and machine learning (ML), namely deep learning (DL) artificial neural networks (ANN) from multimodal data, such as videos of facial expressions, stance, and gesture, voice, and bio-physiological data (such as eye movement, ECG, respiration, EEG, FMRT, EMG, eye tracking). In this study, we reviewed existing literature based on ML and data analytics techniques being used to detect emotions in speech. The efficacy of data analytics and ML techniques in this unique area of multimodal data processing and extracting emotions from speech. This study analyzes how emotional chatbots, facial expressions, images, and social media texts can be effective in detecting emotions. PRISMA methodology is used to review the existing survey. Support Vector Machines (SVM), Naïve Bayes (NB), Random Forests (RF), Recurrent Neural Networks (RNN), Logistic Regression (LR), etc., are commonly used ML techniques for emotion extraction purposes. This study provides a new taxonomy about the application of ML in SER. The result shows that Long-Short Term Memory (LSTM) and Convolutional Neural Networks (CNN) are found to be the most useful methodology for this purpose.

A surface electromyography based hand gesture recognition framework leveraging variational mode decomposition technique and deep learning classifier

Deciphering and classifying surface electromyography (sEMG) signals is highly essential in rehabilitation robotics, myoelectric prosthetic control, sign languages and human–computer-interface. Researchers have generally focused on utilizing multi-channel sEMG signals for gesture classification. However, in case of amputees, the residual muscles are limited and the gesture classification needs to be reframed with minimum sEMG channels. Moreover, another key problem in the gesture classification system is the nonlinear and non-stationary nature of sEMG signals, leading to poor generalization ability. Hence, to address the aforementioned problems, this study puts forward a novel deep learning classifier framework which leverages the potentials of variational mode decomposition (VMD) technique and a hybrid convolutional neural network–long short term memory (CNN–LSTM) classifier model to recognize the hand gestures from single channel sEMG signals. Collecting the sEMG signals from forearm muscles of 25 intact subjects for ten functional and grasping actions, this work implements a VMD technique to identify the prominent frequency modes in the sEMG signals. From the decomposed modes of sEMG signals, the prominent intrinsic mode functions (IMFs) are extracted through spectral analysis to minimize the computation burden on the hybrid classifier model. Furthermore, as the muscle contractions result in substantial temporal dependencies, this work exploits the potentials of CNN and LSTM networks and extracts the spatiotemporal features of the sEMG signals for various hand gestures. The experimental results corroborate that the proposed classifier framework can achieve an average classification accuracy of 98.04% and provide 3% improvement in the classification accuracy compared to conventional CNN classifier.

Enhancing source code retrieval with joint Bi-LSTM-GNN architecture: A comparative study with ChatGPT-LLM

Retrieving relevant source code from large repositories is a significant and ongoing challenge in the field of software engineering, primarily due to the vast and ever-expanding amount of available code. Existing deep learning methods, although effective to some extent, exhibit limitations in capturing the intricate and complex structural information embedded within source code, which hinders their ability to provide highly accurate retrieval results. This study endeavors to tackle this prominent issue by introducing a novel and innovative approach known as the Joint Bi-directional LSTM and Graph Neural Networks (JBLG) model for source code retrieval. The central aim is to harness the combined strengths and capabilities of Bi-directional Long Short-Term Memory (LSTM) networks and Graph Neural Networks (GNNs) to significantly enhance the model’s capacity to capture and interpret the complex structural characteristics intrinsic to source code. The proposed JBLG model employs a unique fusion of Bi-directional LSTM, which excels in capturing sequential and temporal dependencies within code, and GNN, which is adept at modeling the intricate graph structure of the code. By leveraging this hybrid architecture, the model aims to provide a comprehensive and highly effective solution for source code retrieval tasks. To assess the efficacy of the JBLG model, extensive experiments are conducted, and the model’s performance is evaluated against well-established benchmarks, including LSTM, GNN, and ChatGPT, using two diverse datasets: CodeSearchNet and CosBench datasets. These evaluations span multiple programming languages, ensuring a comprehensive and robust assessment of the model’s capabilities. The experimental results indicate that the JBLG model consistently outperforms its counterparts, including Bi-LSTM, GNN, ChatGPT, and DGMS, across various evaluation metrics. the JBLG model showcases an exceptional ability to handle and extract the intricate structural information inherent in source code, resulting in significantly enhanced retrieval accuracy. The JBLG model emerges as a highly promising solution for real-world source code retrieval applications, with the potential to revolutionize the field. The success of this model underscores the importance of combining deep learning techniques like Bi-directional LSTM and GNNs for tackling complex software engineering challenges. Furthermore, future research directions could involve exploring advanced techniques such as attention mechanisms and extending the model’s applicability to other software engineering tasks like code summarization and code completion. The findings of this study are expected to have a lasting impact on the advancement of source code retrieval methodologies.

Evaluation of the impact of intensive PM2.5 reduction policy in Seoul, South Korea using machine learning

The average PM2.5 concentration in South Korea decreased steadily, but the monthly average PM2.5 concentration in January–March increased over time. The Seasonal Fine Dust Management System (hereafter, seasonal management) was implemented from December 2019 to March 2020 in order to reduce high PM2.5 concentrations. This study evaluated the effect of seasonal management using machine learning algorithms, long–short term memory (LSTM), convolutional neural network–LSTM (CNN–LSTM), and ensemble models (Random forest and extreme gradient boosting), and the business–as–usual (BAU) approach. The CNN–LSTM model was defined as four models (multi-headed model 1 and model 2; single-headed model 1 and model 2) according to the method of handling independent variables and the composition of layers. All models performed well; however, the single-headed CNN–LSTM model 1 outperformed the other models. In the BAU period, the observed PM2.5 concentration was 31.98 μg/m3 and the seasonal management reduced PM2.5 concentrations to between 1.09 μg/m3 and 2.73 μg/m3. Overall, the change in PM2.5 concentrations through the seasonal management was apparent during the BAU period, and the machine learning algorithms and BAU approach can be used for the evaluation of policy impact.