Long Short-term Memory Module Research Articles

An Improved CNN-BILSTM Model for Power Load Prediction in Uncertain Power Systems

Power load prediction is fundamental for ensuring the reliability of power grid operation and the accuracy of power demand forecasting. However, the uncertainties stemming from power generation, such as wind speed and water flow, along with variations in electricity demand, present new challenges to existing power load prediction methods. In this paper, we propose an improved Convolutional Neural Network–Bidirectional Long Short-Term Memory (CNN-BILSTM) model for analyzing power load in systems affected by uncertain power conditions. Initially, we delineate the uncertainty characteristics inherent in real-world power systems and establish a data-driven power load model based on fluctuations in power source loads. Building upon this foundation, we design the CNN-BILSTM model, which comprises a convolutional neural network (CNN) module for extracting features from power data, along with a forward Long Short-Term Memory (LSTM) module and a reverse LSTM module. The two LSTM modules account for factors influencing forward and reverse power load timings in the entire power load data, thus enhancing model performance and data utilization efficiency. We further conduct comparative experiments to evaluate the effectiveness of the proposed CNN-BILSTM model. The experimental results demonstrate that CNN-BILSTM can effectively and more accurately predict power loads within power systems characterized by uncertain power generation and electricity demand. Consequently, it exhibits promising prospects for industrial applications.

Research on lung sound classification model based on dual-channel CNN-LSTM algorithm

ulmonary diseases have a significant impact on human health and life safety, and abnormalities in the lungs are a direct response to lung diseases. Establishing an effective lung sound classification model that can assist in diagnosis is of great significance for electronic auscultation.In addressing the issue of lung sound signal classification, this study introduces a deep learning classification model based on a dual-channel CNN-LSTM algorithm. Initially, Mel-scale Frequency Cepstral Coefficients (MFCC) are employed for feature extraction from the dataset, transforming lung sound signals into Mel spectrograms. On this foundation, a dual-channel algorithm classification model is constructed, with parallel Convolutional Neural Network (CNN) and Long Short-Term Memory (LSTM) modules. The CNN module is designed to capture spatial dimension features of the input data, while the LSTM module focuses on temporal dimension features. These two feature sets are fused together, enabling the model to classify lung sounds and thereby assisting in diagnosing pulmonary diseases for healthcare practitioners. This experiment used the ICBHI2017 Challenge Lungs dataset and obtained 5054 pieces of data through data augmentation and sampling techniques.The results show that the accuracy, recall, and F1 score of this model reach 99.01%, 99.13%, and 0.9915, respectively, significantly superior to other models, highlighting its practical application value.

MetaKernel: Learning Variational Random Features With Limited Labels.

Few-shot learning deals with the fundamental and challenging problem of learning from a few annotated samples, while being able to generalize well on new tasks. The crux of few-shot learning is to extract prior knowledge from related tasks to enable fast adaptation to a new task with a limited amount of data. In this paper, we propose meta-learning kernels with random Fourier features for few-shot learning, we call MetaKernel. Specifically, we propose learning variational random features in a data-driven manner to obtain task-specific kernels by leveraging the shared knowledge provided by related tasks in a meta-learning setting. We treat the random feature basis as the latent variable, which is estimated by variational inference. The shared knowledge from related tasks is incorporated into a context inference of the posterior, which we achieve via a long-short term memory module. To establish more expressive kernels, we deploy conditional normalizing flows based on coupling layers to achieve a richer posterior distribution over random Fourier bases. The resultant kernels are more informative and discriminative, which further improves the few-shot learning. To evaluate our method, we conduct extensive experiments on both few-shot image classification and regression tasks. A thorough ablation study demonstrates that the effectiveness of each introduced component in our method. The benchmark results on fourteen datasets demonstrate MetaKernel consistently delivers at least comparable and often better performance than state-of-the-art alternatives.

Flood prediction using nonlinear instantaneous unit hydrograph and deep learning: A MATLAB program

In this study, we developed a MATLAB program for flood prediction in a watershed. The program consists of three modules. The instantaneous unit hydrograph (IUH) generation module utilizes a power-law based interpolation method to generate IUHs. The generated IUH is a function of the rainfall excess intensity and therefore considers nonlinearity. The long short-term memory (LSTM) module employs “lstmLayer” from the MATLAB deep learning toolbox to predict total rainfall excess; this is then used to estimate the curve number (CN) value for each flood event. The LSTM module uses a land surface modeling dataset and rainfall-runoff data as inputs. The flood hydrograph generation module calculates effective rainfall hyetographs and then predicts flood hydrographs using a convolution integration. A detailed description of the program is provided along with an application example for real watersheds. The application results demonstrated that our program can be effectively used for flood prediction in practice, particularly for large flood events.

A hybrid cloud load balancing and host utilization prediction method using deep learning and optimization techniques

Virtual machine (VM) integration methods have effectively proven an optimized load balancing in cloud data centers. The main challenge with VM integration methods is the trade-off among cost effectiveness, quality of service, performance, optimal resource utilization and compliance with service level agreement violations. Deep Learning methods are widely used in existing research on cloud load balancing. However, there is still a problem with acquiring noisy multilayered fluctuations in workload due to the limited resource-level provisioning. The long short-term memory (LSTM) model plays a vital role in the prediction of server load and workload provisioning. This research presents a hybrid model using deep learning with Particle Swarm Intelligence and Genetic Algorithm (“DPSO-GA”) for dynamic workload provisioning in cloud computing. The proposed model works in two phases. The first phase utilizes a hybrid PSO-GA approach to address the prediction challenge by combining the benefits of these two methods in fine-tuning the Hyperparameters. In the second phase, CNN-LSTM is utilized. Before using the CNN-LSTM approach to forecast the consumption of resources, a hybrid approach, PSO-GA, is used for training it. In the proposed framework, a one-dimensional CNN and LSTM are used to forecast the cloud resource utilization at various subsequent time steps. The LSTM module simulates temporal information that predicts the upcoming VM workload, while a CNN module extracts complicated distinguishing features gathered from VM workload statistics. The proposed model simultaneously integrates the resource utilization in a multi-resource utilization, which helps overcome the load balancing and over-provisioning issues. Comprehensive simulations are carried out utilizing the Google cluster traces benchmarks dataset to verify the efficiency of the proposed DPSO-GA technique in enhancing the distribution of resources and load balancing for the cloud. The proposed model achieves outstanding results in terms of better precision, accuracy and load allocation.

Fusion Neural Network for Gas Concentration Prediction in Mixed Gas Environments

Due to the inherent complexity and nonlinearity of mixed gas data, existing pattern recognition algorithms utilized in electronic noses often encounter difficulties in accurately predicting gas concentrations. Addressing this issue, we propose a fusion neural network that merges Long Short-Term Memory (LSTM) and Temporal Convolutional Network (TCN), which we denote as the LSTM-TCN fusion model. The LSTM module effectively captures long-term dependencies in time-series data, while the TCN targets local correlations, thereby enhancing the prediction accuracy for complex gas concentrations. Experimental validation was conducted using a mixed gas dataset comprising ethylene and carbon monoxide. When compared with traditional models, including LSTM, TCN, and GRU, the proposed LSTM-TCN model demonstrated superior performance, achieving an R2 value as high as 0.9922. This research holds considerable practical significance and shows promising application prospects, contributing novel insights and methods to the study and application of electronic nose technology.

An Expressway ETC Missing Data Restoration Model Considering Multi-Attribute Features

Electronic toll collection (ETC) data mining has become one of the hotspots in the research of intelligent expressway extension applications. Ensuring the integrity of ETC data stands as a critical measure in upholding data quality. ETC data are typical structured data, and although deep learning holds great potential in the ETC data restoration field, its applications in structured data are still in the early stages. To address these issues, we propose an expressway ETC missing transaction data restoration model considering multi-attribute features (MAF). Initially, we employ an entity embedding neural network (EENN) to automatically learn the representation of categorical features in multi-dimensional space, subsequently obtaining embedding vectors from networks that have been adequately trained. Then, we use long short-term memory (LSTM) neural networks to extract the changing patterns of vehicle speeds across several continuous sections. Ultimately, we merge the processed features with other features as input, using a three-layer multilayer perceptron (MLP) to complete the ETC data restoration. To validate the effectiveness of the proposed method, we conducted extensive tests using real ETC datasets and compared it with methods commonly used for structured data restoration. The experimental results demonstrate that the proposed method significantly outperforms others in restoration accuracy on two different datasets. Specifically, our sample data size reached around 400,000 entries. Compared to the currently best method, our method improved the restoration accuracy by 19.06% on non-holiday ETC datasets. The MAE and RMSE values reached optimal levels of 12.394 and 23.815, respectively. The fitting degree of the model to the dataset also reached its peak (R2 = 0.993). Meanwhile, the restoration stability of our method on holiday datasets increased by 5.82%. An ablation experiment showed that the EENN and LSTM modules contributed 7.60% and 9% to the restoration accuracy, as well as 4.68% and 7.29% to the restoration stability. This study indicates that the proposed method not only significantly improves the quality of ETC data but also meets the timeliness requirements of big data mining analysis.

A novel deep time‐index framework with limited data and feature extraction for battery capacity degradation prediction

AbstractAccurate capacity estimation and degradation trajectory predictions are crucial part of effectively monitoring the current and future health status of lithium‐ion batteries, which ensures their optimal performance and longevity. However, it is hard to acquire precise predictions under various application conditions when relying on limited historical data. In this paper, we propose a novel deep time‐index framework based on meta‐learning to achieve online capacity estimation and degradation trajectory predictions with only a few monitoring data from early cycles. Highly correlated health feature from lithium‐ion battery degradation data is extracted for online capacity estimation. The deep time‐index framework consists of three parts, Implicit Neural Representations (INRs), Long Short‐Term Memory (LSTM) and Ridge regressor. Cycle numbers are selected as time feature and fed into the INRs to obtain informative representations. A LSTM module is employed to process the temporal representations and capture the long‐term dependencies to Ridge regressor to generate a direct multi‐step predictions. Three distinct lithium‐ion battery degradation datasets are applied to validate the efficacy of the proposed approach. The experimental results validate that within the deep time‐index framework, the short‐term predictions show remarkable performance. This high level of accuracy makes the proposed deep time‐index framework suitable for a wide range of real‐world scenarios, as it demonstrates precise predictions even under varying charging and discharging conditions with limited historical data.

Improved Deep Reinforcement Learning for Intelligent Traffic Signal Control Using ECA_LSTM Network

Reinforcement learning is one of the most widely used methods for traffic signal control, but the method experiences issues with state information explosion, inadequate adaptability to special scenarios, and low security. Therefore, this paper proposes a traffic signal control method based on the efficient channel attention mechanism (ECA-NET), long short-term memory (LSTM), and double Dueling deep Q-network (D3QN), which is EL_D3QN. Firstly, the ECA-NET and LSTM module are included in order to lessen the state space’s design complexity, improve the model’s robustness, and adapt to various emergent scenarios. As a result, the cumulative reward is improved by 27.9%, and the average queue length, average waiting time, and CO2 emissions are decreased by 15.8%, 22.6%, and 4.1%, respectively. Next, the dynamic phase interval tgap is employed to enable the model to handle more traffic conditions. Its cumulative reward is increased by 34.2%, and the average queue length, average waiting time, and CO2 emissions are reduced by 19.8%, 30.1%, and 5.6%. Finally, experiments are carried out using various vehicle circumstances and unique scenarios. In a complex environment, EL_D3QN reduces the average queue length, average waiting time, and CO2 emissions by at least 13.2%, 20.2%, and 3.2% compared to the four existing methods. EL_D3QN also exhibits good generalization and control performance when exposed to the traffic scenarios of unequal stability and equal stability. Furthermore, even when dealing with unique events like a traffic surge, EL_D3QN maintains significant robustness.

Remaining useful life prediction method combining the life variation laws of aero-turbofan engine and auto-expandable cascaded LSTM model

The real-time status monitoring and health management of aero-turbofan engines (ATEs) can effectively reduce the risk of engine failure and ensure aircraft flight safety. Accurate prediction of the remaining useful life (RUL) of ATE is a vital tool for effectively monitoring the engine operating condition, in which long-short term memory (LSTM) networks are often applied to RUL prediction. However, because of the complex mechanical structure and operation mode of the aero-engine, the prediction accuracy of the LSTM network is not enough to meet the actual demand. This paper proposes an auto-expandable cascaded long-short term memory (ACLSTM) prediction model that incorporates the lifetime variation laws of ATE, which is mainly applied for the degradation assessment of ATEs and the accurate prediction of RUL. The ACLSTM model adopts the network structure of multiple LSTM modules connected step by step to continuously set the prediction error of the previous module as the training outputs of the latter module. This data processing method transforms the prediction process of the original data into that of the output error, effectively reducing the prediction error and improving the prediction effect. In addition, to further improve the prediction accuracy, this paper comprehensively proposes several empirical formulas for further correction of the prediction effect obtained by the ACLSTM model. In the experimental part, the prediction effectiveness of the proposed method is tested based on four subsets of the C-MAPSS dataset published by the National Aeronautics and Space Administration. The experimental results on the four datasets show that the root mean square error (RMSE) of the ACLSTM prediction model decreases by 95.44% on average compared to the traditional LSTM network. In addition, the RMSE of the model decreases by 96.48% on average after incorporating the empirical formula. The proposed method has the lowest RMSE compared to other methods with the highest prediction accuracy. The experiments thoroughly verify that the ACLSTM model and the proposed empirical formula are feasible and effective for improving the prediction accuracy of the RUL of ATE.

MTDN: Learning Multiple Temporal Dynamics Representation for Emotional Valence Classification with EEG.

Emotion recognition from electroencephalogram (EEG) requires computational models to capture the crucial features of the emotional response to external stimulation. Spatial, spectral, and temporal information are relevant features for emotion recognition. However, learning temporal dynamics is a challenging task, and there is a lack of efficient approaches to capture such information. In this work, we present a deep learning framework called MTDN that is designed to capture spectral features with a filterbank module and to learn spatial features with a spatial convolution block. Multiple temporal dynamics are jointly learned with parallel long short-term memory (LSTM) embedding and self-attention modules. The LSTM module is used to embed the time segments, and then the self-attention is utilized to learn the temporal dynamics by intercorrelating every embedded time segment. Multiple temporal dynamics representations are then aggregated to form the final extracted features for classification. We experiment on a publicly available dataset, DEAP, to evaluate the performance of our proposed framework and compare MTDN with existing published results. The results demonstrate improvement over the current state-of-the-art methods on the valence dimension of the DEAP dataset.

Supervised Contrastive Learned Deep Model for Question Continuation Evaluation

Question continuation evaluation (QCE) is a branch task of dialogue act prediction (DAP) in the natural language processing area, which is aimed at predicting whether each question in a dialogue is worthy of being followed-up under a specific context. QCE is important for communication, education, and even entertainment. Regrettably, QCE has always been disregarded as an auxiliary task for conversational machine reading comprehension. QCE involves more information and relationships than the original DAP task, making it more complex. Moreover, the classification of QCE inherently renders the samples confusing. In this article, a transformer long short-term memory (LSTM)-based supervised contrastive learned model for QCE is proposed to automatically distribute QCE labels. This model is mainly constructed with transformer encoder blocks and LSTM modules, and supervised contrastive learning (SCL) is innovatively introduced to the training process. This model is good at extracting both information about corpora and the relationships among corpora, and SCL alleviates any confusion. With the only applicable dataset, i.e., Question Answering in Context (QuAC), experiments are conducted. This model is proven to perform well and is robust to missing data. The performance is 2.3% (accuracy) and 12.2% (macro-F1 score) higher than baselines from QuAC and only decreases by approximately 2.3% when 10% data remain.

Unified Framework for Identity and Imagined Action Recognition From EEG Patterns

We present a unified deep learning framework for the recognition of user identity and the recognition of imagined actions, based on electroencephalography (EEG) signals, for application as a brain–computer interface. Our solution exploits a novel shifted subsampling preprocessing step as a form of data augmentation, and a matrix representation to encode the inherent local spatial relationships of multielectrode EEG signals. The resulting image-like data are then fed to a convolutional neural network to process the local spatial dependencies, and eventually analyzed through a bidirectional long-short term memory module to focus on temporal relationships. Our solution is compared against several methods in the state of the art, showing comparable or superior performance on different tasks. Specifically, we achieve accuracy levels above 90% both for action and user classification tasks. In terms of user identification, we reach 0.39% equal error rate in the case of known users and gestures, and 6.16% in the more challenging case of unknown users and gestures. Preliminary experiments are also conducted in order to direct future works toward everyday applications relying on a reduced set of EEG electrodes.

Development and Experimental Verification of an Intelligent Isolation System Based on Long Short-Term Memory Module Model for Ground Motion Characteristics Prediction

Semiactive seismic control requires appropriate control laws that dictate the behavior of seismic suppression devices based on measurements and feedback during an earthquake. Optimal control parameters should be determined in real time to achieve high performance. The initial ground motion characteristics of an earthquake substantially affect control performance. In this study, a genetic algorithm (GA)-optimized long short-term memory (LSTM)-based intelligent control system (hereafter denoted GA-LSTM system) for obtaining optimal control parameters for a semiactive variable-stiffness isolation system was proposed. First, the LSTM module classifies earthquakes as near-fault or far-field events to determine the optimal control strategy. Second, earthquakes from a global database are analyzed to determine a fuzzy inference surface for the optimal parameters of the earthquake suppression system. Both numerical simulations and shaking table experimental results indicated that the proposed GA-LSTM system exhibited superior isolation displacement and superstructure acceleration suppression for both near-fault and far-field earthquakes when compared with other control techniques. The proposed intelligent control system is highly efficient and could reliably protect structures from earthquakes with any ground motion characteristics.

A Deep Learning Model for Ship Trajectory Prediction Using Automatic Identification System (AIS) Data

The rapid growth of ship traffic leads to traffic congestion, which causes maritime accidents. Accurate ship trajectory prediction can improve the efficiency of navigation and maritime traffic safety. Previous studies have focused on developing a ship trajectory prediction model using a deep learning approach, such as a long short-term memory (LSTM) network. However, a convolutional neural network (CNN) has rarely been applied to extract the potential correlation among different variables (e.g., longitude, latitude, speed, course over ground, etc.). Therefore, this study proposes a deep-learning-based ship trajectory prediction model (namely, CNN-LSTM-SE) that considers the potential correlation of variables and temporal characteristics. This model integrates a CNN module, an LSTM module and a squeeze-and-excitation (SE) module. The CNN module is utilized to extract data on the relationship among different variables (e.g., longitude, latitude, speed and course over ground), the LSTM module is applied to capture temporal dependencies, and the SE module is introduced to adaptively adjust the importance of channel features and focus on the more significant ones. Comparison experiments of two cargo ships at a time interval of 10 s show that the proposed CNN-LSTM-SE model can obtain the best prediction performance compared with other models on evaluation indexes of average root mean squared error (ARMSE), average mean absolute percentage error (AMAPE), average Euclidean distance (AED), average ground distance (AGD) and Fréchet distance (FD).

Developing machine learning methods for automatic recognition of fishing vessel behaviour in the Scomber japonicus fisheries

IntroductionWith a higher degree of automation, fishing vessels have gradually begun adopting a fishing monitoring method that combines human and electronic observers. However, the objective data of electronic monitoring systems (EMS) has not yet been fully applied in various fishing boat scenarios such as ship behavior recognition.MethodsIn order to make full use of EMS data and improve the accuracy of behaviors recognition of fishing vessels, the present study proposes applying popular deep learning technologies such as convolutional neural network, long short-term memory, and attention mechanism to Chub mackerel (Scomber japonicus) fishing vessel behaviors recognition. The operation process of Chub mackerel fishing vessels was divided into nine kinds of behaviors, such as “pulling nets”, “putting nets”, “fish pick”, “reprint”, etc. According to the characteristics of their fishing work, four networks with different convolutional layers were designed in the pre-experiment. And the feasibility of each network in behavior recognition of the fishing vessels was observed. The pre-experiment is optimized from the perspective of the data set and the network. From the standpoint of the data set, the size of the optimized data set is significantly reduced, and the original data characteristics are preserved as much as possible. From the perspective of the network, different combinations of pooling, long short-term memory(LSTM) network, and attention(including CBAM and SE) are added to the network, and their effects on training time and recognition effect are compared.ResultsThe experimental results reveal that the deep learning methods have outstanding performance in behaviors recognition of fishing vessels. The LSTM and SE module combination produced the most apparent optimization effect on the network, and the optimized model can achieve an F1 score of 97.12% in the test set, surpassing the classic ResNet, VGGNet, and AlexNet.DiscussionThis research is of great significance to the management of intelligent fishery vessels and can promote the development of electronic monitoring systems for ships.

HFCM-LSTM: A novel hybrid framework for state-of-health estimation of lithium-ion battery

Accurate estimating the state of health (SOH) of lithium-ion battery plays a significant role in the safe operation of electric vehicles. With the development of deep learning, neural network-based methods have attracted much attention from researchers. While most of the existing SOH estimation methods are built by a single network, failing to sufficiently extract data information, and thus leading to the limited accuracy and generality (i.e., such a single network makes it difficult to estimate the SOH of battery, with different types and working conditions). Towards this issue, a novel hybrid network, called HFCM (Hierarchical Feature Coupled Module)-LSTM (Long–short-term memory), is designed to fully extract the original data information, making it more accurate to estimate the SOH of battery, with different types and working conditions. Specifically, the proposed HFCM-LSTM mainly consists of two components, HCFM and LSTM. The HCFM is proposed to comprehensively extract the original data feature information from the original samples. On the other hand, following the HFCM, a LSTM module is employed to model time series information. Based on this HFCM-LSTM network, the data obtained directly from the battery are fed into the model as input, enabling an end-to-end SOH estimation of the battery. A series of experiments are conducted on both NASA dataset and Oxford dataset, the experimental results demonstrate that the proposed SOH estimation algorithm outperforms several existing state-of-the-art methods, in terms of accuracy and generality.

Spatiotemporal Co-Attention Hybrid Neural Network for Pedestrian Localization Based on 6D IMU

In this paper, we propose spatiotemporal co-attention hybrid neural network (SC-HNN), a novel hybrid neural network model with both spatial and temporal attention mechanisms for pose-invariant inertial odometry. The main idea is to extract both local and global features from a window of IMU measurements for velocity prediction. SC-HNN leverages the convolutional neural network (CNN) to capture the sectional features and long short-term memory (LSTM) recurrent neural network (RNN) to extract the long-range dependencies. Attention mechanisms are designed and embedded in both CNN and LSTM modules for better model representation. Specifically, in the CNN attention block, the convolved features are refined along both channel and element dimensions. For the LSTM module, softmax scoring is applied to update the weights of the hidden states along the temporal axis. We evaluate SC-HNN on the benchmark with the largest and most natural IMU data, RoNIN. Extensive ablation experiments demonstrate the effectiveness of our SC-HNN model. Compared with the state of the art, the 50th percentile accuracy of SC-HNN is 18.21% higher and the 90th percentile accuracy is 21.15% higher for all the phone holders not appeared in the training set. The real scenario inertial tracking trials in the CUHK campus further prove the superior generalization ability of the SC-HNN model. Note to Practitioners—This paper aims at improving the localization accuracy of deep inertial odometry. We focus on the problem of indoor localization only from the low-cost IMU embedded in the smartphone without any restriction on the phone’s daily use. IMU is a perfect solution for indoor localization because of its low power consumption, high privacy protection, and external infrastructure free. This paper suggests a novel hybrid convolutional and recurrent neural network with a set of carefully designed attention mechanisms to improve the representation ability of deep inertial odometry model. Specifically, the convolutional layer is applied to extract the local spatial features among the 6D IMU signals, following a cascaded channel attention module and element attention module to boost the representation ability of CNN. The complex long-term dependencies are then identified by the LSTM layers. To adaptively capture the temporal features of the multimodal inertial signals, an attention mechanism is applied to weigh the hidden states for the generation of the final features. The effectiveness of the SC-HNN design is validated by extensive ablation studies. To the best of our knowledge, our model is the first HNN fused attention mechanism for inertial tracking. Extensive experiments show that the proposed method outperforms the state of the art.

EEG-Based Emotion Recognition Using Spatial-Temporal Graph Convolutional LSTM With Attention Mechanism.

The dynamic uncertain relationship among each brain region is a necessary factor that limits EEG-based emotion recognition. It is a thought-provoking problem to availably employ time-varying spatial and temporal characteristics from multi-channel electroencephalogram (EEG) signals. Although deep learning has made remarkable achievements in emotion recognition, the biological topological information among brain regions does not fully exploit, which is vital for EEG-based emotion recognition. In response to this problem, we design a hybrid model called ST-GCLSTM, which comprises a spatial-graph convolutional network (SGCN) module and an attention-enhanced bi-directional Long Short-Term Memory (LSTM) module. The main advantage of ST-GCLSTM is that it can consider the biological topology information of each brain region to extract representative spatial-temporal features from multiple EEG channels. Specifically, we construct two layers SGCN by introducing adjacency matrices to adaptively learn the intrinsic connection among different EEG channels. Moreover, an attention-enhanced mechanism is placed into a bi-directional LSTM module to extract the crucial spatial-temporal features from sequential EEG data, and then these features serve as the input layer of the classifier to learn discriminative emotion-related features. Extensive experiments on the DEAP, SEED, and SEED-IV datasets demonstrate the effectiveness of the proposed ST-GCLSTM model, revealing that our model had an absolute performance improvement over state-of-the-art strategies.

Effectiveness of the use of algorithms and methods of artificial technologies for sign language recognition for people with disabilities

According to WHO, the number of people with disabilities in the world has exceeded 1 billion. At the same time, 80 percent of all people with disabilities live in developing countries. In this regard, the demand for the use of applications for people with disabilities is growing every day. The paper deals with neural network methods like MediaPipe Holistic and the LSTM module for determining the sign language of people with disabilities. MediaPipe has demonstrated unprecedented low latency and high tracking accuracy in real-world scenarios thanks to built-in monitoring solutions. Therefore, MediaPipe Holistic was used in this work, which combines pose, hand, and face control with detailed levels. The main purpose of this paper is to show the effectiveness of the HAR algorithm for recognizing human actions, based on the architecture of in-depth learning for classifying actions into seven different classes. The main problem of this paper is the high level of recognition of the sign language of people with disabilities when implementing their work in cross-platform applications, web applications, social networks that facilitate the daily life of people with disabilities and interact with society. To solve this problem, an algorithm was used that combines the architecture of a convolutional neural network (CNN) and long short-term memory (LSTM) to study spatial and temporal capabilities from three-dimensional skeletal data taken only from a Microsoft Kinect camera. This combination allows you to take advantage of LSTM when modeling temporal data and CNN when modeling spatial data. The results obtained based on calculations carried out by adding a new layer to the existing model showed higher accuracy than calculations carried out on the existing model.