LSTM Network Research Articles

Advanced Machine Learning-Driven Security and Anomaly Identification in Inverter-Based Cyber-Physical Microgrids

In a landscape dominated by the digitalization of energy networks, safeguarding cyber-physical microgrids and against breaches anomalies emerges as a critical imperative. This study pioneers inventive methodologies for Intelligent Detection of Intrusions and Anomalies in Inverter-centric Cyber-Physical Microgrids, leveraging state-of-the-art machine learning paradigms. Using the combined power of LSTM and Convolutional Neural Networks, our suggested approach finds breaches and aberrations in microgrid structures with remarkable accuracy and efficiency. Additionally, we integrate the effectiveness of Gradient Boosting Machines to enhance the overall detection capabilities. Experimental findings underscore the efficacy of our machine learning-driven approach, with CNN achieving a precision of 95%, recall of 92%, and F1-score of 93% in intrusion detection, while LSTM attains a precision of 93%, recall of 91%, and F1-score of 92% in anomaly identification. Furthermore, GBM contributes to achieving an overall efficiency of 96%. Our approach not only bolsters microgrid resilience but also lays the foundation for a secure energy future. In an increasingly interconnected world, the integration of these sophisticated techniques not only bolsters microgrid resilience but also lays the foundation for a secure energy future. By embracing innovation at the convergence of machine learning and energy systems, we stride toward fortified and dependable cyber-physical microgrids.

DEMA: A Deep Learning-Enabled Model for Non-Invasive Human Vital Signs Monitoring Based on Optical Fiber Sensing.

Optical fiber sensors are extensively employed for their unique merits, such as small size, being lightweight, and having strong robustness to electronic interference. The above-mentioned sensors apply to more applications, especially the detection and monitoring of vital signs in medical or clinical. However, it is inconvenient for daily long-term human vital sign monitoring with conventional monitoring methods under the uncomfortable feelings generated since the skin and devices come into direct contact. This study introduces a non-invasive surveillance system that employs an optical fiber sensor and advanced deep-learning methodologies for precise vital sign readings. This system integrates a monitor based on the MZI (Mach-Zehnder interferometer) with LSTM networks, surpassing conventional approaches and providing potential uses in medical diagnostics. This could be potentially utilized in non-invasive health surveillance, evaluation, and intelligent health care.

A Prolonged Artificial Nighttime-light Dataset of China (1984-2020)

Nighttime light remote sensing has been an increasingly important proxy for human activities. Despite an urgent need for long-term products and pilot explorations in synthesizing them, the publicly available long-term products are limited. A Night-Time Light convolutional LSTM network is proposed and applied the network to produce a 1-km annual Prolonged Artificial Nighttime-light DAtaset of China (PANDA-China) from 1984 to 2020. Assessments between modeled and original images show that on average the RMSE reaches 0.73, the coefficient of determination (R2) reaches 0.95, and the linear slope is 0.99 at the pixel level, indicating a high confidence in the quality of generated data products. Quantitative and visual comparisons witness PANDA-China’s superiority against other NTL datasets in its significantly longer NTL dynamics, higher temporal consistency, and better correlations with socioeconomics (built-up areas, gross domestic product, population) characterizing the most relevant indicator in different development phases. The PANDA-China product provides an unprecedented opportunity to trace nighttime light dynamics in the past four decades.

A Meta heuristic based deep learning classifier for effective dengue disease prediction in IoT‐Fog system

AbstractThe primary aim of this article is to propose an effective dengue disease monitoring system by integrating deep learning models with Internet of Things (IoT) and fog computing. In this context, the disease related parameters are collected using IoT devices and further, the reports are securely transmitted to the healthcare facilities utilizing fog computing. In this article, the misdiagnosis error is reduced utilizing attention based long short term memory (ALSTM) network with coati optimization algorithm (COA). The Attention model allocates higher weight to significant parameters and pay additional consideration while training the model. Therefore, the conventional LSTM network is combined with a self‐attention mechanism for enabling this model to focus on relevant parts of the input sequences for precise dengue disease prediction. The Attention Model differs from the conventional model since it delivers higher amount of data into the decoder and improves additional stages to the attention decoder before making its output. Furthermore, the COA is integrated with the ALSTM network for optimal hyper‐parameter selection. The ALSTM‐COA model evaluates the dengue related parameters like nausea, vomiting, skin rash, joint pain, muscle pain, soft bleeding, temperature of water‐sites, humidity of water‐sites, and so forth, for timely dengue disease diagnosis and clinical decision‐making. The experiments performed on a real time dataset state that the ALSTM‐COA model achieves significant prediction results by utilizing different performance measures like recall, accuracy, fall‐out, miss rate, training time, and testing time. The ALSTM‐COA model obtains an accuracy of 99.27%, latency of 0.03 s, and time complexity of 8 s for 500 health records, which are superior to the comparative deep learning models.

Forecasting Energy Consumption from EV Station Charging Using RNN, LSTM and GRU Neural Network

The increase in electric vehicles (EVs) has resulted in a substantial escalation in electricity consumption. This increased demand puts more stress on the overall power system. The current study offers a method to predict energy usage patterns by looking closely at when electric vehicles typically need to charge during the day. After that, the collected data were used to create a predictive model using three different deep learning methods: Recurrent Neural Networks (RNNs), Long Short-Term Memory networks (LSTMs), and Gated Recurrent Units (GRUs). This study employs data pertaining to electric power consumption for EVs charging derived from Kamphaeng Phet Rajabhat University. The practical results show that the proposed model significantly outperforms in predicting power needs at the mentioned charging spots. This is evident in its precise prediction of the total power demands using the algorithm. Among the three types of deep learning structures studied, it's clear that the LSTMs type stands out as the best, achieving the most accurate results. This is supported by a Root Mean Square Error (RMSE) of 0.372 and a Mean Absolute Percentage Error (MAPE) of 11.508%. Additionally, the inquiry facilitates a comprehensive comparison between the dynamics of demand and the parameters of supply. This process yields data that offers valuable insights crucial for the strategic identification of potential electric vehicle charging stations. It also enables the prudent utilization of remaining electrical capacity derived from production processes. These combined efforts converge to ensure the utmost extraction of utility.

Dual-Stage Attention-Based LSTM Network for Multiple Time Steps Flood Forecasting

Dual-Stage Attention-Based LSTM Network for Multiple Time Steps Flood Forecasting

Optimization Strategies for Real-time Energy Management of Electric Vehicles Based on LSTM Network Learning

The orderly control of electric vehicle load can improve the load characteristics of regional power grid and reduce the charging cost. Since it is impossible to predict the accurate access time and charging demand of electric vehicles in the future, it is impossible to make a global optimal arrangement for accessing the grid when electric vehicles are charging. Aiming at this problem, a real-time energy management system and optimization strategy of electric vehicle based on deep long-term and short-term memory neural network are proposed. Firstly, a three-tier electric vehicle management architecture including power grid layer, regional energy management system and charging station energy management system is constructed to manage large-scale electric vehicles in layers and regions; Then, a region station interaction strategy based on deep long-term and short-term memory neural network is proposed. The historical optimal solution solved by historical load information is used to train the learning network to guide the new real-time optimization; The proposed strategy can further reduce the charging cost and improve the peak valley characteristics of regional load on the premise of ensuring the charging demand of users. Finally, a simulation example is given to verify the effectiveness and superiority of the proposed layered architecture and management strategy.

Physical mechanism-corrected degradation trend prediction network under data missing

Accurate degradation trend prediction (DTP) is crucial for optimizing equipment operation and maintenance, thereby boosting production efficiency. This study introduces a novel Data Repair and Dual-data-stream LSTM (DR-DLSTM) network to tackle the challenge of missing data in equipment DTP. The proposed DR-DLSTM framework employs convex optimization to consider both the trend and periodic variations in the data, incorporating polynomial and trigonometric functions into the implicit feature matrix to construct latent vectors for missing data rectification. The network features a Dual-LSTM block with dual data streams to enhance feature extraction, with two gating update units correlating time series components and redistributing feature weights. The Dual-LSTM enables separate and accurate prediction of trend and periodic components, thereby enhancing the feature extraction capability of the prediction model. Additionally, the integration of physical rule information through Fourier and wavelet transform frequency correction modules allows for dynamic adjustments in prediction outcomes, from global trends to localized details. The DR-DLSTM's effectiveness is demonstrated through comprehensive comparisons with state-of-the-art models across multiple datasets, highlighting its superior performance. The results demonstrate the superiority of the proposed model. These algorithms were implemented in Python using Torch on a 2.9 GHz Intel I7 CPU and TITAN Xp GPU.

Stock Price Prediction and Pattern Detection Using Deep Learning

Accurately predicting stock market movements remains a significant challenge, yet one that continues to attract intense interest. This project leverages data mining and warehousing techniques to explore historical stock price data, aiming to uncover recurring patterns, forecast potential future trends, and evaluate the consistency of a specific stock's behaviour relative to its identified patterns. By employing various data mining algorithms and implementing effective warehousing strategies, the project seeks to extract valuable insights that can inform investment decisions and potentially contribute to superior market performance. But among these goals are some serious obstacles. Accurate prediction is hampered by the noise and uncertainties that are present in stock market data. Furthermore, fine-tuning and experimentation are necessary to achieve optimal results when optimizing the hyperparameters of long short-term memory networks (LSTM networks). Moreover, strong preprocessing methods are required to handle datasets that contain inconsistent or missing data. Lastly, proficient data visualization and user interface design abilities are required when creating an educational and engaging dashboard with Plotly Dash.

Multi-temporal Scale Wind Power Forecasting Based on Lasso-CNN-LSTM-LightGBM

Due to the increasingly severe climate problems, wind energy has received widespread attention as the most abundant energy on Earth. However, due to the uncertainty of wind energy, a large amount of wind energy is wasted, so accurate wind power prediction can greatly improve the utilization of wind energy. To increase the forecast for wind energy accuracy across a range of time scales, this paper presents a multi-time scale wind power prediction by constructing an ICEEMDAN-CNN-LSTM-LightGBM model. Initially, feature selection is performed using Lasso regression to identify the most significant variables affecting the forecast for wind energy across distinct time intervals. Subsequently, the ICEEMDAN is utilized to break down the wind power data into various scales to capture its nonlinear and non-stationary characteristics. Following this, a deep learning model based on CNN and LSTM networks is developed, with the CNN responsible for extracting spatial features from the time series data, and the LSTM designed to capture the temporal relationships. Finally, the outputs of the deep learning model are fed into the LightGBM model to leverage its superior learning capabilities for the ultimate prediction of wind power. Simulation experiments demonstrate that the proposed ICEEMDAN-CNN-LSTM-LightGBM model achieves higher accuracy in multi-time scale wind power prediction, providing more reliable decision assistance with the management and operation of wind farms.

Utilizing LSTM Networks for the Prediction of Driver Behavior

Utilizing LSTM Networks for the Prediction of Driver Behavior

A View on technology and controls for epidemic preparedness and the growth of COVID-19 disease in Saudi Arabia

This work emphasizes that the best chance of ending the pandemic lies in the Measures that can help us respond to these epidemic threats more effectively than before in the scope of protection from epidemic risks that we were not aware of before the outbreak of the disease, which is to benefit from the study of previous experiences. The work show the importance of technology about how to prepare for and handle any pandemic by discussing a successful case from the past. We examine Saudi Arabia's COVID-19 pandemic response experiences. We proposed a case evolution study from the southern region of Jazan to demonstrate the consequences of these techniques. Data on the daily Jazan COVID-19 infection curve collected from July 1st to September 30th, 2020, was processed using multiple Python software modules, such as encoder-decoder LSTM, 1D-CNN, and Complex network analysis of time series to identify changes within the infection curve's structure. A Gaussian modeling study was performed to compare the data movement with the protection from epidemic risks computed by stratification based on the data group.The network analysis shows that hubs are stable while medium and low-degree nodes are unstable. Also, from the perspective of Gaussian modeling, the COVID-19 infection curve parts indicate that the pandemic in Saudi Arabia is about to peak and move into the second (declining) half of the bell-shaped distribution.Technology in many areas and internet access make Saudi Arabia's social separation strategies a pandemic-eradication success story

PredictOptiCloud: A hybrid framework for predictive optimization in hybrid workload cloud task scheduling

In the realm of e-commerce, the growing complexity of dynamic workloads and resource management poses a substantial challenge for platforms aiming to optimize user experiences and operational efficiency. To address this issue, the PredictOptiCloud framework is introduced, offering a solution that combines sophisticated methodologies with comprehensive performance analysis. The framework encompasses a domain-specific approach that extracts and processes historical workload data, utilizing Domain-specific Hierarchical Attention Bi LSTM networks. This enables PredictOptiCloud to effectively predict and manage both stable and dynamic workloads. Furthermore, it employs the Spider Wolf Optimization (SWO) for load balancing and offloading decisions, optimizing resource allocation and enhancing user experiences. The performance analysis of PredictOptiCloud involves a multifaceted evaluation, with key metrics including response time, throughput, resource utilization rate, cost-efficiency, conversion rate, rate of successful task offloading, precision, accuracy, task volume, and churn rate. By meticulously assessing these metrics, PredictOptiCloud demonstrates its prowess in not only predicting and managing workloads but also in optimizing user satisfaction, operational efficiency, and cost-effectiveness, ultimately positioning itself as an invaluable asset for e-commerce platforms striving for excellence in an ever-evolving landscape.

Drones Detection Using a Fusion of RF and Acoustic Features and Deep Neural Networks.

The use of drones has recently gained popularity in a diverse range of applications, such as aerial photography, agriculture, search and rescue operations, the entertainment industry, and more. However, misuse of drone technology can potentially lead to military threats, terrorist acts, as well as privacy and safety breaches. This emphasizes the need for effective and fast remote detection of potentially threatening drones. In this study, we propose a novel approach for automatic drone detection utilizing the usage of both radio frequency communication signals and acoustic signals derived from UAV rotor sounds. In particular, we propose the use of classical and deep machine-learning techniques and the fusion of RF and acoustic features for efficient and accurate drone classification. Distinct types of ML-based classifiers have been examined, including CNN- and RNN-based networks and the classical SVM method. The proposed approach has been evaluated with both frequency and audio features using common drone datasets, demonstrating better accuracy than existing state-of-the-art methods, especially in low SNR scenarios. The results presented in this paper show a classification accuracy of approximately 91% at an SNR ratio of -10 dB using the LSTM network and fused features.

Adaptive control for uncrewed aerial vehicles based on communication information optimization in complex environments.

The utilization of drone technology thrives in diverse domains, including aviation, military operations, and logistics. The pervasive adoption of this technology aims to enhance efficiency while mitigating hazards and expenditures. In complex contexts, the governing parameters of uncrewed aerial vehicles (UAV) require real-time adjustments for flight safety and efficacy. To improve the attitude estimation accuracy, this article introduces a ATT-Bi-LSTM framework for optimizing UAVs through adaptive parameter control, which integrates the state information gleaned from communication signals. The ATT-Bi-LSTM achieves data feature extraction by means of a two-layer Bidirectional Long Short-Term Memory (BI-LSTM) at its inception to enhance the feature. Subsequently, it harnesses the attention mechanism to amplify the LSTM network's output, thereby enabling the optimal control of UAV positioning. During the empirical phase, we employ optical system data for the comparative validation of the model. The outcomes underscore the commendable performance of the proposed framework in this study, particularly with regard to the three pivotal position indicators: yaw, pitch, and roll. In the comparison of indicators such as RMSR and MAE, the proposed model has the lowest error, which provides algorithm support and important reference for future UAV optimization control research.

Seat belt detection using gated Bi-LSTM with part-to-whole attention on diagonally sampled patches

One of the high-risk behaviors leading to severe traffic injuries is not wearing a seat belt. It is therefore very important to be able to automatically detect seat belts from surveillance images, encourage drivers to wear seat belts, and enhance passenger safety. In this paper, a novel deep neural network, Gated Bi-directional Long Short-Term Memory network with part-to-whole attention (GBL-PA), is proposed for seat belt detection from surveillance images. The innovation of our model lies in its unique diagonal sampling strategy, which meticulously captures the seat belt’s fine details, typically oriented from top right to bottom left across the torso of vehicle occupants. Our framework’s novelty is further encapsulated by the part-to-whole attention mechanism, which intelligently harmonizes the detailed local information from seat belt-specific patches with the broader contextual insights from the regional proposals. The pioneering design of a Gated Bi-directional LSTM network facilitates the dynamic integration of interactions across patches to deliver an optimized final prediction. The superiority of GBL-PA is established through rigorous comparison with the state-of-the-art methods on a new, large benchmark dataset comprising 14,936 images from traffic surveillance footage. Our framework demonstrates a notable improvement, achieving a mean Average Precision (mAP) of 72.3%, which surpasses the second best, YOLOX, by 0.9% mAP. This significant and consistent outperformance across various metrics underscores the transformative potential of GBL-PA in the realm of traffic safety enforcement. The source code of our framework is available at ANONYMISED.

The Prediction of the Remaining Useful Life of Rotating Machinery Based on an Adaptive Maximum Second-Order Cyclostationarity Blind Deconvolution and a Convolutional LSTM Autoencoder.

The prediction of the remaining useful life (RUL) is important for the conditions of rotating machinery to maintain reliability and decrease losses. This study proposes an efficient approach based on an adaptive maximum second-order cyclostationarity blind deconvolution (ACYCBD) and a convolutional LSTM autoencoder to achieve the feature extraction, health index analysis, and RUL prediction for rotating machinery. First, the ACYCBD is used to filter noise from the vibration signals. Second, based on the peak value properties, a novel health index (HI) is designed to analyze the health conditions for the denoising signal, showing a high sensitivity for the degradation of bearings. Finally, for better prognostics and health management of the rotating machinery, based on convolutional layers and LSTM, an autoencoder can achieve a transform convolutional LSTM network to develop a convolutional LSTM autoencoder (ALSTM) model that can be applied to forecast the health trend for rotating machinery. Compared with the SVM, CNN, LSTM, GRU, and DTGRU methods, our experiments demonstrate that the proposed approach has the greatest performance for the prediction of the remaining useful life of rotating machinery.

Optimized Image Captioning: Hybrid Transformers Vision Transformers and Convolutional Neural Networks: Enhanced with Beam Search

Deep learning has improved image captioning. Transformer, a neural network architecture built for natural language processing, excels at image captioning and other computer vision applications. This paper reviews Transformer-based image captioning methods in detail. Convolutional neural networks (CNNs) extracted image features and RNNs or LSTM networks generated captions in traditional image captioning. This method often has information bottlenecks and trouble capturing long-range dependencies. Transformer architecture revolutionized natural language processing with its attention strategy and parallel processing. Researchers used Transformers' language success to solve image captioning problems. Transformer-based image captioning systems outperform previous methods in accuracy and efficiency by integrating visual and textual information into a single model. This paper discusses how the Transformer architecture's self-attention mechanisms and positional encodings are adapted for image captioning. Vision Transformers (ViTs) and CNN-Transformer hybrid models are discussed. We also discuss pre-training, fine-tuning, and reinforcement learning to improve caption quality. Transformer-based image captioning difficulties, trends, and future approaches are also examined. Multimodal fusion, visual-text alignment, and caption interpretability are challenges. We expect research to address these issues and apply Transformer-based image captioning to medical imaging and distant sensing. This paper covers how Transformer-based approaches have changed image captioning and their potential to revolutionize multimodal interpretation and generation, advancing artificial intelligence and human-computer interactions.

A state estimation method based on CNN-LSTM for ball screw

Ball screw is widely used in the engineering field, and accurate estimation of their state is crucial for the reliability of system operation. However, existing methods often overlook the time series characteristics and spatial correlation of vibration signals, unable to provide complete degradation information and divide the degradation process, resulting in limited prediction accuracy. Therefore, a state estimation method for ball screw based on Convolutional Neural Networks (CNN) and Long Short-Term Memory Neural Networks (LSTM) is proposed. An experiment of ball screw transmission equipment was conducted to collect vibration signals throughout the entire life cycle and verify the proposed method. Firstly, the frequency domain amplitude signal of the transformed ball screw is normalized to eliminate scale differences, which serves as the input for CNN feature extraction. Then, these deep features are input into the LSTM network to capture the fault evolution patterns that reveal the degradation of ball screw performance, and achieve accurate estimation of ball screw state. The final prediction accuracy was 97.87%, verifying the effectiveness of the proposed method.

Construction health indicator using physically-informed 1D-WGAN-GP joint attention LSTM-DenseNet method

In data-driven prognosis methods, the accuracy of predicting the remaining useful life (RUL) of mechanical systems is predominantly contingent upon the efficacy of system health indicators (HI), typically amalgamated from statistical features derived from collected signals. Nevertheless, the majority of extant HI are beset by two principal shortcomings: (1) during traditional data denoising processes, degradation information from raw data is prone to loss owing to the lack of incorporation of the true physical properties of the data; and (2) the performance evaluation of constructed HI is imbalanced due to the influence of network structures on single models, often resulting in strong performance in only one or two indicators. To overcome such shortcomings, a mechanical health indicator construction method based on physical properties was proposed, termed 1D-WGAN-GP Joint attention LSTM-DenseNet. Firstly, artificial sample data is generated by analyzing the physical properties of the original dataset, which is then used to train the 1D-WGAN-GP model to achieve data denoising. Subsequently, the fusion of the attention LSTM (A-LSTM) network and DenseNet network is utilized to extract crucial feature vectors of HI under varying health conditions from the denoised data. Finally, the extracted feature vectors are used to construct system HI using the Euclidean distance method, and these indicators are used for predicting the system’s RUL. The results indicate that the proposed method outperformed traditional methods in terms of denoising effectiveness. Further, through ablation experiment analysis, the HI constructed by the proposed method demonstrated obvious complementarity in terms of monotonicity, correlation, robustness, and comprehensive evaluation. In RUL prediction applications, the proposed method also exhibited good performance, thereby validating its effectiveness.