Short-term Memory Neural Network Research Articles

Sqrt-Loglogish CNN and Markov model for 5G spectrum sensing application

The research presents innovative methods for spectrum sensing in 5G networks, using the Sqrt-Loglogish convolutional neural network (SL-CNN) and hidden orthogonal intuitionistic fuzzy Markov model (HOIFMM). The proposed methods aim to tackle issues related to detecting principal user signals accurately, mitigating interference, and efficiently utilizing the spectrum in wideband spectrum environments due to their diverse and ever changing characteristics. The Sqrt-Loglogish CNN improves spectrum sensing by addressing static threshold dependency and potential overfitting. The HOIFMM offers a complex framework for predicting sparsity levels and primary user patterns. The results highlight the effectiveness of the suggested techniques in differentiating primary user signals from noise and interference, resulting in enhanced interference management tactics and overall network performance. MATLAB simulation is performed and compared the proposed methods performance with existing state-of-the-art methods such as CNN, deep neural network (DNN), long short-term memory (LSTM) and artificial neural networks (ANN). The proposed method has outperformed existing methods in terms of sensitivity, accuracy, and precision. Future endeavors include improving these methods, investigating sophisticated machine learning algorithms, and doing real world validations to guarantee scalability and resilience in various 5G deployment situations. This research advances the spectrum sensing capabilities in 5G networks, potentially improving efficiency, reliability, and quality of service.

Precipitation measurements experiment using microwave links in Eastern China from October 2020 to March 2022

Precipitation is an important piece of information needed in many areas such as transportation, military and agriculture, and microwave links have proven to be an effective means of acquiring it and can be used as a complementary means for professional precipitation measurement instruments such as rain gauges, weather radar, rain measuring satellites, etc. In this paper, two microwave links (at 26 GHz, both vertically polarised) located across a river in eastern China and a nearby OTT PARSIVEL disdrometer were used to carry out a precipitation observation experiment from October 2020 to March 2022 (March to November 2021 for liquid precipitation, others for possible non-liquid precipitation). The applicability of the dry and rainy period identification methods (standard deviation method and correlation coefficient method) based on this link data is analyzed first. In addition, a wet antenna attenuation correction model based on a Long Short-Term Memory (LSTM) neural network was applied to non-winter precipitation inversion. The results show that the rain rate and cumulative rainfall calculated from the two links are in good agreement with that calculated by the disdrometer data. Furthermore, quantitative inversion of winter precipitation using two microwave links is performed, and the variation characteristics of the link levels during non-liquid precipitation periods are also analyzed.

Efficient grid management: smart forecasting of short-term power load using PSO-LSTM

Recent load forecasting techniques combining machine learning models and hyperparameter optimization algorithms have shown success for short-term load forecasting (STLF) task, but they often require complex programming, higher computational costs, and greater parameter tuning. In this paper, we introduce an improved STLF model that combines Long Short-Term Memory (LSTM) neural network with Particle Swarm Optimization (PSO) for enhanced performance. In the proposed approach, the number of hidden neurons in different LSTM layers, learning rate and the number of iterations for training are optimized using the PSO algorithm. To validate the effectiveness of this method, meteorological data and historical load data from a real-world power grid are used as input. The experimental results reveal PSO significantly enhances hyperparameter tuning for LSTM neural networks, leading to improved predictive modelling. The PSO-LSTM model performed better than the LSTM model by more than 20% (in terms of Mean Absolute Error), and showed low sensitivity to hyperparameters. Comparative analysis with alternative approaches from the literature further validates the PSO-LSTM’s effectiveness in STLF. Additionally, the model achieved stable multi-step prediction capabilities, with average errors of 3.6445 for MAE, 4.6509 for RMSE, and 4.6519 for MAPE over a 1–4 day ahead lead times. This study highlights PSO-LSTM’s enhanced robustness and accuracy in power load prediction while addressing hyperparameter tuning challenges through self-optimization.

A decoupling estimation method for baseline load of photovoltaic user clusters based on multidimensional abnormal data restoration

Abstract With the widespread installation of distributed photovoltaics, their fluctuation characteristics with weather changes have posed significant challenges to baseline load estimation. In addition, due to measurement and collection anomalies, it is easy to cause continuous missing and abnormal data collection. To address the issue of low baseline estimation accuracy caused by distributed photovoltaic fluctuation characteristics and abnormal data collection, this paper proposes a photovoltaic user cluster baseline load decoupling estimation method based on multi-dimensional abnormal data repair. Firstly, a multidimensional anomaly data repair system is established based on extreme learning machines and multi-step prediction. Secondly, using the repaired data for baseline estimation, photovoltaic users and non-photovoltaic users are decoupled and separated. Long short-term memory neural network estimation models are used for non-photovoltaic users, and baseline load estimation methods based on weather type analysis are used for photovoltaic users. Finally, the baseline results are accumulated. The simulation results show that compared with the direct estimation method of baseline load, the accuracy of decoupling estimation has improved by 33.7%.

Research on the refinement of atmospheric weighted average temperature model in Xi’an based on machine learning

One of the most important parameters for calculating the atmospheric precipitable water vapor (PWV) is the atmospheric weighted mean temperature (Tm). Meteorological parameters that are measured typically constrain the accuracy and regional applicability of commonly used empirical models. When precipitable water is inverted using the Global Navigation Satellite System (GNSS), it results in erroneous Tm being obtained. Thus, in this paper, a small-scale Tm model for Xi’an is proposed by least squares method based on data provided by the European Centre for Mesoscale Weather Forecasts (ECMWF) from 2019 to 2022, while improving the functional model’s capacity to fit nonlinear residuals through the Random Forest method and enabling it to predict peaks more accurately. It is based on a random forest to confirm the weighted average temperature model’s correctness (RF Tm). When comparing the RF Tm model’s projected Tm accuracy to the GPT3, UNB3, and regional Bevis models, using the 2023 ERA5 data as the experimental data, the improvements are 23.6 %, 18.6 %, and 11.4 %, respectively. With the sounding station data as the reference value, the BIAS of RF Tm is only 1.51 K. In determining the atmospheric weighted average temperatures, the two models’ accuracy was comparable to that of the Long Short-Term Memory neural network model (LSTM). However, the LSTM model requires short-term forward data for practical usage, the RF Tm model compensated for the shortfall. The RF Tm model is used to invert the atmospheric precipitable water vapor at the appropriate monitoring stations using data from GNSS monitoring stations in Xi’an. It was demonstrated that the RF Tm model’s PWV inversion accuracy outperformed the PWV inverted by the GPT3 model by at least 30 %. Moreover, Xi’an rainfall data integration shows that the PWV inverted by the RF Tm model reacts to brief rainfall episodes efficiently. To sum up, this study’s RF Tm model provides a dependable and effective approach to meteorological observation and climate research.

Spatial–temporal prediction model for unsteady near-wall flow around cylinder based on hybrid neural network

A hybrid neural network based on Densely Connected Convolutional Networks (DenseNet), Convolutional Long Short-Term Memory Neural Network (ConvLSTM), and Deconvolutional Neural Network (DeCNN) is employed to predict unsteady flow fields. The utilization of DenseNet makes the model more compact and makes the prediction of three-dimensional flow affordable. The ConvLSTM is implemented to predict multiple future time steps which improves prediction efficiency. The proposed model transforms the time sequences of velocity and pressure fields into uniform spatial–temporal topology as input and captures nonlinear feature information in the spatial–temporal domain. Numerical simulations are conducted for the flow around cylinder at different Reynolds numbers and the near-wall flow around cylinder with different gap ratios, and training samples for the neural network inputs are established. The predicted results are compared with the numerical simulation results, showing good agreement. From the prediction cycle, it can be seen that good prediction results can be maintained in the first three prediction cycles. The prediction results of the three-dimensional unsteady flow around a cylinder near a plane wall, exhibit remarkable accuracy, successfully capturing the evolution of turbulent vortex structures. This signifies that the prediction model is highly effective in capturing the spatial–temporal variations of complex unsteady flows.

Study on rapid prediction of flow field in a knudsen compressor based on multi-fidelity reduced-order models

The safe and stable operation of a hydrogen Knudsen compressor is essential for transporting hydrogen in microfluidic systems. This study uses proper orthogonal decomposition to identify the coherent structures within the hydrogen flow field during non-equilibrium evolution. A long short-term memory neural network is then used to create a multi-fidelity reduced-order model, connecting two-dimensional and three-dimensional data to uncover transient flow mechanisms and enable rapid flow field prediction. The results show that the coherent structures of hydrogen flow, representing the most energetic modes, retain 99% of the flow energy and significantly influence the evolution of Poiseuille and thermal transpiration flows during non-equilibrium processes. The multi-fidelity reduced-order model effectively captures hydrogen transient flow and instabilities at various stages, achieving a 99.4% reduction in computational time while maintaining a maximum relative error of 0.53%. This approach facilitates the rapid prediction and control of flow states during hydrogen transport.

Multivariate Deep Learning Long Short-Term Memory-Based Forecasting for Microgrid Energy Management Systems

In the scope of energy management systems (EMSs) for microgrids, the forecasting module stands out as an essential element, significantly influencing the efficacy of optimal solution policies. Forecasts for consumption, generation, and market prices play a crucial role in both day-ahead and real-time decision-making processes within EMSs. This paper aims to develop a machine learning-based multivariate forecasting methodology to account for the intricate interplay pertaining to these variables from the perspective of day-ahead energy management. Specifically, our approach delves into the dynamic relationship between load demand variations and electricity price fluctuations within the microgrid EMSs. The investigation involves a comparative analysis and evaluation of recurrent neural networks’ performance to recognize the most effective technique for the forecasting module of microgrid EMSs. This study includes approaches based on Long Short-Term Memory Neural Networks (LSTMs), with architectures ranging from Vanilla LSTM, Stacked LSTM, Bi-directional LSTM, and Convolution LSTM to attention-based models. The empirical study involves analyzing real-world time-series data sourced from the Australian Energy Market (AEM), specifically focusing on historical data from the NSW state. The findings indicate that while the Triple-Stacked LSTM demonstrates superior performance for this application, it does not necessarily lead to more optimal operational costs, with forecast inaccuracies potentially causing deviations of up to forty percent from the optimal cost.

Reservoir Operation based Machine Learning Models: Comprehensive Review for Limitations, Research Gap, and Possible Future Research Direction

The operation of dams and reservoirs is critical for water resource management, including flood control, irrigation, hydropower generation, and environmental conservation. Traditional optimization techniques like Dynamic Programming (DP), Linear Programming (LP), and Nonlinear Programming (NLP) have been foundational in managing these operations. However, they often fall short in addressing the complexities of modern water management challenges posed by climate variability and increasing water demands. Machine learning (ML) techniques have emerged as powerful tools to enhance the efficiency and accuracy of dam and reservoir operations. This paper provides a comprehensive review of various ML models, including Neural Networks, Genetic Algorithms, Decision Trees, and Ensemble Methods, highlighting their applications in predicting reservoir inflows, optimizing water release schedules, and improving flood risk management. Notably, ML models like Long Short-Term Memory (LSTM) and Convolutional Neural Networks (CNNs) have shown significant improvements in forecasting accuracy and operational decision-making. Despite these advancements, several limitations and research gaps persist, including the need for real-time data integration, adaptive learning mechanisms, and models that consider socio-economic and climatic factors. This review underscores the importance of addressing these gaps to develop more robust and generalizable ML models. Future research directions are suggested to focus on hybrid models combining ML with traditional optimization techniques, comprehensive validation across diverse conditions, and the integration of ecological and economic considerations. By systematically identifying and addressing these limitations, this research aims to pave the way for more effective and sustainable dam and reservoir management practices. Leading towards suggesting that enhancing real-time data integration and developing adaptive learning mechanisms are in order to improve model responsiveness.

Decoding compositional complexity: Identifying composers using a model fusion-based approach with nonlinear signal processing and chaotic dynamics

Music, a universal medium that effortlessly transcends the confines of language and culture, serves as a vessel for the distinctive expression of a composer's ingenuity, particularly palpable through the elaborate symphony of melodies, harmonies, and rhythms. This phenomenon is acutely observable in the realm of Turkish Classical Music, where the identification of individual composers poses a formidable challenge due to a confluence of diverse stylistic expressions and sophisticated techniques. Shaped by centuries of cultural interchanges, this genre is celebrated for its convoluted rhythmic frameworks and deep melodic modes, often exhibiting fractal characteristics that compound the complexity of composer classification based on mere audio signals. In response to these complexities, this study introduces an advanced analytical paradigm that amalgamates Multi-resolution analysis, spectral entropy assessments, and a spectrum of multidimensional chaotic and statistical descriptors. By invoking chaos theory, the research delineates distinct patterns and features inherent to musical compositions, subsequently deploying these discoveries for composer categorization. Employing a model fusion-based strategy, the approach utilizes esteemed base estimators for section-level probabilistic determinations, subsequently amalgamated at the song level through a Long Short-Term Memory (LSTM) neural network model to classify a corpus of 380 compositions from 15 distinct composers. The results of this study not only highlight the efficacy of chaos-based approaches in Musical Information Retrieval but also provide a nuanced understanding of the unique characteristics of Turkish Classical Music, thus advancing the boundaries of how musicological data is scrutinized and conceptualized within scholarly discourse.

Digital twin method for intelligent lifting of cable structures under multi-collaborative mode

In order to ensure the high efficiency and quality consistency of cable structure in each construction stage, this study proposes a digital twin method for intelligent lifting of cable structure under multi-collaborative mode. Firstly, a construction and operation methods of the digital twin logic model are formed. In the logical model, a relationship between the transmission and iteration of construction information in the collaborative process is defined. In order to realize the adaptability of working conditions, the coupling relationship between construction step and structural elevation is established. A cable structure lifting analysis model driven by an improved long short-term memory neural network is proposed, which effectively solves the problem of poor convergence of the traditional finite element method. An automatic time window selection method is formed for the neural network. An adaptive moment estimation method is used for neural network parameter optimization, which improves the efficiency and accuracy of the analysis. Taking the lifting process of a cable truss structure test model as an example, a digital twin collaborative construction platform is established to verify the effectiveness and feasibility of the research method. The results show that the digital twin logic model realizes the process collaboration, scene collaboration and data collaboration, and significantly improves the stability of construction quality and the linkage of construction process. Through the improved long short-term memory neural network, the coupling relationship between the construction step and the structural elevation is established, which accurately guides the whole construction process.

A Hybrid Algorithm of LSTM and Factor Graph for Improving Combined GNSS/INS Positioning Accuracy during GNSS Interruptions.

In urban road environments, global navigation satellite system (GNSS) signals may be interrupted due to occlusion by buildings and obstacles, resulting in reduced accuracy and discontinuity of combined GNSS/inertial navigation system (INS) positioning. Improving the accuracy and robustness of combined GNSS/INS positioning systems for land vehicles in the presence of GNSS interruptions is a challenging task. The main objective of this paper is to develop a method for predicting GNSS information during GNSS outages based on a long short-term memory (LSTM) neural network to assist in factor graph-based combined GNSS/INS localization, which can provide a reliable combined localization solution during GNSS signal outages. In an environment with good GNSS signals, a factor graph fusion algorithm is used for data fusion of the combined positioning system, and an LSTM neural network prediction model is trained, and model parameters are determined using the INS velocity, inertial measurement unit (IMU) output, and GNSS position incremental data. In an environment with interrupted GNSS signals, the LSTM model is used to predict the GNSS positional increments and generate the pseudo-GNSS information and the solved results of INS for combined localization. In order to verify the performance and effectiveness of the proposed method, we conducted real-world road test experiments on land vehicles installed with GNSS receivers and inertial sensors. The experimental results show that, compared with the traditional combined GNSS/INS factor graph localization method, the proposed method can provide more accurate and robust localization results even in environments with frequent GNSS signal loss.

Time series analysis of sea surface temperature change in the coastal seas of Türkiye

Sea surface temperature (SST) is a crucial geophysical parameter in assessing heat exchange between the air and sea surface. Changes in SST and its accurate prediction play a pivotal role in explaining the global heat balance, determining atmospheric circulations, and constructing global climate models. This work aims to reveal a model for one-month-ahead forecasting of SST time series data along the Türkiye coasts, encompassing the Mediterranean, Aegean, Marmara, and Black Seas, and their long-term future forecast. A long short-term memory (LSTM) neural network and seasonal autoregressive integrated moving average (SARIMA) models are used for this purpose. The ECMWF ERA5 (0.5ox0.5°) monthly SST dataset spanning the years 1970–2023 is used for model development. The results obtained from the LSTM and SARIMA models show that there will be an increasing trend in SSTs along these seacoasts until 2050. The SST measurements of 23.4 °C, 20.2 °C, 17.0 °C, and 16.6 °C recorded along the Mediterranean, Aegean, Marmara, and Black Seas in 2023 are expected to rise to 25.1 °C, 21.9 °C, 18.1 °C, and 18.8 °C, respectively, by 2050. These figures indicate an increase of 7.3%, 8.4%, 6.5%, and 13.3% in the SST values across these coastal seas over the next quarter century.

A Study on Stock Price Prediction Based on Attention GCN-BiLSTM Modeling

It is of great significance to accurately predict stock price movements. By combining the R-Vine-Copula structure with the relationship between the conceptual sectors to which the stocks belong, the price correlation between stocks is measured and the adjacency matrix is constructed, and the model AGLSTM, which is constructed by combining the temporal attention mechanism, Graph Convolutional Neural Network and Bi-directional Long and Short-Term Memory Neural Network, is proposed to predict the price of 46 stocks from the constituent stocks of the SSE 50 by modeling them. The experimental results show that, compared with the baseline, the AGLSTM can predict the stock price. The experimental results show that compared with the baseline model, the MAE and RMSE results of the AGLSTM model predicted one step forward outperform all the comparison models, and the MAPE results are close to the best baseline model results. The results of MAPE are close to those of the best baseline model. Moreover, good results are also achieved in the experiment of multi-step forward prediction, which demonstrates the ability of AGLSTM model in long-term prediction and can provide some reference for investors to make investment decisions.

Multi-Scale Spatio-Temporal Attention Networks for Network-Scale Traffic Learning and Forecasting.

Accurate and timely forecasting of traffic on local road networks is crucial for deploying effective dynamic traffic control, advanced route planning, and navigation services. This task is particularly challenging due to complex spatio-temporal dependencies arising from non-Euclidean spatial relations in road networks and non-linear temporal dynamics influenced by changing road conditions. This paper introduces the spatio-temporal network embedding (STNE) model, a novel deep learning framework tailored for learning and forecasting graph-structured traffic data over extended input sequences. Unlike traditional convolutional neural networks (CNNs), the model employs graph convolutional networks (GCNs) to capture the spatial characteristics of local road network topologies. Moreover, the segmentation of very long input traffic data into multiple sub-sequences, based on significant temporal properties such as closeness, periodicity, and trend, is performed. Multi-dimensional long short-term memory neural networks (MDLSTM) are utilized to flexibly access multi-dimensional context. Experimental results demonstrate that the STNE model surpasses state-of-the-art traffic forecasting benchmarks on two large-scale real-world traffic datasets.

Deep residual ensemble model for predicting remaining useful life of turbo fan engines

Abstract Capturing degradation trends from the Condition monitored signals is a proven technique for predicting the Remining Useful Life (RUL) of the equipment, which has gained more prominence in Prognostics and Health Management (PHM) in Industry 4.0. However, this process is tiresome and comprehending all the physical parameters of the system to construct a Health Index that characterize the health state is a complex process, especially if multiple sensors are involved. This work proposes a Deep residual ensemble model which constructs Fused Health Index (FHI) by harnessing temporal property of signals. The proposed Residual network integrates Bi-directional Long Short Term Memory (Bi-LSTM) and Deep Neural Network (DNN) which absorbs individual residuals of both the forward and reverse LSTMs that acts as an important feature to improve the overall prediction process. The work validated using CMAPPS dataset using various unique performance metrics to portray the effectiveness of the model.

A multi-source data fusion driven power field effect transistor health state assessment and remaining useful life prediction method

Abstract The metal oxide semiconductor field effect transistor (MOSFET) is subjected to various stresses due to the external and internal operating environments, leading to ageing and failure of the device. With multiple degradation mechanisms, a single piece of information can no longer fully reflect the health state of MOSFETs, so how to use multi-source data to characterise the state of the device and predict the remaining useful life (RUL) is an issue worth exploring. To address this problem, a method for constructing health indicators (HI) as well as predicting RUL using multi-source data is proposed. In this method, firstly, the features are computed by selecting the appropriate ageing signal from the ageing mechanism. Secondly, the extracted features are filtered using Pearson’s algorithm to find the features that are strongly correlated with longevity. Then, the filtered features are merged by dimensionality reduction using the kernel principal component analysis algorithm and the HI is constructed from the reduced result. Finally, an unsupervised clustering algorithm is used to classify the states based on the data distribution in HI, and the filtered features are used as input to the grey wolf optimisation bidirectional long short-term memory neural network to predict the RUL of the device. In this paper, the proposed method is validated using data from the MOSFET Accelerated Aging Experiment at the NASA Ames Centre of Excellence for Prediction. The results show that the method is able to achieve good results in health state assessment and RUL prediction of MOSFETs. The proposed method is an effective and comprehensive strategy that is more helpful for the operation and maintenance of electronics.

A Deep Learning-Based Dual-Scale Hybrid Model for Ultra-Short-Term Photovoltaic Power Forecasting

Ultra-short-term photovoltaic (PV) power forecasting is crucial in the scheduling and functioning of contemporary electrical systems, playing a key role in promoting renewable energy integration and sustainability. In this paper, a novel hybrid model, termed AI_VMD-HS_CNN-BiLSTM-A, is introduced to tackle the challenges associated with the volatility and unpredictability inherent in PV power output. Firstly, Akaike information criterion variational mode decomposition (AI_VMD) integrates the Akaike information criterion with variational mode decomposition (VMD) and reduces data complexity, enhancing grid optimization and energy efficiency. The adaptive selection of optimal parameters enhances VMD decomposition performance, supporting sustainable energy management. Secondly, the hierarchical scale-transform convolutional architecture (HS_CNN) supplements the traditional convolutional neural network (CNN) with two channels featuring distinct dilation rates, thereby extracting dual levels of time-scale information for a more comprehensive data representation. Finally, a bidirectional long short-term memory neural network (BiLSTM) with an attentional mechanism combines past and future data to enable more accurate forecasts, aiding in carbon reduction and smart grid advancements. Experimentation with data from the Alice Springs PV plant in Australia demonstrates that the proposed AI_VMD-HS_CNN-BiLSTM-A model exhibits superior adaptability and accuracy in multiple time-scale forecasting compared to the baseline models. This approach is important for decision-making and scheduling in grid-connected photovoltaic systems, enhancing energy resilience and promoting the sustainable development of renewable energy.

Optimized energy management for interconnected networked microgrids: A hybrid NEGCN-PFOA approach with demand response and marginal pricing

This manuscript proposes a hybrid approach for operating cost reduction in networked microgrids. The proposed hybrid system is the combined performance of both the Neighbour Enhanced Graph Convolutional Networks (NEGCN) and the Piranha Foraging Optimization Algorithm (PFOA). Commonly it is named as NEGCN-PFOA technique. The primary goal of the proposed strategy is operating cost minimization in networked micro grids. The NEGCN is used for the demand prediction in networked microgrids and the PFOA is used for the operating cost optimization of networked microgrids. Moreover, the result of the system critic behaviour is researched, where the critic is a function of the control range. By then, the proposed method has been included in the working MATLAB platform, and its execution is calculated using the existing procedures. Compared to all current approaches, the proposed technique yields better outcomes like Deep Reinforcement Learning, Long Short-Term Memory and Deep Learning Artificial Neural Network. The proposed method is useful in reducing the operating cost in the networked micro grids. From the outcome, it is concluded that the proposed technique based costs is less contrasted with existing methods.

Short-term wind power forecasting using the hybrid model of multivariate variational mode decomposition (MVMD) and long short-term memory (LSTM) neural networks

Short-term wind power forecasting using the hybrid model of multivariate variational mode decomposition (MVMD) and long short-term memory (LSTM) neural networks