TPA-net: Temporal pyramid convolution network with attentional BiLSTM for power load forecasting

Protein secondary structure prediction (PSSP) is a challenging task in computational biology. However, existing models with deep architectures are not sufficient and comprehensive for deep long-range feature extraction of long sequences. This paper proposes a novel deep learning model to improve Protein secondary structure prediction. In the model, our proposed bidirectional temporal convolutional network (BTCN) can extract the bidirectional deep local dependencies in protein sequences segmented by the sliding window technique, the bidirectional long short-term memory (BLSTM) network can extract the global interactions between residues, and our proposed multi-scale bidirectional temporal convolutional network (MSBTCN) can further capture the bidirectional multi-scale long-range features of residues while preserving the hidden layer information more comprehensively. In particular, we also propose that fusing the features of 3-state and 8-state Protein secondary structure prediction can further improve the prediction accuracy. Moreover, we also propose and compare multiple novel deep models by combining bidirectional long short-term memory with temporal convolutional network (TCN), reverse temporal convolutional network (RTCN), multi-scale temporal convolutional network (multi-scale bidirectional temporal convolutional network), bidirectional temporal convolutional network and multi-scale bidirectional temporal convolutional network, respectively. Furthermore, we demonstrate that the reverse prediction of secondary structure outperforms the forward prediction, suggesting that amino acids at later positions have a greater impact on secondary structure recognition. Experimental results on benchmark datasets including CASP10, CASP11, CASP12, CASP13, CASP14, and CB513 show that our methods achieve better prediction performance compared to five state-of-the-art methods.

In cybersecurity, the rise of fileless malware poses a significant challenge to endpoint security. Traditional detection methods often fail against these sophisticated attacks, necessitating advanced techniques like deep learning models. This study highlights the limitations of Bi-Directional Long Short-Term Memory (BLSTM) models in dynamic malware analysis and proposes enhancements through Convolutional Long Short-Term Memory (ConvLSTM) architecture. BLSTM models process input sequences in forward and backward directions, combining the results into one output. While this dual-layer approach improves analysis, it is time-consuming, potentially increasing the risk of fileless malware attacks. A key limitation of BLSTM is the lack of parameter sharing between forward and backward directions. This reduces its ability to capture spatial and temporal features simultaneously, hindering effectiveness in detecting fileless malware. To address this, the ConvLSTM model consolidates feature extraction within a single LSTM cell layer. ConvLSTM breaks down samples into subsequence and uses timesteps for additional feature extraction, enabling spatial-temporal data analysis and improving malware prediction accuracy. The model was tested using a dynamic malware dataset. Unlike traditional LSTM, ConvLSTM integrates convolutional layers, allowing parameter sharing across both spatial and temporal dimensions. This reduces computational complexity and improves model performance in handling multidimensional data. The research re-simulated prior work with BLSTM using the same malware dataset. The Spyder app ran the event simulator and the ConvLSTM model's results replaced BLSTM's using identical parameters. Time, accuracy and loss were the main performance metrics. ConvLSTM outperformed BLSTM, achieving 98% detection accuracy compared to BLSTM's 90%. It also significantly reduced processing time, averaging 10 seconds, while BLSTM took 22 seconds. ConvLSTM experienced lower losses, averaging 10% per epoch versus BLSTM's 20%. In conclusion, ConvLSTM offers superior performance over BLSTM in fileless malware detection. Its enhanced computational efficiency and ability to quickly mitigate threats make it a robust solution for fortifying endpoint security against evolving cyber threats. ConvLSTM holds potential in strengthening defence mechanisms against sophisticated malware attacks, providing a proactive approach to safeguarding networks and data.

A new framework for ultra-short-term electricity load forecasting model using IVMD–SGMD two–layer decomposition and INGO–BiLSTM–TPA–TCN

Accurate power load forecasting can significantly improve the economic benefits of power systems. To improve the prediction accuracy, aiming at the complexity and volatility of power load, a forecasting model based on improved whale optimization algorithm (IWOA) optimized the bidirectional long short-term memory (BiLSTM) combined with attention mechanism (IWOA-Attention- BiLSTM) is proposed. The model comprehensively considers the influence of meteorological factors and date types, learns the bidirectional series features of power load data by BiLSTM, calculates the weights of the hidden layer state by the attention mechanism, and finds the hyperparameters of Attention-BiLSTM by IWOA, such as the learning rate, iteration times and batch size. The results show that compared with BP, LSTM and Seq2Seq, IWOA-Attention-BiLSTM has the highest prediction accuracy, and its MAPE, RMSE, MAE and R2 are 1.44 %, 128.83MW, 97.83MW and 0.9931 respectively, which are the best among all the prediction models. It is proved that IWOA-Attention- BiLSTM can effectively improve the prediction accuracy of short-term power load.

As part of the field of DNA methylation identification, this study tackles the challenge of enhancing recognition performance by introducing a specialized deep learning framework called DeepPGD. DNA methylation, a crucial biological modification, plays a vital role in gene expression analyses, cellular differentiation, and the study of disease progression. However, accurately and efficiently identifying DNA methylation sites remains a pivotal concern in the field of bioinformatics. The issue addressed in this paper is the presence of methylation in DNA, which is a binary classification problem. To address this, our research aimed to develop a deep learning algorithm capable of more precisely identifying these sites. The DeepPGD framework combined a dual residual structure involving Temporal convolutional networks (TCNs) and bidirectional long short-term memory (BiLSTM) networks to effectively extract intricate DNA structural and sequence features. Additionally, to meet the practical requirements of DNA methylation identification, extensive experiments were conducted across a variety of biological species. The experimental results highlighted DeepPGD's exceptional performance across multiple evaluation metrics, including accuracy, Matthews' correlation coefficient (MCC), and the area under the curve (AUC). In comparison to other algorithms in the same domain, DeepPGD demonstrated superior classification and predictive capabilities across various biological species datasets. This significant advancement in algorithmic prowess not only offers substantial technical support, but also holds potential for research and practical implementation within the DNA methylation identification domain. Moreover, the DeepPGD framework shows potential for application in genomics research, biomedicine, and disease diagnostics, among other fields.

Systolic blood pressure (SBP) and heart rate (HR) are critical for proactive clinical decision-making, particularly in intensive care environments. Early identification of abnormalities in these vital signs helps patients receive appropriate treatment and reduce associated health degradation. This study presents FCMS-iDMM, a fog/cloud-based real-time forecasting system that simultaneously predicts future SBP and HR values using deep learning models integrated with big data streaming platforms. The research involves two main phases: offline model and online forecasting pipeline optimization. During the offline model development, we explore the single-task and multi-task modeling. Different optimization steps have been explored. The single task includes forecasting HR and SBP in multi-step heads using Temporal Convolutional Networks (TCN), Long Short-Term Memory (LSTM), and Gated Recurrent Unit (GRU). Multi-task comprises forecasting HR and SBP in diverse multi-step heads as puerperal employing TCN, sequence-to-sequence (seq2seq), and Autoencoder models using LSTM and GRU. Extensive results are accomplished by the Medical Information Mart for Intensive Care III (MIMIC III) to assess the performance of the proposed multi-task DL model. Multi-task learning (MTL) models based on Temporal Convolutional Networks (TCNs) achieved superior performance. In forecasting 8 min, TCN recorded the best performance compared to other models with 1.5428 RMSE, 1.0871 MAE, and 1.269 MAPE for HR and 4.1446 RMSE, 2.4323 MAE and 2.5237 MAPE for SBP in multi-task. Simulated sensors, Apache Kafka, and Apache Spark are used to develop the real-time HR and SBP online forecasting pipeline. Experimental validation using the MIMIC-III database confirmed that multi-task models outperform single-task approaches across multiple forecasting horizons. The proposed system offers a scalable and efficient solution for real-time monitoring of vital signs, paving the way for predictive, patient-centered healthcare systems.

Accurate power load forecasting is crucial for the sustainable operation of smart grids. However, the complexity and uncertainty of load, along with the large-scale and high-dimensional energy information, present challenges in handling intricate dynamic features and long-term dependencies. This paper proposes a computational approach to address these challenges in short-term power load forecasting and energy information management, with the goal of accurately predicting future load demand. The study introduces a hybrid method that combines multiple deep learning models, the Gated Recurrent Unit (GRU) is employed to capture long-term dependencies in time series data, while the Temporal Convolutional Network (TCN) efficiently learns patterns and features in load data. Additionally, the attention mechanism is incorporated to automatically focus on the input components most relevant to the load prediction task, further enhancing model performance. According to the experimental evaluation conducted on four public datasets, including GEFCom2014, the proposed algorithm outperforms the baseline models on various metrics such as prediction accuracy, efficiency, and stability. Notably, on the GEFCom2014 dataset, FLOP is reduced by over 48.8%, inference time is shortened by more than 46.7%, and MAPE is improved by 39%. The proposed method significantly enhances the reliability, stability, and cost-effectiveness of smart grids, which facilitates risk assessment optimization and operational planning under the context of information management for smart grid systems.

Higher penetration of renewable and smart home technologies at the residential level challenges grid stability as utility-customer interactions add complexity to power system operations. In response, short-term residential load forecasting has become an increasing area of focus. However, forecasting at the residential level is challenging due to the higher uncertainties involved. Recently deep neural networks have been leveraged to address this issue. This paper investigates the capabilities of a bidirectional long short-term memory (BiLSTM) and a convolutional neural network-based BiLSTM (CNN-BiLSTM) to provide a day ahead (24 hr.) forecasting at an hourly resolution while minimizing the root mean squared error (RMSE) between the actual and predicted load demand. Using a publicly available dataset consisting of 34 homes, the BiLSTM and CNN-BiLSTM models are trained to forecast the aggregated active power demand for each hour within a 24 hr. span, given the previous 24 hr. load data. The BiLSTM model achieved the lowest RMSE of 1.4842 for the overall daily forecast. In addition, standard LSTM and CNN-LSTM models are trained and compared with the BiLSTM architecture. The RMSE of BiLSTM is 5.60%, 2.85% and 2.60% lower than LSTM, CNN-LSTM and CNN-BiLSTM models respectively.

Temperature prediction is an important and significant step for monitoring global warming and the environment to save and protect human lives. The climatology parameters such as temperature, pressure, and wind speed are time-series data and are well predicted with data driven models. However, data-driven models have certain constraints, due to which these models are unable to predict the missing values and erroneous data caused by factors like sensor failure and natural disasters. In order to solve this issue, an efficient hybrid model, i.e., attention-based bidirectional long short term memory temporal convolution network (ABTCN) architecture is proposed. ABTCN uses k-nearest neighbor (KNN) imputation method for handling the missing data. A bidirectional long short term memory (Bi-LSTM) network with self-attention mechanism and temporal convolutional network (TCN) model that aids in the extraction of features from complex data and prediction of long data sequence. The performance of the proposed model is evaluated in comparison to various state-of-the-art deep learning models using error metrics such as MAE, MSE, RMSE, and R2 score. It is observed that our proposed model is superior over other models with high accuracy.

Bidirectional long short-term memory (BLSTM) recurrent neural networks are powerful acoustic models in terms of recognition accuracy. When BLSTM acoustic models are used in decoding, the speech decoder needs to wait until the end of a whole sentence is reached, such that forward-propagation in the backward direction can then be performed. The nature of BLSTM acoustic models makes them inappropriate for real-time online speech recognition because of the latency issue. Recently, the context-sensitive-chunk BLSTM and latency-controlled BLSTM acoustic models have been proposed, both chop a whole sentence into several overlapping chunks. By appending several left and/or right contextual frames, forward-propagation of BLSTM can be down within a controlled time delay, while the recognition accuracy is maintained when comparing with conventional BLSTM models. In this paper, two improved versions of latency-controlled BLSTM acoustic models are presented. By using different types of neural network topology to initialize the BLSTM memory cell states, we aim at reducing the computational cost introduced by the contextual frames and enabling faster online recognition. Experimental results on a 320-hour Switchboard task have shown that the improved versions accelerate from 24% to 61% in decoding without significant loss in recognition accuracy.

This paper addresses restoration of acoustic signals for robot audition. A robot usually listens to target acoustic signals such as speech and music in noisy conditions. Acoustic information on such signals inevitably contaminated with noise. Even when noise reduction techniques such as sound source separation are performed, the noise-reduced acoustic signals contain distortion and/or residual noise after the noise reduction to some extent. The distortion and residual noise basically degrade the performance of recognition processes such as automatic speech recognition (ASR). We decided to use bidirectional long short-term memory (Bi-LSTM) for acoustic signal restoration since it can represent dynamic behaviors well for a temporal sequence in the forward and backward directions. When applying Bi-LSTM to recover acoustic signals, there is an issue, that is, acoustic signals tend to be sparse in high frequencies, and thus Bi-LSTM training becomes insufficient in such high frequencies due to a lack of training data. Therefore, we propose a new restoration method based on Bi-LSTM with spectral filtering. The spectral filter and the corresponding inverse filter are introduced to a Bi-LSTM framework to accelerate training in high frequencies. Preliminary results showed that the proposed Bi-LSTM with spectral filtering can perform signal restoration even when a small amount of training data is available.

Diacritic restoration has gained importance with the growing need for machines to understand written texts. The task is typically modeled as a sequence labeling problem and currently Bidirectional Long Short Term Memory (BiLSTM) models provide state-of-the-art results. Recently, Bai et al. (2018) show the advantages of Temporal Convolutional Neural Networks (TCN) over Recurrent Neural Networks (RNN) for sequence modeling in terms of performance and computational resources. As diacritic restoration benefits from both previous as well as subsequent timesteps, we further apply and evaluate a variant of TCN, Acausal TCN (A-TCN), which incorporates context from both directions (previous and future) rather than strictly incorporating previous context as in the case of TCN. A-TCN yields significant improvement over TCN for diacritization in three different languages: Arabic, Yoruba, and Vietnamese. Furthermore, A-TCN and BiLSTM have comparable performance, making A-TCN an efficient alternative over BiLSTM since convolutions can be trained in parallel. A-TCN is significantly faster than BiLSTM at inference time (270%-334% improvement in the amount of text diacritized per minute).

The dynamic variations in connected loads and the intermittent nature of renewable energy sources significantly impact smart grid reliability. Accurate load and generation forecasting stand pivotal for enhancing grid reliability and efficient operations. In this study, a comprehensive four-step approach is introduced for short-term load forecasting (STLF) aimed at precisely estimating power demand and generation. The process unfolds with data collection, followed by rigorous standardization, preprocessing, and cleansing of demand and generation data. Subsequently, a hybrid deep learning model, comprising bidirectional long short-term memory (BiLSTM), bidirectional gated recurrent unit (BiGRU), and a fully connected layer is trained using the clean data. This model harnesses the temporal dependencies within the data for accurate predictions. The model's performance is then evaluated using mean square error (MSE), root mean square error (RMSE), mean absolute error (MAE), and mean absolute percentage error (MAPE), providing forecasted values for both generation and demand on minute, hourly, daily, and weekly intervals. Notably, the proposed approach achieves a remarkable MSE of 0.0058 for load forecasting and 0.0033 for generation forecasting. Comparative analysis with state-of-the-art (SOTA) techniques in terms of accuracy and computational cost underscores the superior accuracy of the proposed framework in forecasting both generation and demand. Importantly, the proposed approach bridges the gap in reliability enhancement for smart grids operating, a facet lacking in many existing methodologies. This signifies the potential of the proposed approach to bolster smart grid reliability, ensuring more reliable operations.

Ultra-short-term photovoltaic power prediction based on similar day clustering and temporal convolutional network with bidirectional long short-term memory model: A case study using DKASC data

An Intrusion Detection System (IDS) is a defence system that provides safety and security against different threats and attacks, acting as a wall of defence against attackers. As internet usage increases, IDSs are becoming an essential part of day-to-day life. Various Machine Learning (ML) and Deep Learning (DL) based IDS are available, and the domain of IDS is still evolving and growing. Here this paper proposes two DL-based IDSs, first is a combination of LuNet and Bidirectional LSTM (Bi-LSTM) and other is a combination of Temporal Convolutional Network (TCN), CNN and Bi-LSTM. Such IDS must be fed with an efficient number of samples to keep them updated and accurate. The first model has been trained and tested against two benchmark datasets, NSL-KDD and UNSW-NB15. The second model has been trained and tested against the NSL-KDD dataset. To overcome the insufficient number of samples, the models have used a technique called Synthetic Minority Oversampling Technique (SMOTE). These models provided better experimental outcomes than traditional ML-based approaches and many DL approaches. They have better results in classification accuracy and, detection rate. The classification accuracy of the first model for UNSW-NB15 and NSL-KDD is 82.19% and 98.87% respectively. The classification accuracy of the second model for NSL-KDD is 98.8%.