Stacked Bidirectional Long Short-term Memory Research Articles

Inverse chi-square-based flamingo search optimization with machine learning-based security solution for Internet of Things edge devices

<abstract> <p>Internet of Things (IoT) edge devices are becoming extremely popular because of their ability to process data locally, conserve bandwidth, and reduce latency. However, with the developing count of IoT devices, threat detection, and security are becoming major concerns. IoT edge devices must avoid cyber threats and protect user data. These devices frequently take limited resources and can run on lightweight operating systems, which makes them vulnerable to security attacks. Intrusion detection systems (IDS) can be run on edge devices to recognize suspicious actions and possible risks. These systems monitor traffic patterns, and behavior, and identify attack signatures to detect and report on possible attacks. This study presents a design for an inverse chi square-based flamingo search optimization algorithm with machine learning (ICSFSO-ML) as a security solution for Internet of Things edge devices. The goal of the ICSFSO-ML technique is to apply ML and metaheuristics for threat recognition in IoT edge devices. To reduce the high dimensionality problem, the ICSFSO-ML technique uses the ICSFSO algorithm for feature selection purposes. Further, the ICSFSO-ML technique exploits the stacked bidirectional long short-term memory (SBiLSTM) model for the threat detection process. To enhance the efficacy of the SBiLSTM model, an arithmetic optimization algorithm (AOA) is applied for the hyperparameter selection process. The simulation performance of the ICSFSO-ML technique can be tested on a benchmark threat database. The performance analysis showed the benefits of the ICSFSO-ML methodology compared to existing methodologies with a maximum accuracy of 98.22%.</p> </abstract>

Equilibrium Optimization Algorithm with Deep Learning Enabled Prostate Cancer Detection on MRI Images.

The enlargement of the prostate gland in the reproductive system of males is considered a form of prostate cancer (PrC). The survival rate is considerably improved with earlier diagnosis of cancer; thus, timely intervention should be administered. In this study, a new automatic approach combining several deep learning (DL) techniques was introduced to detect PrC from MRI and ultrasound (US) images. Furthermore, the presented method describes why a certain decision was made given the input MRI or US images. Many pretrained custom-developed layers were added to the pretrained model and employed in the dataset. The study presents an Equilibrium Optimization Algorithm with Deep Learning-based Prostate Cancer Detection and Classification (EOADL-PCDC) technique on MRIs. The main goal of the EOADL-PCDC method lies in the detection and classification of PrC. To achieve this, the EOADL-PCDC technique applies image preprocessing to improve the image quality. In addition, the EOADL-PCDC technique follows the CapsNet (capsule network) model for the feature extraction model. The EOA is based on hyperparameter tuning used to increase the efficiency of CapsNet. The EOADL-PCDC algorithm makes use of the stacked bidirectional long short-term memory (SBiLSTM) model for prostate cancer classification. A comprehensive set of simulations of the EOADL-PCDC algorithm was tested on the benchmark MRI dataset. The experimental outcome revealed the superior performance of the EOADL-PCDC approach over existing methods in terms of different metrics.

Multivariate Time Series Sensor Feature Forecasting Using Deep Bidirectional LSTM

Multivariate Time series data forecasting (MTSF) is the assignment of forecasting future estimates of a particular series employing historic data. Lately, this work has enticed the focus of machine and deep learning researchers to tackle the complex and time consuming aspects of conventional forecasting techniques. Along with increasing access to historic documented data and the dire need of carrying out precise time-series feature forecasting. In this paper, we put forward a deep-learning(DL) technique proficient to tackle the setback of conventional forecasting techniques and display precise forecasting. The proposed technique is a Stacked Bidirectional long-short term memory architecture, which is an improvised version of existing Bidirectional LSTM in which multiple Bidirectional LSTM blocks are stacked, such that each layer contains multiple cells. In the direction of fair assessment, the functioning of the proposed technique is compared with existing LSTMs. Using varied evaluation measures, the obtained results show that the proposed Deep Bidirectional long-short term memory model exceeds standard approaches.

Arithmetic Optimization with Deep Learning Enabled Churn Prediction Model for Telecommunication Industries

Customer retention is one of the challenging issues in different business sectors, and various firms utilize customer churn prediction (CCP) process to retain existing customers. Because of the direct impact on the company revenues, particularly in the telecommunication sector, firms are needed to design effective CCP models. The recent advances in machine learning (ML) and deep learning (DL) models enable researchers to introduce accurate CCP models in the telecommunication sector. CCP can be considered as a classification problem, which aims to classify the customer into churners and non-churners. With this motivation, this article focuses on designing an arithmetic optimization algorithm (AOA) with stacked bidirectional long short-term memory (SBLSTM) model for CCP. The proposed AOA-SBLSTM model intends to proficiently forecast the occurrence of CC in the telecommunication industry. Initially, the AOA-SBLSTM model performs pre-processing to transform the original data into a useful format. Besides, the SBLSTM model is employed to categorize data into churners and non-churners. To improve the CCP outcomes of the SBLSTM model, an optimal hyperparameter tuning process using AOA is developed. A widespread simulation analysis of the AOA-SBLSTM model is tested using a benchmark dataset with 3333 samples and 21 features. The experimental outcomes reported the promising performance of the AOA-SBLSTM model over the recent approaches.

Longitudinal study of early mild cognitive impairment via similarity-constrained group learning and self-attention based SBi-LSTM

Alzheimer’s disease (AD) is an incurable neurodegenerative disease. Mild cognitive impairment (MCI) is often considered a critical time window for predicting early conversion to Alzheimer’s disease (AD), with approximately 80% of amnestic MCI patients developing AD within 6 years. MCI can be further categorized into two stages (i.e., early MCI (EMCI) and late MCI (LMCI)). To identify EMCI effectively and understand how it changes brain function, the brain functional connectivity network (BFCN) has been widely used. However, the conventional methods mainly focused on detection from a single time-point data, which could not discover the changes during the disease progression without using multi-time points data. Therefore, in this work, we carry out a longitudinal study based on multi-time points data to detect EMCI and validate them on two public datasets. Specifically, we first construct a similarity-constrained group network (SGN) from the resting state functional magnetic resonance imaging (rs-fMRI) data at different time-points, and then use a stacked bidirectional long short term memory (SBi-LSTM) network to extract features for longitudinal analysis. Also, we use a self-attention mechanism to leverage high-level features to further improve the detection accuracy. Evaluated on the public Alzheimer’s Disease Neuroimaging Initiative Phase II and III (ADNI-2 and ADNI-3) databases, the proposed method outperforms several state-of-the-art methods.

Multivariate stacked bidirectional long short term memory for lithium-ion battery health management

Prognostics and health management (PHM) will ensure the safe and reliable operation of the battery systems. The remaining useful life (RUL) prediction as one of the major PHM strategies gives early warning of faults, which can be applied to recognize the necessary battery maintenance and replacement in advance. This study investigates a novel deep learning method for predicting lithium-ion battery RUL, which can learn the long-term dependency of degradation trend of batteries. This is the first time which a stacked bidirectional long short-term memory (SBLSTM) based on extreme gradient boosting (XGBoost) is applied to predict the battery capacity degradation trajectories. Using the XGBoost technique, important time-domain features are selected as multivariate inputs to feed the deep learning model for predicting. To improve the trained model, Bayesian optimization (BO) is also performed to tune the hyper-parameters. The findings show that the SBLSTM model achieves a low root mean square percentage error of 1.94%, which is lower than the state-of-the-art methods due to two-way learning. The suggested model will provide excellent support for the maintenance strategy development and health management of the battery systems.

Research on Event Logic Knowledge Graph Construction Method of Robot Transmission System Fault Diagnosis

Knowledge graph technology has important guiding significance for efficient and orderly fault diagnosis of robot transmission system. Taking the historical robot maintenance logs of robot transmission system as the research object, a top-down fault diagnosis event logic knowledge graph construction method is proposed. Firstly, we define event arguments of fault phenomenon and fault cause events, define event argument classes and relation between classes, and construct an event logic knowledge ontology model. According to the event logic knowledge ontology, the fault diagnosis event argument entity and relation in the corpus are labeled, and an event logic knowledge extraction dataset is formed. Secondly, an event argument entity and relation joint extraction model is proposed. Using stacked bidirectional long short-term memory(BiLSTM) to obtain deep context features of text. As a supplement to stacked BiLSTM, self-attention mechanism extracts character dependency features from multiple subspaces, and uses conditional random field(CRF) to realize entity recognition. The character dependency features are mapped to the entity label weight embedding, and spliced with deep context features to extract relations. Bidirectional graph convolutional network(BiGCN) is introduced for relation inference, graph convolution features are used to update deep context features to perform joint extraction in the second phase. Experimental results show that this method can improve the effect of event argument entity and relation joint extraction and is better than other methods. Finally, an event logic knowledge graph of robot transmission system fault diagnosis is constructed, which provides decision support for autonomous fault diagnosis of robot transmission system.

Imaginary Finger Movements Decoding Using Empirical Mode Decomposition and a Stacked BiLSTM Architecture

Motor Imagery Electroencephalogram (MI-EEG) signals are widely used in Brain-Computer Interfaces (BCI). MI-EEG signals of large limbs movements have been explored in recent researches because they deliver relevant classification rates for BCI systems. However, smaller and noisy signals corresponding to hand-finger imagined movements are less frequently used because they are difficult to classify. This study proposes a method for decoding finger imagined movements of the right hand. For this purpose, MI-EEG signals from C3, Cz, P3, and Pz sensors were carefully selected to be processed in the proposed framework. Therefore, a method based on Empirical Mode Decomposition (EMD) is used to tackle the problem of noisy signals. At the same time, the sequence classification is performed by a stacked Bidirectional Long Short-Term Memory (BiLSTM) network. The proposed method was evaluated using k-fold cross-validation on a public dataset, obtaining an accuracy of 82.26%.

Deepfake tweets classification using stacked Bi-LSTM and words embedding.

The spread of altered media in the form of fake videos, audios, and images, has been largely increased over the past few years. Advanced digital manipulation tools and techniques make it easier to generate fake content and post it on social media. In addition, tweets with deep fake content make their way to social platforms. The polarity of such tweets is significant to determine the sentiment of people about deep fakes. This paper presents a deep learning model to predict the polarity of deep fake tweets. For this purpose, a stacked bi-directional long short-term memory (SBi-LSTM) network is proposed to classify the sentiment of deep fake tweets. Several well-known machine learning classifiers are investigated as well such as support vector machine, logistic regression, Gaussian Naive Bayes, extra tree classifier, and AdaBoost classifier. These classifiers are utilized with term frequency-inverse document frequency and a bag of words feature extraction approaches. Besides, the performance of deep learning models is analyzed including long short-term memory network, gated recurrent unit, bi-direction LSTM, and convolutional neural network+LSTM. Experimental results indicate that the proposed SBi-LSTM outperforms both machine and deep learning models and achieves an accuracy of 0.92.

Semantic Features with Contextual Knowledge-Based Web Page Categorization Using the GloVe Model and Stacked BiLSTM

Internet technologies are emerging very fast nowadays, due to which web pages are generated exponentially. Web page categorization is required for searching and exploring relevant web pages based on users’ queries and is a tedious task. The majority of web page categorization techniques ignore semantic features and the contextual knowledge of the web page. This paper proposes a web page categorization method that categorizes web pages based on semantic features and contextual knowledge. Initially, the GloVe model is applied to capture the semantic features of the web pages. Thereafter, a Stacked Bidirectional long short-term memory (BiLSTM) with symmetric structure is applied to extract the contextual and latent symmetry information from the semantic features for web page categorization. The performance of the proposed model has been evaluated on the publicly available WebKB dataset. The proposed model shows superiority over the existing state-of-the-art machine learning and deep learning methods.

A data-driven approach for tool wear recognition and quantitative prediction based on radar map feature fusion

Tool wear monitoring during the cutting process is crucial for ensuring part quality and productivity. A data-driven monitoring approach based on radar map feature fusion is proposed for tool wear recognition and quantitative prediction, aiming at tracking the evolution of tool wear comprehensively. Specifically, the sensitive features from multi-source signals are fused by a radar map, and health indicators capable of characterizing the tool wear evolution are obtained. For the recognition of tool wear state and the quantitative prediction of tool wear values, the Adaboost Decision Tree (Adaboost-DT) ensemble learning model and stacked bi-directional long short-term memory (SBiLSTM) deep learning network are established, respectively. Experimental results demonstrated that the proposed approach could recognize the current wear state quickly and accurately whilst predicting wear values based on limited historical data available. Combining tool wear recognition and prediction results contributes to making a more flexible tool replacement decision in intelligent manufacturing processes.

Intelligent tool wear monitoring based on parallel residual and stacked bidirectional long short-term memory network

Effective tool wear monitoring (TWM) is essential for accurately assessing the degree of tool wear and for timely preventive maintenance. Existing data-driven monitoring methods mainly rely on complex feature engineering, which reduces the monitoring efficiency. This paper proposes a novel TWM model based on a parallel residual and stacked bidirectional long short-term memory (PRes–SBiLSTM) network. First, a parallel residual network (PResNet) is used to extract the multi-scale local features of sensor signals adaptively. Subsequently, a stacked bidirectional long short-term memory (SBiLSTM) network is used to obtain the time-series features related to the tool wear characteristics. Finally, the predicted tool wear value is outputted through a fully connected network. A smoothing correction method is applied to improve the prediction accuracy. The proposed model is experimentally verified to have a high prediction accuracy without sacrificing its generalization ability. A TWM system framework based on the PRes–SBiLSTM network is proposed, which has a certain reference value for TWM in actual industrial environments.

DeepSBD: A Deep Neural Network Model With Attention Mechanism for SocialBot Detection

Online Social Networks (OSNs) are witnessing sophisticated cyber threats, that are generally conducted using fake or compromised profiles. Automated agents ( aka socialbots), a category of sophisticated and modern threat entities, are the native of the social media platforms and responsible for various modern weaponized information-related attacks, such as astroturfing, misinformation diffusion, and spamming. Detecting socialbots is a challenging and vital task due to their deceiving character of imitating human behavior. To this end, this paper presents an attention-aware deep neural network model, DeepSBD , for detecting socialbots on OSNs. The DeepSBD models users’ behavior using profile , temporal , activity , and content information. It jointly models OSN users’ behavior using Bidirectional Long Short Term Memory (BiLSTM) and Convolutional Neural Network (CNN) architectures. It models profile , temporal , and activity information as sequences, which are fed to a two-layers stacked BiLSTM, whereas content information is fed to a deep CNN. We have evaluated DeepSBD over five real-world benchmark datasets and found that it performs significantly better in comparison to the state-of-the-arts and baseline methods. We have also analyzed the efficacy of DeepSBD at different ratios of socialbots and benign users and found that an imbalanced dataset moderately affects the classification accuracy. Finally, we have analyzed the discrimination power of different behavioral components, and it is found that both profile characteristics and content behavior are most impactful, whereas diurnal temporal behavior is the least effective for detecting socialbots on OSNs.

Stacked bidirectional long short-term memory networks for state-of-charge estimation of lithium-ion batteries

State-of-charge (SOC) estimation of lithium-ion batteries based on deep learning techniques, especially recurrent neural networks (RNNs), has recently garnered much attention. However, the potential of RNNs in SOC estimation has not been fully exploited in terms of the capture of temporal dependencies and the depth of model structure. In this paper, a stacked bidirectional long short-term memory (SBLSTM) neural network is proposed for SOC estimation. In contrast to unidirectional RNN-based methods, the proposed model employs bidirectional LSTM layers that enable it to capture battery temporal information in both forward and backward directions and summarize long-term dependencies from past and future contexts. Furthermore, the bidirectional LSTM layers are stacked to construct a deep structure that enables the model to characterize the non-linear and dynamic relationship between the input battery measurements and the output SOC on a layer-by-layer basis. By introducing the stacked multilayer and bidirectional recurrent structure, SBLSTM can completely utilize the battery temporal information to estimate SOC value. The experiments were conducted using two public battery datasets to evaluate the validity and applicability of the SBLSTM, and our findings indicate that it can achieve good SOC estimation accuracy for different battery types at various ambient temperature conditions.

A Stacked BiLSTM Neural Network Based on Coattention Mechanism for Question Answering.

Deep learning is the crucial technology in intelligent question answering research tasks. Nowadays, extensive studies on question answering have been conducted by adopting the methods of deep learning. The challenge is that it not only requires an effective semantic understanding model to generate a textual representation but also needs the consideration of semantic interaction between questions and answers simultaneously. In this paper, we propose a stacked Bidirectional Long Short-Term Memory (BiLSTM) neural network based on the coattention mechanism to extract the interaction between questions and answers, combining cosine similarity and Euclidean distance to score the question and answer sentences. Experiments are tested and evaluated on publicly available Text REtrieval Conference (TREC) 8-13 dataset and Wiki-QA dataset. Experimental results confirm that the proposed model is efficient and particularly it achieves a higher mean average precision (MAR) of 0.7613 and mean reciprocal rank (MRR) of 0.8401 on the TREC dataset.