Bidirectional Long Short-term Memory Network Research Articles

Mathematical modeling of a Hybrid Mutated Tunicate Swarm Algorithm for Feature Selection and Global Optimization

<p>The latest advances in engineering, science, and technology have contributed to an enormous generation of datasets. This vast dataset contains irrelevant, redundant, and noisy features that adversely impact classification performance in data mining and machine learning (ML) techniques. Feature selection (FS) is a preprocessing stage to minimize the data dimensionality by choosing the most prominent feature while improving the classification performance. Since the size data produced are often extensive in dimension, this enhances the complexity of search space, where the maximal number of potential solutions is 2nd for n feature datasets. As n becomes large, it becomes computationally impossible to compute the feature. Therefore, there is a need for effective FS techniques for large-scale problems of classification. Many metaheuristic approaches were utilized for FS to resolve the challenges of heuristic-based approaches. Recently, the swarm algorithm has been suggested and demonstrated to perform effectively for FS tasks. Therefore, I developed a Hybrid Mutated Tunicate Swarm Algorithm for FS and Global Optimization (HMTSA-FSGO) technique. The proposed HMTSA-FSGO model mainly aims to eradicate unwanted features and choose the relevant ones that highly impact the classifier results. In the HMTSA-FSGO model, the HMTSA is derived by integrating the standard TSA with two concepts: A dynamic s-best mutation operator for an optimal trade-off between exploration and exploitation and a directional mutation rule for enhanced search space exploration. The HMTSA-FSGO model also includes a bidirectional long short-term memory (BiLSTM) classifier to examine the impact of the FS process. The rat swarm optimizer (RSO) model can choose the hyperparameters to boost the BiLSTM network performance. The simulation analysis of the HMTSA-FSGO technique is tested using a series of experiments. The investigational validation of the HMTSA-FSGO technique showed a superior outcome of 93.01%, 97.39%, 61.59%, 99.15%, and 67.81% over diverse datasets.</p>

MFOA-Bi-LSTM: An optimized bidirectional long short-term memory model for short-term traffic flow prediction

Within Intelligent Transportation Systems (ITSs), modeling of traffic flow assumes a pivotal role, as it is crucial for alleviating traffic congestion and reducing carbon emissions. Developing a reliable and sustainable model for predicting traffic flow is highly challenging due to the fluctuations and nonlinear characteristics of traffic flow. In addition to being a practical problem, accurately forecasting traffic flow provides substantial challenges to the researchers working in this area. The aim of this study is to develop a complex framework for forecasting short-term traffic flow with the purpose of improving prediction accuracy. In order to attain this goal, a Bidirectional Long Short-Term Memory (Bi-LSTM) model with a Modified Firefly Optimization Algorithm (MFOA) known as MFOA-Bi-LSTM has been proposed. The MFOA optimization technique has been utilized to optimize the hyperparameters of the deep learning models. Moreover, the optimization of the Bi-LSTM network prediction model is achieved through the application of the MFOA technique, which is renowned for its ability to converge quickly, maintain robustness and perform extensive global searches. To better capture complex patterns in the input data and increase prediction accuracy, model’s architecture makes use of the bidirectional capabilities of Bi-LSTM layers. This combination of Bi-LSTM and FOA is proposed for optimizing complex sequence modeling tasks where selecting the right hyperparameters for the model is challenging and it allows for automated hyperparameter tuning and often leads to improved model performance. In terms of performance metrics including Root Mean Square Error (RMSE), Mean Absolute Error (MAE), Mean Absolute Percentage Error (MAPE) and Correlation Coefficient (r), the proposed model significantly outperforms the other selected models and study findings confirm its appropriateness for the predictive task.

Enhancing source code retrieval with joint Bi-LSTM-GNN architecture: A comparative study with ChatGPT-LLM

Retrieving relevant source code from large repositories is a significant and ongoing challenge in the field of software engineering, primarily due to the vast and ever-expanding amount of available code. Existing deep learning methods, although effective to some extent, exhibit limitations in capturing the intricate and complex structural information embedded within source code, which hinders their ability to provide highly accurate retrieval results. This study endeavors to tackle this prominent issue by introducing a novel and innovative approach known as the Joint Bi-directional LSTM and Graph Neural Networks (JBLG) model for source code retrieval. The central aim is to harness the combined strengths and capabilities of Bi-directional Long Short-Term Memory (LSTM) networks and Graph Neural Networks (GNNs) to significantly enhance the model’s capacity to capture and interpret the complex structural characteristics intrinsic to source code. The proposed JBLG model employs a unique fusion of Bi-directional LSTM, which excels in capturing sequential and temporal dependencies within code, and GNN, which is adept at modeling the intricate graph structure of the code. By leveraging this hybrid architecture, the model aims to provide a comprehensive and highly effective solution for source code retrieval tasks. To assess the efficacy of the JBLG model, extensive experiments are conducted, and the model’s performance is evaluated against well-established benchmarks, including LSTM, GNN, and ChatGPT, using two diverse datasets: CodeSearchNet and CosBench datasets. These evaluations span multiple programming languages, ensuring a comprehensive and robust assessment of the model’s capabilities. The experimental results indicate that the JBLG model consistently outperforms its counterparts, including Bi-LSTM, GNN, ChatGPT, and DGMS, across various evaluation metrics. the JBLG model showcases an exceptional ability to handle and extract the intricate structural information inherent in source code, resulting in significantly enhanced retrieval accuracy. The JBLG model emerges as a highly promising solution for real-world source code retrieval applications, with the potential to revolutionize the field. The success of this model underscores the importance of combining deep learning techniques like Bi-directional LSTM and GNNs for tackling complex software engineering challenges. Furthermore, future research directions could involve exploring advanced techniques such as attention mechanisms and extending the model’s applicability to other software engineering tasks like code summarization and code completion. The findings of this study are expected to have a lasting impact on the advancement of source code retrieval methodologies.

Simultaneous temporal and spectral evolution of ultrafast optical pulse propagation using a single bidirectional LSTM network

Numerical simulations of optical pulse evolution are often a cumbersome task, mostly due to the presence of nonlinearities, and standard numerical approaches like splitting methods can be too complex and resource-hungry to compute them. However, under certain conditions, deep learning (DL) techniques can provide accurate alternative approaches to predict such propagation behavior. Here, we show a bidirectional long short-term memory (BiLSTM) network that predicts both temporal and spectral evolution of a short optical pulse going through a 13 m highly nonlinear fiber (HNLF). The BiLSTM performed an RMSE metric of 0.004 and 0.012 for temporal and spectral domains, respectively. A supercontinuum generation was also performed in a 20-cm photonic-crystal fiber, and the BiLSTM returns an RMSE metric of 0.018 and 0.013 for temporal and spectral domains, respectively.

EEG-Based Evaluation of Aesthetic Experience Using BiLSTM Network

Evaluation of aesthetic design fulfills a pivotal function in product development, which urges for an efficacious objective method to measure customers’ experience. The stability and effectiveness of electroencephalography (EEG) make it a suitable tool for aesthetic experience measurement. Nevertheless, existing studies have several limitations, especially regarding the stimuli and the algorithm. The potential of an EEG-based deep learning model has not been verified in pinpointing subtle differences in physical product aesthetics. To fill the research gap in this issue, we recorded EEG signals in real-life scenarios when participants were presented with different types of physical smartphones, and asked participants to rate them from four dimensions of aesthetic experience (arousal, valence, likeness, and aesthetic evaluation). Then, the time–frequency data were fed into a spatial feature extraction network and an attention-based bidirectional long short-term memory (BiLSTM) optimized by the cross-entropy loss function. The result showed that at 16s window size, the four outcome models yielded the best joint recognition performance of aesthetic experience with an average accuracy of over 85% (arousal: 88.10%, valence: 87.97%, likeness: 85.99%, and aesthetic evaluation: 87.23%). It provides an objective cross-subject recognition method with multi-faceted evaluation results of aesthetic experience. Additionally, we verified the ability of EEG as a reliable and informative resource in terms of aesthetic experience evaluation, even with subtle differences. More practically, a future direction of incorporating EEG signals into subjective product aesthetics measurement could be given more credit.

Ultra-Short-term Electric Load Forecasting Based on VMD-BiLSTM Model

Fast and accurate ultra-short-term load forecasting is beneficial for building an efficient and modern smart grid. This paper proposes an ultra-short-term load forecasting model based on Variational Mode Decomposition (VMD) and improved Bi-directional Long Short-Term Memory (BiLSTM) network. The model first decomposes the complex load historical sequence using VMD, reducing the non-stationarity and complexity of the load sequence to facilitate the prediction task using neural networks. Then, the improved BiLSTM network is used to allow input data to skip some transformations in the network, thereby deeply and bidirectionally mining various hidden information from historical sequences. Finally, this paper selects temperature as the factor most correlated with power load forecasting and incorporates it into the prediction process to achieve effective ultra-short-term load forecasting. Compared with existing models such as LSTM, SVM, LeNet, DRNet4-7, and unimproved VMD-BiLSTM based on example data, the simulation results show that the proposed model can predict the load changes within one hour in the future, achieving ultra-short-term load forecasting, and has good prediction accuracy and algorithm robustness.

Medium‐term load forecasting of power system based on BiLSTM and parallel feature extraction network

AbstractWith the diversification of users’ energy demands, accurate load forecasting is an important prerequisite for optimal scheduling and economic operation of the system, but a single‐load forecasting method cannot effectively predict multi‐energy loads accurately. Therefore, this paper proposes a multi‐energy load forecasting method based on bidirectional long short‐term memory (BiLSTM) and parallel feature extraction networks. Firstly, residual network and convolutional block attention module were used to extract the spatial coupling features of multi‐energy load data. Secondly, BiLSTM is used to capture the temporal features and long‐term dependencies in the load data, and the spatial coupling features are fused to obtain non‐linear prediction results. Finally, the non‐linear prediction results and the linear prediction results obtained by using multi‐energy linear regression were linearly superimposed to obtain the final prediction results. In this paper, IES load data of Tempe Campus of Arizona State University was used to verify and compared with several existing methods, and the results showed that Weighted Mean Absolute Percentage Error decreased by more than 20%.

M5UMCB: Prediction of RNA 5-methyluridine sites using multi-scale convolutional neural network with BiLSTM

As a prevalent RNA modification, 5-methyluridine (m5U) plays a critical role in diverse biological processes and disease pathogenesis. High-throughput identification of m5U typically relies on labor-intensive biochemical experiments using various sequencing-based techniques, which are not only time-consuming but also expensive. Consequently, there is a pressing need for more efficient and cost-effective computational methods to complement these high-throughput techniques. In this study, we present m5UMCB, a novel approach that harnesses a multi-scale convolutional neural network (CNN) in tandem with bidirectional long short-term memory (BiLSTM) to recognize m5U sites. Our method involves segmenting RNA sequences into smaller fragments based on a 3-mer length and subsequently mapping each fragment to a lower-dimensional vector representation using the global vectors for word representation (GloVe) technique. Through a series of multi-scale convolution and pooling operations, local features are extracted from RNA sequences and transformed into abstract, high-level features. The feature matrix is then inputted into a BiLSTM network, enabling the capture of contextual information and long-term dependencies within the sequence. Ultimately, a fully connected layer is employed to classify m5U sites. The validation results from 5-fold cross-validation (5-fold CV) test indicate that m5UMCB outperforms existing state-of-the-art predictive methods, demonstrating a 1.98% increase in the area under ROC curve (AUC) and significant improvements in relevant evaluation metrics. We are confident that m5UMCB will serve as a valuable tool for m5U prediction.

WDP-GAN: Weighted Graph Generation With GAN Under Differential Privacy

Many real-world networks can be represented as weighted graphs, where weights represent the closeness or importance of relationships between node pairs. Sharing these graphs is beneficial for many applications while potentially leading to privacy breaches. Variants of deep learning approaches have been developed for synthetic graph publishing, but privacy-preserving graph (especially weighted graph) publishing has not been fully addressed. To bridge this gap, we propose WDP-GAN, a generative adversarial network (GAN) based privacy-preserving weighted graph generation approach, which can generate unlimited synthetic graphs of a given weighted graph while ensuring individual privacy. To do this, we devise a new node sequence sampling method to generate the training set while preserving both the edge weight and topological structure of the original graph. Moreover, we apply the bi-directional long-short term memory (Bi-LSTM) network to capture the interdependence of node pairs. WDP-GAN then approximates the edge weight information using the frequencies of edges produced by the generator. Furthermore, we propose an adaptive gradient perturbation algorithm to improve the speed and stability of the training process while ensuring individual privacy. Theoretical analysis and experiments on real-world network datasets show that WDP-GAN can generate graphs that effectively preserve structural utility while satisfying differential privacy.

Human centric attention with deep multiscale feature fusion framework for activity recognition in Internet of Medical Things

Recent advancements in the Internet of Medical Things (IoMT) have revolutionized the healthcare sector, making it an active research area in the academic and industrial sectors. Following these advances, an automatic Human Activity Recognition (HAR) is now integrated into the IoMT, facilitating remote patient monitoring systems for smart healthcare. However, implementing HAR via computer vision is intricate due to complex spatiotemporal patterns, single stream fusion, and clutter backgrounds. Mainstream approaches practice pre-trained CNN model, which extract non-salient features due to their generalized weight optimization and limited discriminative feature fusion. In addition, their sequential models have inadequate performance in complex scenarios due to the vanishing gradients encountered during backpropagation across multiple layers. In response to these challenges, we propose a multiscale feature fusion framework for both indoor and outdoor environments to enhance HAR in healthcare monitoring systems, which is mainly composed of two stages: First, the proposed Human Centric Attentional Fusion (HCAF) network is fused with the intermediate convolutional feature of lightweight MobileNetV3 backbone to enriches spatial learning capabilities for accurate HAR. Next, a Deep Multiscale Features Fusion (DMFF) network is proposed that enhanced the long-range temporal dependencies by redesigning the traditional bidirectional LSTM network into a residual fashion followed by Sequential Multihead Attention (SMA) to eliminate non-relevant information and optimized spatiotemporal feature vectors. The performance of the proposed fusion model is evaluated on benchmark healthcare and general activity datasets. In the healthcare, we used Multiple Camera Fall and UR Fall Detection datasets that achieved 99.941% and 100% accuracy. Despite this, our fusion strategy is rigorously evaluated over three challenging general HAR datasets, including HMDB51,UCF101, and UCF50, demonstrating 74.942%, 97.337%, and 96.156% superior performance compared to State-of-The-Art (SOTA) methods. The run time analysis shows that the proposed method is 2x times faster than the existing methods.

Utilizing Deep Learning Algorithms for Signal Processing in Electrochemical Biosensors: From Data Augmentation to Detection and Quantification of Chemicals of Interest.

Nanomaterial-based aptasensors serve as useful instruments for detecting small biological entities. This work utilizes data gathered from three electrochemical aptamer-based sensors varying in receptors, analytes of interest, and lengths of signals. Our ultimate objective was the automatic detection and quantification of target analytes from a segment of the signal recorded by these sensors. Initially, we proposed a data augmentation method using conditional variational autoencoders to address data scarcity. Secondly, we employed recurrent-based networks for signal extrapolation, ensuring uniform signal lengths. In the third step, we developed seven deep learning classification models (GRU, unidirectional LSTM (ULSTM), bidirectional LSTM (BLSTM), ConvGRU, ConvULSTM, ConvBLSTM, and CNN) to identify and quantify specific analyte concentrations for six distinct classes, ranging from the absence of analyte to 10 μM. Finally, the second classification model was created to distinguish between abnormal and normal data segments, detect the presence or absence of analytes in the sample, and, if detected, identify the specific analyte and quantify its concentration. Evaluating the time series forecasting showed that the GRU-based network outperformed two other ULSTM and BLSTM networks. Regarding classification models, it turned out signal extrapolation was not effective in improving the classification performance. Comparing the role of the network architectures in classification performance, the result showed that hybrid networks, including both convolutional and recurrent layers and CNN networks, achieved 82% to 99% accuracy across all three datasets. Utilizing short-term Fourier transform (STFT) as the preprocessing technique improved the performance of all datasets with accuracies from 84% to 99%. These findings underscore the effectiveness of suitable data preprocessing methods in enhancing neural network performance, enabling automatic analyte identification and quantification from electrochemical aptasensor signals.

A Comprehensive Hybrid Model for Language-Independent Defect Prediction in Microservices Architecture

The transformation of software development from monolithic frameworks to microservices-based architectures, focusing on the challenges of creating a unified defect prediction model that spans various programming languages in practice of automating integration of code modification into a single codebase. It proposes a hybrid machine learning approach to enhance defect prediction accuracy by integrating different data sources and algorithms. The goal is to create a language and project-independent model. The hybrid model combines Bi-Directional LSTM (BiD-LSTM) networks and Attention mechanisms, static code metrics, and BERT-based language models. BiLSTM-Attention captures temporal dependencies within Abstract Syntax Trees (ASTs), static code metrics provide insights into software complexity, and BERT interprets textual context for a holistic understanding of code snippets. The research methodology involves quantitative techniques, starting with a literature review to establish the theoretical foundation. An empirical study follows, encompassing data gathering, feature crafting and pre-processing, model building, training and evaluation, validation and analysis and conclusions. The research’s insights aim to improve defect prediction techniques, contributing to software engineering’s pursuit of better quality and reliability.

Enhancing Electrical Load Prediction Using a Bidirectional LSTM Neural Network

Precise anticipation of electrical demand holds crucial importance for the optimal operation of power systems and the effective management of energy markets within the domain of energy planning. This study builds on previous research focused on the application of artificial neural networks to achieve accurate electrical load forecasting. In this paper, an improved methodology is introduced, centering around bidirectional Long Short-Term Memory (LSTM) neural networks (NN). The primary aim of the proposed bidirectional LSTM network is to enhance predictive performance by capturing intricate temporal patterns and interdependencies within time series data. While conventional feed-forward neural networks are suitable for standalone data points, energy consumption data are characterized by sequential dependencies, necessitating the incorporation of memory-based concepts. The bidirectional LSTM model is designed to furnish the prediction framework with the capacity to assimilate and leverage information from both preceding and forthcoming time steps. This augmentation significantly bolsters predictive capabilities by encapsulating the contextual understanding of the data. Extensive testing of the bidirectional LSTM network is performed using multiple datasets, and the results demonstrate significant improvements in accuracy and predictive capabilities compared to the previous simpleRNN-based framework. The bidirectional LSTM successfully captures underlying patterns and dependencies in electrical load data, achieving superior performance as gauged by metrics such as root mean square error (RMSE) and mean absolute error (MAE). The proposed framework outperforms previous models, achieving a remarkable RMSE, attesting to its remarkable capacity to forecast impending load with precision. This extended study contributes to the field of electrical load prediction by leveraging bidirectional LSTM neural networks to enhance forecasting accuracy. Specifically, the BiLSTM’s MAE of 0.122 demonstrates remarkable accuracy, outperforming the RNN (0.163), LSTM (0.228), and GRU (0.165) by approximately 25%, 46%, and 26%, in the best variation of all networks, at the 24-h time step, while the BiLSTM’s RMSE of 0.022 is notably lower than that of the RNN (0.033), LSTM (0.055), and GRU (0.033), respectively. The findings highlight the significance of incorporating bidirectional memory and advanced neural network architectures for precise energy consumption prediction. The proposed bidirectional LSTM framework has the potential to facilitate more efficient energy planning and market management, supporting decision-making processes in power systems.

Machinery Prognostics and High-Dimensional Data Feature Extraction Based on a Transformer Self-Attention Transfer Network.

Machinery degradation assessment can offer meaningful prognosis and health management information. Although numerous machine prediction models based on artificial intelligence have emerged in recent years, they still face a series of challenges: (1) Many models continue to rely on manual feature extraction. (2) Deep learning models still struggle with long sequence prediction tasks. (3) Health indicators are inefficient for remaining useful life (RUL) prediction with cross-operational environments when dealing with high-dimensional datasets as inputs. This research proposes a health indicator construction methodology based on a transformer self-attention transfer network (TSTN). This methodology can directly deal with the high-dimensional raw dataset and keep all the information without missing when the signals are taken as the input of the diagnosis and prognosis model. First, we design an encoder with a long-term and short-term self-attention mechanism to capture crucial time-varying information from a high-dimensional dataset. Second, we propose an estimator that can map the embedding from the encoder output to the estimated degradation trends. Then, we present a domain discriminator to extract invariant features from different machine operating conditions. Case studies were carried out using the FEMTO-ST bearing dataset, and the Monte Carlo method was employed for RUL prediction during the degradation process. When compared to other established techniques such as the RNN-based RUL prediction method, convolutional LSTM network, Bi-directional LSTM network with attention mechanism, and the traditional RUL prediction method based on vibration frequency anomaly detection and survival time ratio, our proposed TSTN method demonstrates superior RUL prediction accuracy with a notable SCORE of 0.4017. These results underscore the significant advantages and potential of the TSTN approach over other state-of-the-art techniques.

Attention-based Spatialized Word Embedding Bi-LSTM Model for Sentiment Analysis

Movie reviews provide a medium of communication for the movie fans community. Movie reviews not only help viewers and potential viewers to obtain a general opinion about a movie but also allow the fans to construct an opinion of the movie. In this work, an analysis of over 60,000 movie reviews has been implemented to find meaningful text representation via text embedding. We improved the text embedding by proposing an attention-based Bidirectional Long-Short Term Memory (Bi-LSTM) network by using over 60,000 movie review text data as the training set and over 20,000 movie review text data as the testing set. Based on the data features, we performed a probabilistic analysis of the information related to words and phrases, combined the analysis results with text embedding, spatialized the text embedding, and compared the performance of the proposed attention-based spatialized word embedding Bi-LSTM model with several traditional machine learning models. The attention-based spatialized word embedding Bi-LSTM model proposed in this paper achieves an F1 score of 0.91 on the movie review sentiment classification dataset, with a prediction accuracy of 91%, outperforming the results of the current state-of-the-art research. The model can effectively identify the sentimental tendencies of movie reviews and use the analyzed sentimental tendencies to guide consumers in their consumption and obtain feedback on movie content.

Text localization and recognition of Chinese characters in natural scenes based on improved faster R-CNN

Text detection and recognition are widely used in daily life. Although it is a very rich market, it has very difficult challenges in practical application. The complex and changeable natural scenes lead to the complex background of the text in the image, which also reflects the research value of text information recognition and extraction in natural scenes. To solve the many problems faced by the recognition of Chinese characters, such as complex shapes and diverse structures, this paper uses VGG16 to extract features and introduces a two-layer bidirectional LSTM network. It improves Faster R-CNN by using a RPN to extract candidate boxes and adjust the position of candidate regions. In this paper, the improved model Faster BLSTM-CNN is tested, and the effectiveness experiment of feature extraction, the difference comparison before and after the improvement of the algorithm, and the comparison experiment with the traditional recognition algorithm are carried out respectively. And it finally carried out an experimental comparison of the combination of text recognition and positioning, and obtained the results. The algorithm Faster BLSTM-CNN in this paper is better in the localization and recognition of Chinese characters in the dataset. In the natural scene, the recognition rate of Faster BLSTM-CNN in this paper is 81.54%, the positioning accuracy is 88.14%, and the detection speed is 86 ms, which has improved performance. Therefore, the improvement of Faster R-CNN is effective. It can effectively locate and recognize Chinese characters in natural scenes.

Daily flow prediction of the Huayuankou hydrometeorological station based on the coupled CEEMDAN–SE–BiLSTM model

Enhancing flood forecasting accuracy, promoting rational water resource utilization and management, and mitigating river disasters all hinge on the crucial role of improving the accuracy of daily flow prediction. The coupled model of Complete Ensemble Empirical Mode Decomposition with Adaptive Noise (CEEMDAN), Sample Entropy (SE), and Bidirectional Long Short-Term Memory (BiLSTM) demonstrates higher stability when faced with nonlinear and non-stationary data, stronger adaptability to various types and lengths of time series data by utilizing sample entropy, and significant advantages in processing sequential data through the BiLSTM network. In this study, in the context of predicting daily flow at the Huayuankou Hydrological Station in the lower reaches of the Yellow River, a coupled CEEMDAN–SE–BiLSTM model was developed and utilized. The results showed that the CEEMDAN–SE–BiLSTM coupled model achieved the utmost accuracy in prediction and optimal fitting performance. Compared with the CEEMDAN–SE–LSTM, CEEMDAN–BiLSTM, and BiLSTM coupled models, the root mean square error (RMSE) of this model is reduced by 42.77, 182.02, and 193.71, respectively; the mean absolute error (MAE) is reduced by 37.62, 118.60, and 126.67, respectively; and the coefficient of determination (R2) is increased by 0.0208, 0.1265, 0.1381.

Prediction of Membrane Protein Amphiphilic Helix Based on Horizontal Visibility Graph and Graph Convolution Network.

Membrane protein amphiphilic helices play an important role in many biological processes. Based on the graph convolution network and the horizontal visibility graph the prediction method of membrane protein amphiphilic helix structure is proposed in this paper. The new dataset of amphiphilic helix is constructed. In this paper, we propose the novel feature extraction method, which characterize the amphiphilicity of membrane protein. We also extract three commonly used protein features together with the new features as protein node features. The neighbor information and long-distance dependence information of proteins are further extracted by sliding window and bidirectional long-short term memory network respectively. From the perspective of horizontal visibility algorithm, we transform protein sequences into complex networks to obtain the graph features of proteins. Then, graph convolutional network model is employed to predict the amphiphilic helix structure of membrane protein. A rigorous ten-fold cross-validation shows that the proposed method outperforms other AH prediction methods on the newly constructed dataset.

Shahmukhi named entity recognition by using contextualized word embeddings

Named Entity Recognition (NER) is an imperative Natural Language Processing (NLP) task which intents to identify and classify predefined named entities in a given span of text. For many Western and Asian languages, NER is a systematically premeditated and established task, however, a little work has been done for Shahmukhi. This paper presents Shahmukhi NER with four key contributions. First, a Bi-directional Long-Short Term Memory (BiLSTM) network based NER model has been developed by incorporating various features including character and word embeddings and Part of Speech (POS) tagging. Second, transfer learning has been employed by training context-free Word2Vec and contextualized Embeddings from Language Models (ELMo) word representations. The word representations have been trained using a Shahmukhi corpus of 14.9 million words. Third, we prepared a cleaner version of an existing Shahmukhi NER corpus by performing Unicode normalization and tokenization errands. The corpus has been deduplicated and results are reported on an unseen evaluation set which produced valid results. Fourth, we have studied the impact of two annotation schemes; Inside-Outside (IO) and Inside-Outside-Beginning (IOB) for Shahmukhi. Transfer learning was quite helpful to enhance the performance of NER models especially ELMo embeddings significantly improved the results by prompting contextualized embedding vectors. This is the first study to use character embeddings, POS tagging and transfer learning for Shahmukhi named entity recognition. The IO scheme based model achieved an accuracy of 98.60% with an f-score of 83.75. The IOB scheme based model performed with an accuracy of 98.43% and an f-score of 75.55. These scores are quite promising for an under-resourced morphologically-rich language.

Software defect prediction using a bidirectional LSTM network combined with oversampling techniques

AbstractSoftware defects are a critical issue in software development that can lead to system failures and cause significant financial losses. Predicting software defects is a vital aspect of ensuring software quality. This can significantly impact both saving time and reducing the overall cost of software testing. During the software defect prediction (SDP) process, automated tools attempt to predict defects in the source codes based on software metrics. Several SDP models have been proposed to identify and prevent defects before they occur. In recent years, recurrent neural network (RNN) techniques have gained attention for their ability to handle sequential data and learn complex patterns. Still, these techniques are not always suitable for predicting software defects due to the problem of imbalanced data. To deal with this problem, this study aims to combine a bidirectional long short-term memory (Bi-LSTM) network with oversampling techniques. To establish the effectiveness and efficiency of the proposed model, the experiments have been conducted on benchmark datasets obtained from the PROMISE repository. The experimental results have been compared and evaluated in terms of accuracy, precision, recall, f-measure, Matthew’s correlation coefficient (MCC), the area under the ROC curve (AUC), the area under the precision-recall curve (AUCPR) and mean square error (MSE). The average accuracy of the proposed model on the original and balanced datasets (using random oversampling and SMOTE) was 88%, 94%, And 92%, respectively. The results showed that the proposed Bi-LSTM on the balanced datasets (using random oversampling and SMOTE) improves the average accuracy by 6 and 4% compared to the original datasets. The average F-measure of the proposed model on the original and balanced datasets (using random oversampling and SMOTE) were 51%, 94%, And 92%, respectively. The results showed that the proposed Bi-LSTM on the balanced datasets (using random oversampling and SMOTE) improves the average F-measure by 43 and 41% compared to the original datasets. The experimental results demonstrated that combining the Bi-LSTM network with oversampling techniques positively affects defect prediction performance in datasets with imbalanced class distributions.