Sparse Autoencoder Research Articles

Uncorrelated Sparse Autoencoder With Long Short-Term Memory for State-of-Charge Estimations in Lithium-Ion Battery Cells

For the safe and reliable operation of battery-driven machines, accurate state-of-charge (SOC) estimations are necessary. Unfortunately, existing methods often fail to identify patterns relevant to long-term SOC estimation due to complex battery cell characteristics such as aging. In this paper, we propose the Uncorrelated Sparse Autoencoder with Long Short-Term Memory (USAL). USAL is a novel neural network that addresses the challenging task of long-term SOC estimation given a limited initial history of a cell’s charge-discharge behavior. USAL uses a multi-task learning strategy to harness the advantages of sparse autoencoders and Long Short-term Memory (LSTM) networks by enforcing correlation penalties. The USAL simultaneously learns to (i) generate a latent space of informative SOC encodings from commonly measured cell characteristics, (ii) penalize for high multicollinearity between encodings, and (iii) identify non-trivial long and short temporal correlations between the encodings using LSTM cells. USAL outperforms benchmarked models in our experiments when trained on five initial charge-discharge cycles across multiple battery cells using three publicly available accelerated aging datasets. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —This paper proposes USAL, a custom-built deep neural network to address the challenging task of long-term SOC estimations in battery cells. Long-term SOC estimation involves estimating SOC for cycles near End-Of-Life (EOL) given some initial charge-discharge cycles. Three fundamental steps involved in long-term SOC estimations using USAL are (i) exploiting a multi-task learning strategy to learn efficient encodings given limited training data, (ii) penalizing these encodings for high correlations to efficiently transform measured inputs into a space of informative features, and (iii) mapping of aging-related trends to support long-term SOC estimations. USAL is designed to be a data-driven SOC estimation method that is (i) capable of alerting the user to a faulty cell when integrated into a real-life Battery Management System (BMS) and (ii) identifying the relative quality of a battery cell from only a few initial charge-discharge cycles.

Enhancing Intrusion Detection Using Binary Arithmetic Optimization with Sparse Auto Encoder for Fog-Assisted Wireless Sensor Networks

Enhancing Intrusion Detection Using Binary Arithmetic Optimization with Sparse Auto Encoder for Fog-Assisted Wireless Sensor Networks

Intrusion Detection System Using Machine Learning by RNN Method

As computer networks continue to grow, network intrusions become more frequent, advanced, and volatile, making it challenging to detect them. This has led to an increase in illegal intrusions that current security tools cannot handle. NIDS is currently available and most reliable ways to monitor network traffic, identify unauthorized usage, and detect malicious attacks. NIDS can provide better visibility of network activity and detect any evidence of attacks and malicious traffic. Recent research has shown that machine learning-based NIDS, particularly with deep learning, is more effective in detecting variants of network attacks compared to traditional rule-based solutions. This proposed model that introduces novel deep learning methodologies for network intrusion detection. The model consists of three approaches: LSTM-RNN, various classifying methodology, and a hybrid Sparse autoencoder with DNN. The LSTM-RNN evaluated NSL-KDD dataset and classified as multi-attack classification. The model outperformed with Adamax optimizer in terms of accuracy, detection rate, and low false alarm rate.

Multiobjective Evolutionary Learning for Multitask Quality Prediction Problems in Continuous Annealing Process.

In industrial production processes, the mechanical properties of materials will directly determine the stability and consistency of product quality. However, detecting the current mechanical property is time-consuming and labor-intensive, and the material quality cannot be controlled in time. To achieve high-quality steel materials, developing a novel intelligent manufacturing technology that can satisfy multitask predictions for material properties has become a new research trend. This article proposes a multiobjective evolutionary learning method based on a two-stage model with topological sparse autoencoder (TSAE) and ensemble learning. For the structure characteristics of a typical autoencoder (AE), a topology-related constraint is incorporated into the loss function of the AE, thus maintaining the global relationship among multistage input data to improve the data reconstruction quality. Then, a sparse representation of the data is added to the AE to achieve dimensionality reduction. Moreover, the extreme gradient boosting (XGBoost) method is applied to predict the mechanical properties of steel materials through collaboration learning mechanisms. To enhance the model accuracy, a multiobjective evolutionary algorithm (MOEA) with a knee solution strategy is used to optimize the network structure and hyperparameters of the two-stage model. Experiments are conducted using real steel production data from a continuous annealing process (CAP). The results verify that the proposed method obtains a higher prediction accuracy than other state-of-the-art methods and can guide practical production and new material design.

A battle royale optimization with feature fusion-based automated fruit disease grading and classification

&lt;abstract&gt; &lt;p&gt;Fruit Disease Detection (FDD) using Computer Vision (CV) techniques is a powerful strategy to accomplish precision agriculture. Because, these techniques assist the farmers in identifying and treating the diseased fruits before it spreads to other plants, thus resulting in better crop yield and quality. Further, it also helps in reducing the usage of pesticides and other chemicals so that the farmers can streamline their efforts with high accuracy and avoid unwanted treatments. FDD and Deep Learning (DL)-based classification involve the deployment of Artificial Intelligence (AI), mainly the DL approach, to identify and classify different types of diseases that affect the fruit crops. The DL approach, especially the Convolutional Neural Network (CNN), has been trained to classify the fruit images as diseased or healthy, based on the presence or absence of the disease symptoms. In this background, the current study developed a new Battle Royale Optimization with a Feature Fusion Based Fruit Disease Grading and Classification (BROFF-FDGC) technique. In the presented BROFF-FDGC technique, the Bilateral Filtering (BF) approach is primarily employed for the noise removal process. Besides, a fusion of DL models, namely Inception v3, NASNet, and Xception models, is used for the feature extraction process with Bayesian Optimization (BO) algorithm as a hyperparameter optimizer. Moreover, the BROFF-FDGC technique employed the Stacked Sparse Autoencoder (SSAE) algorithm for fruit disease classification. Furthermore, the BRO technique is also employed for optimum hyperparameter tuning of the SSAE technique. The proposed BROFF-FDGC system was simulated extensively for validation using the test database and the outcomes established the enhanced performance of the proposed system. The obtained outcomes emphasize the superior performance of the BROFF-FDGC approach than the existing methodologies.&lt;/p&gt; &lt;/abstract&gt;

Channel Estimation and Pilot Design for Uplink Sparse Code Multiple Access System based on Complex-Valued Sparse Autoencoder

Channel estimation is one of the most important aspects of wireless communication. Especially in Sparse Code Multiple Access (SCMA) system, the accuracy of channel estimation has a significant impact on decoding performance. Various methods, so far, have been developed for channel estimation. Most of these methods regard channel estimation as a parameter estimation problem of linear models. However, these methods require lots of time-frequency resources to ensure high estimation accuracy. In massive connection scenarios, high pilot overhead makes the spectrum resource more scarce. Therefore, the drawback of conventional channel estimation methods limits the further improvement of system capacity in the Internet of Things(IoT) when time-frequency resources is restricted. To address this problem, in this paper, we propose an efficient channel estimation scheme and sparse pilot structure design method in SCMA system based on complex-valued sparse autoencoder which is effective to learn features of wireless channel. Complex-valued sparse autoencoder is a kind of neural network with complex-valued weights. It contains two parts: encoder and decoder. In our work, the encoder part is used to realize pilot design. Channel estimation is implemented by the decoder. Complex-valued weights obtained from training are used as baseband pilots. Compared with maximum likelihood channel estimation (MLE) of linear model, the proposed method can achieve higher channel estimation accuracy with more sparse pilot structure. The bit-error rates performance of the SCMA receiver in our work is very close to that of the perfect channel state information (CSI).

Non-line-of-sight multipath classification method for BDS using convolutional sparse autoencoder with LSTM

Non-line-of-sight multipath classification method for BDS using convolutional sparse autoencoder with LSTM

Application of Power Data Mining Analysis in Fault Diagnosis and Preventive Maintenance

Abstract In order to ensure the stable transmission of electric power, it is an effective way to diagnose and maintain the operating status of electric power equipment from the operation data of electric power equipment. This paper uses a stacked sparse autoencoder to design a training model to realize the data function operation function in the fault detection model. After collecting and classifying the power system data, the line current is standardized and transformed. Then, the processed data is input into the stacked sparse autoencoder, and the model is trained layer by layer. On this basis, the long-term memory network model is introduced to establish a fault diagnosis model. To solve the double-sample situation of power data, the maximum mean difference method must be used. A preventive maintenance strategy is constructed based on failure prediction and remaining life to optimize the implementation path. Evaluate the model’s value in terms of its performance, reliability, and economic benefits of preventive O&amp;M methods. However, judging from the fuzzy fault degree, the electrical components with a high probability of failure are T 1, T 3, L 3, L 9, and B 1, and the fuzzy fault degrees are 0.3154, 0.2789, 0.0648, 0.2657, and 0.0678, respectively—fusion of multidimensional evidence. The components most likely to fail are T 1, T 3, and L 9. From the perspective of operation and maintenance costs, when the MMC maintenance time of Dublin Fang Electric Farm is 17 times, the lowest operation and maintenance cost is 22.8615 million yuan.

Comparing the Performance of Deep Denoising Sparse Autoencoder with Other Defense Methods Against Adversarial Attacks for Arabic letters

The aim of this paper is to compare how effectively the Deep Denoising Sparse Autoencoder (DDSA) method performs compared to other defense strategies - like adversarial training, defensive distillation and feature squeezing - in dealing with adversarial attacks for Arabic letters. We strive to evaluate both the accuracy and robustness as well as efficiency of these methods by examining a test set from the Arabic Handwritten Characters Dataset while considering adversarial attacks. Fast Gradient Sign Method (FGSM), Projected Gradient Descent (PGD), and Carlini and Wagner (C&amp;W) are all part of this. Our research findings demonstrate that DDSA surpasses the rest of the defense methods in terms of classification accuracy and robustness. This exceptional performance is due to the distinctive attributes of DDSA, which concentrate on acquiring distinguishing features and integrating spatial information to improve defense against adversarial perturbations. While it necessitates more computational resources, DDSA's superior performance validates the additional expenses, particularly in critical applications where misclassification may have severe implications.

Efficient Intrusion Detection System Data Preprocessing Using Deep Sparse Autoencoder with Differential Evolution

A great amount of data is generated by the Internet and communication areas’ rapid technological improvement, which expands the size of the network. These cutting‐edge technologies could result in unique network attacks that present security risks. This intrusion launches many attacks on the communication network which is to be monitored. An intrusion detection system (IDS) is a tool to prevent from intrusions by inspecting the network traffic and to make sure the network integrity, confidentiality, availability, and robustness. Many researchers are focused to IDS with machine and deep learning approaches to detect the intruders. Yet, IDS face challenges to detect the intruders accurately with reduced false alarm rate, feature selection, and detection. High dimensional data affect the feature selection methods effectiveness and efficiency. Preprocessing of data to make the dataset as balanced, normalized, and transformed data is done before the feature selection and classification process. Efficient data preprocessing will ensure the whole IDS performance with improved detection rate (DR) and reduced false alarm rate (FAR). Since datasets are required for the various feature dimensions, this article proposes an efficient data preprocessing method that includes a series of techniques for data balance using SMOTE, data normalization with power transformation, data encoding using one hot and ordinal encoding, and feature reduction using a proposed deep sparse autoencoder (DSAE) with differential evolution (DE) on data before feature selection and classification. The efficiency of the transformation methods is evaluated with recursive Pearson correlation‐based feature selection and graphical convolution neural network (G‐CNN) methods.

Local Entropy Based Remora Optimization and Sparse Autoencoders for Cancer Diagnosis Through Microarray Gene Expression Analysis

Local Entropy Based Remora Optimization and Sparse Autoencoders for Cancer Diagnosis Through Microarray Gene Expression Analysis

Test Case Prioritization via Embedded Autoencoder Model for Software Quality Assurance

Software quality can be ensured by passing the process of software testing before the software is released. However, the software testing process involves many phases which leads to more resources and time consumption. Test Case Prioritization (TCP) has gained widespread acceptance because it prioritizes the tasks and reduces the number of phases and also produces results in good quality software free from defects. The coverage-based prioritization can be useful to distinguish each test case and result in a better prioritization process by using some algorithm. In this work, we propose a coverage-based prioritized test case generation using an Embedded Auto Encoder (EAE) algorithm which will produce an ordered sequence of the prioritized test cases. Initially, the code coverage for each benchmark has been extracted from the source code repository and is further processed to eliminate the noise in the data. The processed data will be given to the embedded autoencoder which consists of an autoencoder and a sparse autoencoder. Once the modeling is done, the model has been trained with the data generated by the Keras Data Generator class. The efficiency of the proposed EAE technique has been evaluated by using the APFD metric and the observations clearly show that the EAE framework proves to provide an APFD metric of 0.72 on average which is a good value in comparison to the previously deployed methodologies.

Dielectric cylinder discrimination with ANN using phase information of synthesized time-domain response

The application of the three-component stacked artificial neural network (ANN) for discrimination of dielectric cylinders of different diameters using phase information of synthesized time-domain response is considered. The network consists of two sparse autoencoders and the softmax unit. Neural networks are not tied to the frequency range, unlike many well-known methods based on the resonant properties of objects, and they are a powerful tool for object recognition. In contrast to well-known results, information about the phase of the time-domain signal, which is synthesized from multi-frequency data, is used for discrimination. For ANN training, phase images for cylinders with the radius of 15 to 35 mm are obtained using the method of auxiliary sources (MAS) The possibility of successful recognition was confirmed for the case of the diameter deviation of 1 mm and the presence of additive Gaussian noise with SNR of up to 0 dB.

Improved network intrusion classification with attention-assisted bidirectional LSTM and optimized sparse contractive autoencoders

Accurately identifying network intrusion cannot only help individuals and enterprises better deal with network security problems, but also maintain the Internet environment. This work proposes a new hybrid classification method named SABD for network intrusion detection. SABD integrates Stacked sparse contractive autoencoders (SSCA), Attention-based Bidirectional long-term and short-term memory (LSTM), and Decision fusion. Specifically, SSCA is used for extracting features, which are sent to the attention-based bidirectional LSTM for the classification. Besides, an improved optimization algorithm named genetic simulated-annealing-based particle swarm optimization is designed to optimize hyperparameters of SSCA. Finally, the decision fusion algorithm is adopted to integrate classification results of multiple classifiers and yield the final results. Based on experimental results from four different types of data sets, the proposed SABD outperforms its most advanced peers in classification accuracy.

Drug Repositioning Based on Deep Sparse Autoencoder and Drug-Disease Similarity.

Drug repositioning is critical to drug development. Previous drug repositioning methods mainly constructed drug-disease heterogeneous networks to extract drug-disease features. However, these methods faced difficulty when we are using structurally simple models to deal with complex heterogeneous networks. Therefore, in this study, the researchers introduced a drug repositioning method named DRDSA. The method utilizes a deep sparse autoencoder and integrates drug-disease similarities. First, the researchers constructed a drug-disease feature network by incorporating information from drug chemical structure, disease semantic data, and existing known drug-disease associations. Then, we learned the low-dimensional representation of the feature network using a deep sparse autoencoder. Finally, we utilized a deep neural network to make predictions on new drug-disease associations based on the feature representation. The experimental results show that our proposed method has achieved optimal results on all four benchmark datasets, especially on the CTD dataset where AUC and AUPR reached 0.9619 and 0.9676, respectively, outperforming other baseline methods. In the case study, the researchers predicted the top ten antiviral drugs for COVID-19. Remarkably, six out of these predictions were subsequently validated by other literature sources.

Damage Identification and Health Assessment of Crane Girder Structure Under Moving Loads

In the degradation evaluation process of the crane girder structure under moving loads, the complex vibration response and low accuracy of health index standards are the main factors limiting the performance of the evaluation method. A method for identifying and evaluating damage to crane girder structures under moving loads has been proposed. In this method, optimized empirical wavelet transform (EWT) is used for decomposing the vibration signal of the crane girder structure to obtain sensitive components. A health indicator based on power spectrum maximum sequence is constructed by combining structural vibration simulation and response characteristics analysis under moving loads. A degradation index acquisition scheme based on an optimized sparse autoencoder is proposed, which utilizes a simulated annealing algorithm to optimize the model parameters of the sparse autoencoder. The effectiveness of the proposed method is verified through girder structure crack tests under moving loads.

Predicting potential microbe-disease associations based on auto-encoder and graph convolution network

The increasing body of research has consistently demonstrated the intricate correlation between the human microbiome and human well-being. Microbes can impact the efficacy and toxicity of drugs through various pathways, as well as influence the occurrence and metastasis of tumors. In clinical practice, it is crucial to elucidate the association between microbes and diseases. Although traditional biological experiments accurately identify this association, they are time-consuming, expensive, and susceptible to experimental conditions. Consequently, conducting extensive biological experiments to screen potential microbe-disease associations becomes challenging. The computational methods can solve the above problems well, but the previous computational methods still have the problems of low utilization of node features and the prediction accuracy needs to be improved. To address this issue, we propose the DAEGCNDF model predicting potential associations between microbes and diseases. Our model calculates four similar features for each microbe and disease. These features are fused to obtain a comprehensive feature matrix representing microbes and diseases. Our model first uses the graph convolutional network module to extract low-rank features with graph information of microbes and diseases, and then uses a deep sparse Auto-Encoder to extract high-rank features of microbe-disease pairs, after which the low-rank and high-rank features are spliced to improve the utilization of node features. Finally, Deep Forest was used for microbe-disease potential relationship prediction. The experimental results show that combining low-rank and high-rank features helps to improve the model performance and Deep Forest has better classification performance than the baseline model.

Fault diagnosis of brushless DC motor based on Stack Sparse Autoencoder

Because of their simple structure, long service life, high efficiency, etc., brushless DC (BLDC) motors have been widely applied in many fields. In some applications, high requirements for BLDC continuous use of motors, so often to BLDC running state monitoring of motors, realizes the early fault diagnosis, to improve the reliability, safety, and prolonged use. To solve this problem, a BLDC fault diagnosis method based on Fast Fourier Transform (FFT), Stacked Sparse Auto-encoder (SSAE), and soft classifier was proposed. In the laboratory, a BLDC model R-3525 was used as the experimental equipment, and the data of 9 kinds of fault conditions such as bearing damage, cage damage, inner and outer ring damage, and the normal condition of the motor were collected. The collected data are transformed into 28 × 28 two-dimensional data sets by FFT. The feature expressions of various faults are adaptively learned from many data, and the intelligent diagnosis of the motor is realized by the feature function expression. The experimental results demonstrate that, in comparison to the single neural network approach, the stacked method can significantly enhance the precision of fault classification and achieve fault diagnosis for brushless DC motors. This method carries valuable implications for other varieties of BLDC motors.

An Adaptive Parallel Feature Learning and Hybrid Feature Fusion-Based Deep Learning Approach for Machining Condition Monitoring.

The rapid development of information and communication technologies has facilitated machining condition monitoring toward a data-driven paradigm, of which the Industrial Internet of Things (IIoT) serves as the fundamental basis to acquire data from physical equipment with sensing technologies as well as to learn the relationship between the system condition and the collected condition monitoring data. However, most data-driven methods suffer from using a single-domain space, ignoring the importance of the learned features, and failing to incorporate the handcrafted features assisted by domain knowledge. To solve these limitations, a novel deep learning approach is proposed for machining condition monitoring in the IIoT environment, which consists of three phases, including: 1) the unsupervised parallel feature extraction; 2) adaptive feature importance weighting; and 3) hybrid feature fusion. First, separate sparse autoencoders are utilized to conduct the unsupervised parallel feature extraction, which enables to learn abstract feature representation from multiple domain spaces simultaneously. Then, an attention module is designed for the adaptive feature importance weighting, which can assign higher weights to those critical features accordingly. Moreover, a hybrid feature fusion is deployed to complement the automatic feature learning and further yield better model performance by fusing the handcrafted features assisted by domain knowledge. Finally, a real-life case study and extensive experiments have been conducted to show the effectiveness and superiority of the proposed approach.

Anomaly detection of aviation data bus based on SAE and IMD

To detect remote terminal (RT) spoofing attacks on MIL-STD-1553B data bus and prevent the network paralysis of integrated avionics system (IAS) caused by misjudgment, an anomaly detection method of aviation data bus based on the combination of sparse autoencoder (SAE) and integrated mahalanobis distance (IMD) is proposed. Aiming at the communication traffic training set with only normal data, an unsupervised learning algorithm SAE is used to train a model that only represents normal behavior. To combine the feature information of each layer within SAE, the IMD, which can measure the similarity between data characteristics, is used to obtain the anomaly score of test data, and the comprehensive anomaly score (CAS) is obtained by considering the reconstruction error between SAE input and output. To solve the problem that data distribution and detection requirements were not considered in a single threshold, a heuristic multi-threshold selection method is proposed, which maximizes the performance of the classifier by considering the accuracy, youden index (YI), and F1. The experimental results demonstrate the effectiveness and feasibility of the method.