Sparse Autoencoder Research Articles

Intelligent Resource Allocation Using an Artificial Ecosystem Optimizer with Deep Learning on UAV Networks

An Unmanned Aerial Vehicle (UAV)-based cellular network over a millimeter wave (mmWave) frequency band addresses the necessities of flexible coverage and high data rate in the next-generation network. But, the use of a wide range of antennas and higher propagation loss in mmWave networks results in high power utilization and UAVs are limited by low-capacity onboard batteries. To cut down the energy cost of UAV-aided mmWave networks, Energy Harvesting (EH) is a promising solution. But, it is a challenge to sustain strong connectivity in UAV-based terrestrial cellular networks due to the random nature of renewable energy. With this motivation, this article introduces an intelligent resource allocation using an artificial ecosystem optimizer with a deep learning (IRA-AEODL) technique on UAV networks. The presented IRA-AEODL technique aims to effectually allot the resources in wireless UAV networks. In this case, the IRA-AEODL technique focuses on the maximization of system utility over all users, combined user association, energy scheduling, and trajectory design. To optimally allocate the UAV policies, the stacked sparse autoencoder (SSAE) model is used in the UAV networks. For the hyperparameter tuning process, the AEO algorithm is used for enhancing the performance of the SSAE model. The experimental results of the IRA-AEODL technique are examined under different aspects and the outcomes stated the improved performance of the IRA-AEODL approach over recent state of art approaches.

Learning-Based Cuckoo Search Algorithm to Schedule a Flexible Job Shop With Sequencing Flexibility.

This work considers an extended version of flexible job-shop problem from a postprinting or semiconductor manufacturing environment, which needs a directed acyclic graph rather than a linear order to describe the precedences among operations. To obtain its reliable and high-quality schedule in a reasonable time, a learning-based cuckoo search (LCS) algorithm is presented. In it, cuckoo search is selected as an optimizer. To produce promising solutions in a high-dimensional solution space, a sparse autoencoder is introduced to compress a high-dimensional solution into an informative low-dimensional one. It extends the application area of autoencoder-embedded evolutionary optimization methods into combinational optimization by developing an improved one-hot encoding method. Then, in order to reveal the linkages among decision variables and enhance the explore ability of the proposed method, a factorization machine (FM) is used, for the first time, to capture the relevant and complementary features of population. Hence, a parallel framework involving three co-evolved subpopulations is constructed. The first one is an autoencoder embedded subpopulation, the second one is assisted by an FM, and the last one undergoes a regular iteration process. To balance the exploration and exploitation of the proposed framework and avoid unnecessary computation, a reinforcement learning algorithm is used to adaptively adjust the proportion of subpopulations and tune parameters of each subpopulation iteratively. Numerical simulations with benchmarks are performed to compare it with CPLEX, some classical heuristics, and several recently developed methods. The results shows that it well outperforms them.

Self-Adaptive Imbalanced Domain Adaptation With Deep Sparse Autoencoder

Domain adaptation aims to transfer knowledge between different domains to develop an effective hypothesis in the target domain with scarce labeled data, which is an effective method for remedying the problem of labeled data requirement in deep learning. In reality, it is unavoidable that the dataset has a large gap in the number of positive and negative instances across different categories in source and target domains, which is the imbalanced domain adaptation problem. However, since the imbalanced degree always varies greatly in different source and target domains datasets, most existing imbalanced domain adaptation models fix the imbalanced parameters which can not adapt to the change of the proportion between positive and negative instances in different domains. To address this problem, in this paper, we propose a self-adaptive imbalanced domain adaptation method via deep sparse autoencoder, which can adjust the model automatically according to the imbalanced extent for bridging the chasm of domains. More specifically, the self-adaptive imbalanced cross entropy loss is designed for emphasizing more on minority categories and compensating the bias of training loss automatically. In addition, to alleviate the deficient problem of labeled data, we further propose the unlabeled information incorporating method by minimizing the distribution discrepancy of high-level representation space between the source and target domains. Experiments on several real-world datasets demonstrate the effectiveness of our method compared to other state-of-the-art methods.

Compressive Reconstruction Based on Sparse Autoencoder Network Prior for Single-Pixel Imaging

The combination of single-pixel imaging and single photon-counting technology enables ultra-high-sensitivity photon-counting imaging. In order to shorten the reconstruction time of single-photon counting, the algorithm of compressed sensing is used to reconstruct the underdetermined image. Compressed sensing theory based on prior constraints provides a solution that can achieve stable and high-quality reconstruction, while the prior information generated by the network may overfit the feature extraction and increase the burden of the system. In this paper, we propose a novel sparse autoencoder network prior for the reconstruction of the single-pixel imaging, and we also propose the idea of multi-channel prior, using the fully connected layer to construct the sparse autoencoder network. Then, take the network training results as prior information and use the numerical gradient descent method to solve underdetermined linear equations. The experimental results indicate that this sparse autoencoder network prior for the single-photon counting compressed images reconstruction has the ability to outperform the traditional one-norm prior, effectively improving the reconstruction quality.

Sparse convolutional autoencoder‐based fault location for drive circuits in nuclear reactors

AbstractDrive circuit is a critical part of instrumentation and control systems in nuclear reactors, and its performance directly influences the operation of nuclear reactors. However, comparing with the open circuit IGBT faults, soft faults caused by the degradation of electronic components present much slighter fluctuations to the performance of drive circuits. If the two fault modes co‐exist, traditional fault diagnosis models are prone to misclassify soft faults as the normal condition. To improve the accuracy of fault diagnosis of drive circuits, it necessitates to accurately locate the faults of drive circuits, while effectively extracting the distinguishable fault features is one of the critical factors for fault location. In this article, a fault location method combining the empirical modal decomposition (EMD) algorithm and sparse convolutional autoencoder (SCAE) is proposed. The EMD algorithm is applied to decompose the three‐phase current signals of drive circuits. An SCAE‐based feature extractor is constructed to capture high‐dimensional and sparse fault feature data with the aid of the powerful feature autonomic extraction capability of deep learning. A deep classifier is designed to locate faults in the driver circuit. A fault simulation model of the drive circuit is developed and the monitor data is collected. The effectiveness of the proposed method is validated via a real case of drive circuit in nuclear reactors.

Exploiting Remote Sensing Imagery for Vehicle Detection and Classification Using an Artificial Intelligence Technique

Remote sensing imagery involves capturing and examining details about the Earth’s surface from a distance, often using satellites, drones, or other aerial platforms. It offers useful data with which to monitor and understand different phenomena on Earth. Vehicle detection and classification play a crucial role in various applications, including traffic monitoring, urban planning, and environmental analysis. Deep learning, specifically convolutional neural networks (CNNs), has revolutionized vehicle detection in remote sensing. This study designs an improved Chimp optimization algorithm with a DL-based vehicle detection and classification (ICOA-DLVDC) technique on RSI. The presented ICOA-DLVDC technique involves two phases: object detection and classification. For vehicle detection, the ICOA-DLVDC technique applies the EfficientDet model. Next, the detected objects can be classified by using the sparse autoencoder (SAE) model. To optimize the SAE’s hyperparameters effectively, we introduce an ICOA which streamlines the parameter tuning process, accelerating convergence and enhancing the overall performance of the SAE classifier. An extensive set of experiments has been conducted to highlight the improved vehicle classification outcomes of the ICOA-DLVDC technique. The simulation values demonstrated the remarkable performance of the ICOA-DLVDC approach compared to other recent techniques, with a maximum accuracy of 99.70% and 99.50% on the VEDAI dataset and ISPRS Postdam dataset, respectively.

Remain useful life forecasting for roller bearings using sparse auto-encoder

A method based on sparse auto-encoder (SAE) in deep learning (DL) for roller bearings remain useful life (RUL) prediction is presented in this paper. Firstly, the roller bearings vibration signals were calculated by different time and frequency domain factors, in which reflect the vibration signals information well. Therefore, the time and frequency domain features were regarded as the input of SAE, then the SAE model in deep learning was used to extract the features through several hidden layers and the sigmoid function was selected as the output function for calculate the prediction value. Finally, compared with other different prediction methods, such as support vector machine (SVM), back propagation (BP) neural network and random forest (RF), the performance of SAE is better than that those models by using mean absolute error (MAE) and root mean square error (RMSE) these two indicators.

A Novel Digital Twin Framework for Aeroengine Performance Diagnosis

Aeroengine performance diagnosis technology is essential for ensuring flight safety and reliability. The complexity of engine performance and the strong coupling of fault characteristics make it challenging to develop accurate and efficient gas path diagnosis methods. To address these issues, this study proposes a novel digital twin framework for aeroengines that achieves the digitalization of physical systems. The mechanism model is constructed at the component level. The data-driven model is built using a particle swarm optimization–extreme gradient boosting algorithm (PSO-XGBoost). These two models are fused using the low-rank multimodal fusion method (LWF) and combined with the sparse stacked autoencoder (SSAE) to form a digital twin framework of the engine for performance diagnosis. Compared to methods that are solely based on mechanism or data, the proposed digital twin framework can effectively use mechanism and data information to improve the accuracy and reliability. The research results show that the proposed digital twin framework has an error rate of 0.125% in predicting gas path parameters and has a gas path fault diagnosis accuracy of 98.6%. Considering that the degradation cost of a typical flight mission for only one aircraft engine after 3000 flight cycles is approximately USD 209.5, the proposed method has good economic efficiency. This framework can be used to improve engine reliability, availability, and efficiency, and has significant value in engineering applications.

Application of machine learning to spectrum and image data

Machine learning is a useful tool when extracting hidden information from complex measurement data obtained via surface analysis, as in secondary ion mass spectrometry. Flexible learning methods often require significant effort to adjust parameters, as these parameters may have a significant effect on results. However, machine learning methods enable the extraction of new information that cannot be found by manual analysis. This paper presents some examples of complex data analyses using conventional multivariate analysis methods based on linear combinations (principal component analysis and multivariate curve resolution), an unsupervised learning method based on artificial neural networks (sparse autoencoder), and a supervised learning method based on decision trees (random forest). To obtain reproducible and useful results from machine learning applications to surface analysis data, the preparation of data sets—including the selection of variables and the raw data conversion process—is crucial. Moreover, sufficient information representing analytical purposes, such as the chemical structures of unknown samples, material types, and physical or chemical properties of particular materials, must be contained in the data set for supervised learning.

A Semi-Supervised Stacked Autoencoder Using the Pseudo Label for Classification Tasks.

The efficiency and cognitive limitations of manual sample labeling result in a large number of unlabeled training samples in practical applications. Making full use of both labeled and unlabeled samples is the key to solving the semi-supervised problem. However, as a supervised algorithm, the stacked autoencoder (SAE) only considers labeled samples and is difficult to apply to semi-supervised problems. Thus, by introducing the pseudo-labeling method into the SAE, a novel pseudo label-based semi-supervised stacked autoencoder (PL-SSAE) is proposed to address the semi-supervised classification tasks. The PL-SSAE first utilizes the unsupervised pre-training on all samples by the autoencoder (AE) to initialize the network parameters. Then, by the iterative fine-tuning of the network parameters based on the labeled samples, the unlabeled samples are identified, and their pseudo labels are generated. Finally, the pseudo-labeled samples are used to construct the regularization term and fine-tune the network parameters to complete the training of the PL-SSAE. Different from the traditional SAE, the PL-SSAE requires all samples in pre-training and the unlabeled samples with pseudo labels in fine-tuning to fully exploit the feature and category information of the unlabeled samples. Empirical evaluations on various benchmark datasets show that the semi-supervised performance of the PL-SSAE is more competitive than that of the SAE, sparse stacked autoencoder (SSAE), semi-supervised stacked autoencoder (Semi-SAE) and semi-supervised stacked autoencoder (Semi-SSAE).

Deep Learning-Based Magnetic Resonance Image Segmentation and Classification for Alzheimer’s Disease Diagnosis

Accurate and rapid detection of Alzheimer’s disease (AD) using magnetic resonance imaging (MRI) gained considerable attention among research workers because of an increased number of current researches being driven by deep learning (DL) methods that have accomplished outstanding outcomes in variety of domains involving medical image analysis. Especially, convolution neural network (CNN) is primarily applied for the analyses of image datasets according to the capability of handling massive unstructured datasets and automatically extracting significant features. Earlier detection is dominant to the success and development interferences, and neuroimaging characterizes the potential regions for earlier diagnosis of AD. The study presents and develops a novel Deep Learning-based Magnetic Resonance Image Segmentation and Classification for AD Diagnosis (DLMRISC-ADD) model. The presented DLMRISC-ADD model mainly focuses on the segmentation of MRI images to detect AD. To accomplish this, the presented DLMRISC-ADD model follows a two-stage process, namely, skull stripping and image segmentation. At the preliminary stage, the presented DLMRISC-ADD model employs U-Net-based skull stripping approach to remove skull regions from the input MRIs. Next, in the second stage, the DLMRISC-ADD model applies QuickNAT model for MRI image segmentation, which identifies distinct parts such as white matter, gray matter, hippocampus, amygdala, and ventricles. Moreover, densely connected network (DenseNet201) feature extractor with sparse autoencoder (SAE) classifier is used for AD detection process. A brief set of simulations is implemented on ADNI dataset to demonstrate the improved performance of the DLMRISC-ADD method, and the outcomes are examined extensively. The experimental results exhibit the effectual segmentation results of the DLMRISC-ADD technique.

A novel method of health indicator construction and remaining useful life prediction based on deep learning

The construction of health indicators (HI) for traditional deep learning requires human training labels and poor interpretability. This paper proposes an HI construction method based on Stacked Sparse Autoencoder (SSAE) and combines SSAE with Long short-term memory (LSTM) network to predict the remaining useful life (RUL). Extracting features from a single domain may result in insufficient feature extraction and cannot comprehensively reflect the degradation status information of mechanical equipment. In order to solve the problem, this article extracts features from time domain, frequency domain, and time-frequency domain to construct a comprehensive original feature set. Based on monotonicity, trendiness, and robustness, the most sensitive features from the original feature set are selected and put into the SSAE network to construct HI for state partitioning, and then LSTM is used for RUL prediction. By comparing with the existing methods, it is proved that the prediction effect of the proposed method in this paper is satisfied.

Preference-inspired coevolutionary algorithm with sparse autoencoder for many-objective optimization

Preference-inspired coevolutionary algorithm with sparse autoencoder for many-objective optimization

Digital twin model for chiller fault diagnosis based on SSAE and transfer learning

The equipment of chiller systems is characterized by a complex mechanical structure and operating environments that vary widely, resulting in high failure rates, energy waste, and costly maintenance. Traditional fault diagnosis methods suffer from low levels of digitalization and intelligence. Therefore, timely and accurate detection and diagnosis of the operating status of chiller systems are critical. To address the above issues, this study proposes a digital twin (DT) model for chiller system fault diagnosis based on stacked sparse auto-encoder (SSAE) and transfer learning (TL). First, a chiller system digital twin mapping model is constructed utilizing digital twin technology. Then, the SSAE model is constructed to provide real-time defect diagnosis and fault result validation. Finally, to address the limited chiller system data in practical applications, the knowledge of fault diagnosis acquired in the source domain is transferred to the target domain via TL. Experimental results demonstrate that the SSAE model outperforms back propagation (BP), recurrent neural network (RNN) and long short-term memory (LSTM) model in term of accuracy. Furthermore, using TL achieves a diagnostic accuracy of over 90% for different degrees of fault severity, with a significant decrease in cross-entropy loss compared to no TL, confirming the effectiveness of the TL method.

Emotion Recognition Based on the Human's Brain

There are several different algorithms to train models in machine learning and for purpose of enhancing the accuracy of the models for emotion recognition systems by using EEG datasets, we can compare different algorithms and methods. To experience more intuitively the different impacts caused by different algorithms and methods, Emotion classification by applying Convolutional Neural Networks, Sparse Auto-encoder, Deep Neural Network, and ZTW-based epoch selection algorithm, these three models of emotion recognition will be mentioned and compared in this essay to find out a way to improve the accuracy of the emotion recognition models based on EEG datasets, which also include the process and final results of these models.

Explainable Multimodal Deep Dictionary Learning to Capture Developmental Differences From Three fMRI Paradigms.

Multimodal-based methods show great potential for neuroscience studies by integrating complementary information. There has been less multimodal work focussed on brain developmental changes. We propose an explainable multimodal deep dictionary learning method to uncover both the commonality and specificity of different modalities, which learns the shared dictionary and the modality-specific sparse representations based on the multimodal data and their encodings of a sparse deep autoencoder. By regarding three fMRI paradigms collected during two tasks and resting state as modalities, we apply the proposed method on multimodal data to identify the brain developmental differences. The results show that the proposed model can not only achieve better performance in reconstruction, but also yield age-related differences in reoccurring patterns. Specifically, both children and young adults prefer to switch among states during two tasks while staying within a particular state during rest, but the difference is that children possess more diffuse functional connectivity patterns while young adults have more focused functional connectivity patterns. To uncover the commonality and specificity of three fMRI paradigms to developmental differences, multimodal data and their encodings are used to train the shared dictionary and the modality-specific sparse representations. Identifying brain network differences helps to understand how the neural circuits and brain networks form and develop with age.

EPILEPTIC EEG SIGNALS RHYTHMS ANALYSIS IN THE DETECTION OF FOCAL AND NON-FOCAL SEIZURES BASED ON OPTIMISED MACHINE LEARNING AND DEEP NEURAL NETWORK ARCHITECTURE

Objective: Most studies in epileptic seizure detection and classification focused on classifying different types of epileptic seizures. However, localization of the epileptogenic zone in epilepsy patient brain’s is paramount to assist the doctor in locating a focal region in patients screened for surgery. Therefore, this paper proposed robust models for the localization of epileptogenic areas for the success of epilepsy surgery. Method: Advanced feature extraction techniques were proposed as effective feature extraction techniques based on Electroencephalogram (EEG) rhythms extracted from Fourier Basel Series Expansion Multivariate Empirical Wavelet Transform (FBSE-MEWT). The proposed extracted EEG rhythms of [Formula: see text] and [Formula: see text] features were used to obtain a joint instantaneous frequency and amplitude components using a sub-band alignment approach. The features are used in Sparse Autoencoder (SAE), Deep Belief Network (DBN), and Support Vector Machine (SVM) with the optimized capability to develop three new models: 1. FMEWT–SVM 2. FMEWT_SAE–SVM, and 3. FMEWT–DBN–SVM. The EEG signal was preprocessed using a proposed Multiscale Principal Component Analysis (mPCA) to denoise the noise embedded in the signal. Main results: The developed models show a significant performance improvement, with the SAE–SVM outperforming other proposed models and some recently reported works in literature with an accuracy of 99.7% using [Formula: see text]-rhythms in channels 1 and 2. Significance: This study validates the EEG rhythm as a means of discriminating the embedded features in epileptic EEG signals to locate the focal and non-focal regions in the epileptic patient’s brain to increase the success of the surgery and reduce computational cost.

HISET: Hybrid interpretable strategies with ensemble techniques for respiratory sound classification

The human respiratory systems can be affected by several diseases and it is associated with distinctive sounds. For advanced biomedical signal processing, one of the most complex issues is automated respiratory sound classification. In this research, five Hybrid Interpretable Strategies with Ensemble Techniques (HISET) which are quite interesting and robust are proposed for the purpose of respiratory sounds classification. The first approach is termed as an Ensemble GSSR technique which utilizes L2 Granger Analysis and the proposed Supportive Ensemble Empirical Mode Decomposition (SEEMD) technique and then Support Vector Machine based Recursive Feature Elimination (SVM-RFE) is used for feature selection and followed by classification with Machine Learning (ML) classifiers. The second approach proposed is the implementation of a novel Realm Revamping Sparse Representation Classification (RR-SRC) technique and third approach proposed is a Distance Metric dependent Variational Mode Decomposition (DM-VMD) with Extreme Learning Machine (ELM) classification process. The fourth approach proposed is with the usage of Harris Hawks Optimization (HHO) with a Scaling Factor based Pliable Differential Evolution (SFPDE) algorithm termed as HHO-SFPDE and it is classified with ML classifiers. The fifth or the final approach proposed analyzes the application of dimensionality reduction techniques with the proposed Gray Wolf Optimization based Support Vector Classification (GWO-SVC) and another parallel approach utilizes a similar kind of analysis with the Grasshopper Optimization Algorithm (GOA) based Sparse Autoencoder. The results are examined for ICBHI dataset and the best results are shown for the 2-class classification when the analysis is carried out with Manhattan distance-based VMD-ELM reporting an accuracy of 95.39%, and for 3-class classification Euclidean distance-based VMD-ELM reported an accuracy of 90.61% and for 4-class classification, Manhattan distance-based VMD-ELM reported an accuracy of 89.27%.

A Duo Autoencoder-SVM based Approach for Secure Performance Monitoring of Industrial Conveyor Belt System

Process industries are fascinated by cyber-physical systems because of the potential to integrate physical systems and the cyber realm, resulting in efficient remote monitoring and control. The conveyor belt system has many critical parameters that require continuous attention, necessitating cyber-physical remote monitoring. Due to cloud-based monitoring of parameters, the system is vulnerable to cyber threats. The proposed technique combines a sparse autoencoder and support vector machine (SVM) to detect false data injection attacks (FDIAs) in the presence of sensor bias fault. The sparse autoencoder extracts sparse features and learns anomaly-free dynamics from the input sensor readings. Then, the trained SVM distinguishes attacks and fault by analysing reconstruction residuals of each measurement reading. The residuals also give an idea about the magnitude of abnormality. The proposed method's efficacy is evaluated in terms of accuracy, precision and false-alarm rate with the help of fault and FDIAs models.

Implementation of a Sequence-to-Sequence Stacked Sparse Long Short-Term Memory Autoencoder for Anomaly Detection on Multivariate Timeseries Data of Industrial Blower Ball Bearing Units.

The advent of Industry 4.0 introduced new ways for businesses to evolve by implementing maintenance policies leading to advancements in terms of productivity, efficiency, and financial performance. In line with the growing emphasis on sustainability, industries implement predictive techniques based on Artificial Intelligence for the purpose of mitigating machine and equipment failures by predicting anomalies during their production process. In this work, a new dataset that was made publicly available, collected from an industrial blower, is presented, analyzed and modeled using a Sequence-to-Sequence Stacked Sparse Long Short-Term Memory Autoencoder. Specifically the right and left mounted ball bearing units were measured during several months of normal operational condition as well as during an encumbered operational state. An anomaly detection model was developed for the purpose of analyzing the operational behavior of the two bearing units. A stacked sparse Long Short-Term Memory Autoencoder was successfully trained on the data obtained from the left unit under normal operating conditions, learning the underlying patterns and statistical connections of the data. The model was evaluated by means of the Mean Squared Error using data from the unit's encumbered state, as well as using data collected from the right unit. The model performed satisfactorily throughout its evaluation on all collected datasets. Also, the model proved its capability for generalization along with adaptability on assessing the behavior of equipment similar to the one it was trained on.