Deep Autoencoder Research Articles

Unveiling the hidden depths: advancements in underwater image enhancement using deep learning and auto-encoders

Unveiling the hidden depths: advancements in underwater image enhancement using deep learning and auto-encoders

Progress and Application of Unsupervised Feature Extraction Methods

Unsupervised feature extraction is crucial in machine learning and data mining for handling high-dimensional and unlabeled data. However, existing methods often ignore feature relationaships, resulting in suboptimal feature subsets. This paper reviews the current state of unsupervised feature extraction methods, discussing the limitations of traditional methods such as Principal Component Analysis (PCA) and Independent Component Analysis (ICA), particularly in terms of interpretability, sensitivity to outliers, and computational resource challenges. In recent years, improvement strategies such as information theory, sparse learning, and deep learning (e.g., deep autoencoders and generative adversarial networks) have significantly progressed in feature extraction. This paper analyzes the practical applications of these methods in image processing, gene analysis, text mining, and network security. For example, in image processing, deep autoencoder-based methods such as Matrix Capsules with EM Routing can effectively extract key features from complex images. In text mining, unsupervised feature selection methods combined with generative adversarial networks significantly improve the efficiency of processing high-dimensional text data. Additionally, this paper explores future research directions such as multimodal data processing, improving real-time processing capabilities, and integration with other machine learning techniques (e.g., reinforcement learning, transfer learning), providing insights for further development of unsupervised feature extraction technologies.

A Smart Tourism Resource Information Management Platform Based on Multi-Source Data Fusion

Leveraging multiple data sources to enhance tourism resource management and visitor behavior analysis has become a key challenge in the context of the booming smart tourism industry. In this study, we explore how to integrate and optimize multiple data sources including social media activities, user reviews, tourism statistics, and geographic information to build a comprehensive information management platform for smart tourism resources. Given the limitations inherent in isolated and decentralized data processing approaches in the smart tourism domain, we propose a new approach using deep learning autoencoders for efficient extraction and fusion of meaningful features from heterogeneous datasets. Our methodology encompasses a rigorous data collection and preprocessing phase, ensuring data quality and consistency, followed by the application of autoencoders to learn high-level feature representations conducive to data integration. The fused data facilitate the development of strategies for the optimal allocation of tourism resources and nuanced analysis of visitor behavior patterns. Experimental evaluations demonstrate the model’s proficiency in capturing intricate data relationships, significantly enhancing the predictive accuracy for tourism demand forecasting, and enabling personalized visitor recommendations. The results underscore the potential of our approach to revolutionize smart tourism management practices by providing actionable insights into resource optimization and visitor engagement strategies, thereby contributing to the sustainable growth of the tourism sector.

A deep neural network to de-noise single-cell RNA sequencing data.

Single-cell RNA sequencing (scRNA-seq), a powerful technique for investigating the transcriptome of individual cells, enables the discovery of heterogeneous cell populations, rare cell types, and transcriptional dynamics in separate cells. Yet, scRNA-seq data analysis is limited by the problem of measurement dropouts, i.e., genes displaying zero expression levels. We introduce ZiPo, a deep artificial neural network for rate estimation and library size prediction in scRNA-seq data which incorporates adjustable zero inflation in the distribution to capture the dropouts. ZiPo builds upon established concepts, including using deep autoencoders and adopting the Poisson and negative binomial distributions, by taking advantage of novel strategies, including library size prediction and residual connections, to improve the overall performance. A significant innovation of ZiPo is the introduction of a scale-invariant loss term, making the weights sparse and, hence, the model biologically more interpretable. ZiPo quickly handles vast singular and mixed datasets, with the processing time directly proportional to the number of cells. In this paper, we demonstrate the power of ZiPo on three datasets and show its advantages over other current techniques. The code used to produce the results in this manuscript is available at https://bitbucket.org/habilzare/ alzheimer/src/master/code/deep/ZiPo/.

Kruskal Szekeres generative adversarial network augmented deep autoencoder for colorectal cancer detection

ABSTRACT Cancer involves abnormal cell growth, with types like intestinal and oesophageal cancer often diagnosed in advanced stages, making them hard to cure. Symptoms are like burning sensations in the stomach and swallowing difficulties are specified as colorectal cancer. Deep learning significantly impacts the medical image processing and diagnosis, offering potential improvements in accuracy and efficiency. The Kruskal Szekeres Generative Adversarial Network Augmented Deep Autoencoder (KSGANA-DA) is introduced for early colorectal cancer detection and it comprises two stages; Initial stage, data augmentation uses Affine Transform via Random Horizontal Rotation and Geometric Transform via Kruskal–Szekeres that coordinates to improve the training dataset diversity, boosting detection performance. The second stage, a Deep Autoencoder Anatomical Landmark-based Image Segmentation preserves edge pixel spatial locations, improving precision and recall for early boundary detection. Experiments validate KSGANA-DA performance and different existing methods are implemented into Python. The results of KSGANA-DA are to provide higher precision by 41%, recall by 7%, and lesser training time by 46% than compared to conventional methods.

Electrical resistivity tomography inversion combining deep variational autoencoder and stochastic adaptive sampling

Geophysical methods, such as electrical resistivity tomography (ERT), can be used to imaging the near-surface electrical resistivity as field measurements depend on the subsurface porosity, water saturation and fluid salinity. ERT has been widely applied to investigate mineral and groundwater resources, and in archaeological, environmental, and engineering studies. The prediction of subsurface electrical conductivity from ERT data requires solving a geophysical inverse problem. For near-surface characterization studies, this is often accomplished with deterministic inverse methods. These methods linearize the problem around an initial solution, and their smoothness depends on an imposed a priori spatial regularization term. Depending on this parameterization, these methodologies might struggle to capture the natural variability of the subsurface. Moreover, deterministic solutions have limited capabilities for uncertainty assessment. On the contrary, stochastic inverse methods can assess uncertainties by predicting multiple model realizations that fit similarly the recorded ERT data. However, they are often more computationally expensive than deterministic solutions. Deep learning algorithms based on deep generative models have been used to re-parametrize model and data spaces into low-dimensional domains and efficiently solve geophysical inverse problems. However, within this context, uncertainty assessment is challenging. We propose a deep convolutional variational autoencoder (VAE) coupled with stochastic adaptive optimization to perform stochastic ERT inversion. Geostatistical simulations of electrical resistivity are used as training data set of the VAE. After training, the VAE generates electrical resistivity models that reproduce the statistics and spatial continuity patterns of the training data set. Then, the VAE latent space is iteratively perturbed and updated with adaptive stochastic sampling based on the misfit between observed and predicted ERT data. The proposed methodology is illustrated in two-dimensional synthetic and real data sets to illustrate the ability of the proposed method to predict reliable electrical resistivity models while generating multiple possible scenarios for uncertainty assessment.

An empirical evaluation of deep autoencoders for anomaly detection in the electricity consumption of buildings

Environmental pollution and energy saving are particularly relevant issues nowadays due to their direct consequences on our daily life and on the future of our planet. Possible countermeasures to reduce pollution and to facilitate energy saving include detecting faults and abnormal usage patterns in the electricity consumption of buildings, which account for 38% of global energy use. In this paper, we focus on data-driven anomaly detection and experimentally compare several topologies of deep autoencoders for detecting anomalies in the electrical energy consumption of buildings. We consider both plain and variational autoencoders, with different mixtures of recurrent, convolutional, and attention layers, and two different anomaly scores, based on reconstruction error and on reconstruction probability. Experiments are performed on two datasets collected from real office buildings. Results suggest that autoencoders represent a valuable tool for detecting anomalies in the electricity consumption of buildings. However, depending on the regularity and noise of the data, the choice of the right autoencoder topology and anomaly score can become important, as some combinations may be better suited than others. Results also show that integrating attention in autoencoders requires particular care due to the bypassing phenomenon. In response, we propose a novel self-attention mechanism, which is less susceptible to bypassing.

Reduced-Order Model of Coal Seam Gas Extraction Pressure Distribution Based on Deep Neural Networks and Convolutional Autoencoders

There has been extensive research on the partial differential equations governing the theory of gas flow in coal mines. However, the traditional Proper Orthogonal Decomposition–Radial Basis Function (POD-RBF) reduced-order algorithm requires significant computational resources and is inefficient when calculating high-dimensional data for coal mine gas pressure fields. To achieve the rapid computation of gas extraction pressure fields, this paper proposes a model reduction method based on deep neural networks (DNNs) and convolutional autoencoders (CAEs). The CAE is used to compress and reconstruct full-order numerical solutions for coal mine gas extraction, while the DNN is employed to establish the nonlinear mapping between the physical parameters of gas extraction and the latent space parameters of the reduced-order model. The DNN-CAE model is applied to the reduced-order modeling of gas extraction flow–solid coupling mathematical models in coal mines. A full-order model pressure field numerical dataset for gas extraction was constructed, and optimal hyperparameters for the pressure field reconstruction model and latent space parameter prediction model were determined through hyperparameter testing. The performance of the DNN-CAE model order reduction algorithm was compared to the POD-RBF model order reduction algorithm. The results indicate that the DNN-CAE method has certain advantages over the traditional POD-RBF method in terms of pressure field reconstruction accuracy, overall structure retention, extremum capture, and computational efficiency.

Video anomaly detection using transformers and ensemble of convolutional auto-encoders

Video anomaly detection is the identification of outliers deviating from the norm within a series of videos. The spatio-temporal dependencies and unstructured nature of videos make video anomaly detection complicated. Many existing methods cannot detect anomalies accurately because they are unable to learn from the learning data effectively and capture dependencies between distant frames. To this end, we propose a model that uses a pre-trained vision transformer and an ensemble of deep convolutional auto-encoders to capture dependencies between distant frames. Moreover, AdaBoost training is used to ensure the model learns every sample in the data properly. To evaluate the method, we conducted experiments on four publicly available video anomaly detection datasets, namely the CUHK Avenue dataset, ShanghaiTech, UCSD Ped1, and UCSD Ped2, and achieved AUC scores of 93.4 %, 78.8 %, 93.5 %, and 95.7 % for these datasets, respectively. The experimental results demonstrate the flexibility and generalizability of the proposed method for video anomaly detection, coming from robust features extracted by a pre-trained vision transformer and efficient learning of data representations by employing the AdaBoost training strategy.

A comparative analysis of deep-learning models for dam discharge forecasting for disaster management

ABSTRACT This paper represents a comprehensive study of dam water discharge prediction using various deep-learning models. The main aim of this paper is to provide a data-driven solution for effective water resource management and thereby controlling the effects of floods and droughts. The Malampuzha dam, which has a spillway, a right bank canal (RBC), and a left bank canal (LBC), is considered for the study. While the spillway helps to control the water level in the reservoir by allowing extra water to flow downstream during times of severe rainfall or high inflow, hence reducing the risk of floods, the LBC and RBC are intended to divert water for agriculture. Altogether, 10 years of consecutive meteorological and dam-related data are utilized for training and testing the models in forecasting water discharge through LBC, RBC, and the spillway. Different deep-learning models, namely, long short-term memory, Bi-directional long-short term memory, recurrent neural network, gated recurrent unit, one-dimensional convolutional neural network (1D-CNN), deep neural network, autoencoder, and residual networks are explored in predicting the accuracy of dam water discharge. After testing eight deep-learning models, it was discovered that 1D-CNN performed well in predicting the discharge of water.

Gyroscope in-assembly drift anomaly detection based on decision re-optimized deep auto-encoder

Abstract In dynamic servo gyroscope assembly, frequent drift performance overproof causes massive part re-assembly and write-off, for which in-assembly gyro drift anomaly detection is required. However, when utilizing common anomaly detection methods, imbalanced assembly data distribution causes severe accuracy reduction and false alarms. To tackle these problems, we first propose a decision re-optimized deep autoencoder model to conduct in-assembly drift anomaly detection under imbalanced assembly data distribution. Second, a decision-based training strategy is introduced to lower the false alarm rate in anomaly detection, for which models based on different training strategies are compared for better performance. Third, a modified SMOTE data augmentation method is utilized to settle the impact of data imbalance under small-sample conditions. The experimental results show that the proposed method can achieve in-assembly drift anomaly detection under imbalanced data distribution with high precision and outperforms all other existing methods, lowering the assembly repetition rate and improving assembly efficiency.

DDDR-33. TRANSCRIPTOMICS-BASED NOVEL DRUG DISCOVERY IN DIFFUSE MIDLINE GLIOMA

Abstract Introduction. Pediatric high-grade gliomas are the most common cause of cancer-related death in children, of which the most malignant and devastating tumors include diffuse midline glioma (DMG). Recent availability of tumor samples and advances in next-generation sequencing enable the profiling of thousands of molecular features in DMG. We employ a system-based approach that uses gene expression as a representation of the molecular state, providing an avenue for therapeutic discovery. Previously, we used this approach to identify repurposed candidates for DMG; however, the final candidate had poor brain penetration, prompting us to discover novel compounds with better brain penetration. To apply it to novel compound discovery, we need the expression profiles of all library compounds, which is impractical for millions of compounds. To overcome this, we have developed a platform, Gene expression profiles Predictor on chemical Structures (GPS), that incorporates machine learning algorithms which leverage existing drug-induced gene expression profiles to predict gene expression based on compound structure. methods. DMG RNA-Seq samples were acquired from St. Jude Children’s Research Hospital and Children’s Brain Tumor Network (CBTN). We developed deep learning autoencoder to search reference normal tissues from Genotype-Tissue Expression (GTEx). A DMG meta-signature was created and fed into GPS to predict drug candidates from the Enamine CNS library, seven of which were experimentally tested in vitro and the most promising one subsequently validated in vivo. Results. Cell proliferation analyzed by MTS assay showed three compounds with IC50< 50 µM in K27M-mutant DMG cell lines (SF8628, DIPG007, SU-DIPG36). Toxicity study showed no adverse effects in the mice treated with 100-200 mg/kg of the most promising compound (IC50 = 1.9 µM in DMG cells). Conclusion. Our novel computational framework leverages deep learning to fill the gap in drug discovery in DMG. Work is underway to determine antitumor activity, brain penetration (via HPLC) and survival benefit in the mice bearing DMG PDX treated with the lead compound.

Enhancing underwater thruster anomaly detection with support vector glow encoding description

As the driving system of autonomous underwater vehicles (AUVs), the healthy operation of underwater thrusters is crucial to ensure the performance of AUVs. Due to the challenge of obtaining faulty samples during normal operation, anomaly detection of underwater thrusters is often carried out using only normal samples. To address this, Support Vector Glow Encoding Description (SVGED) is proposed for anomaly detection of underwater thrusters trained with only normal samples. Specifically, the deep features of the operational vibration signals collected from the underwater thruster are extracted by a deep convolutional autoencoder based on a glow model. The glow-encoded data are then used to identify the boundary of normal operation for the underwater thruster. When a new sample enters the trained SVGED model, it can be effectively identified as representing a normal or anomalous condition of the underwater thruster. The developed SVGED was evaluated using AUV experiments. When using the proposed SVGED method, the accuracy reaches 99.81%, Results show that the proposed method can effectively real time detect anomalies in the underwater thruster compared to other methods. It lays the foundation for ensuring the healthy operation of AUVs.

Deep learning model for smart wearables device to detect human health conduction

With the proliferation of smart wearables, motion wristbands provide a wealth of data essential for comprehending the dynamic nature of health. However, outlier detection is typically necessary due to the presence of unknown outliers in their multidimensional activity data. Conventional approaches frequently result in incorrect object identification due to the curse of dimensionality. Using the Gaussian Mixture Generative Model (GMGM), we provide a method to identify outliers and address this problem. Training on raw data is done using a VariationalAutoencoder (VAE). While avoiding rebuilding mistakes, we want to achieve as many brief features as possible. To predict the likelihood that examples contain many types of data, a DBN will utilise feature extractions and latent distributions in the future. The model's robustness is enhanced by enhancing the VAE, deep learning components, and the GMM overall. When densities surpass the training level, the Gaussian Mixture Model identifies outliers. To achieve this, it makes educated guesses about the densities of each data point. Compared to the deep learning Autoencoding Gaussian Mixture Model (DAGMM), GMGM achieves a 5.5 % higher area under the curve (AUC) on the ODDS standard dataset. Experiments conducted on real datasets further demonstrate the efficacy of this strategy.

Deep Autoencoder for Real-Time Single-Channel EEG Cleaning and Its Smartphone Implementation Using TensorFlow Lite With Hardware/Software Acceleration.

To remove signal contamination in electroencephalogram (EEG) traces coming from ocular, motion, and muscular artifacts which degrade signal quality. To do this in real-time, with low computational overhead, on a mobile platform in a channel count independent manner to enable portable Brain-Computer Interface (BCI) applications. We propose a Deep AutoEncoder (DAE) neural network for single-channel EEG artifact removal, and implement it on a smartphone via TensorFlow Lite. Delegate based acceleration is employed to allow real-time, low computational resource operation. Artifact removal performance is quantified by comparing corrupted and ground-truth clean EEG data from public datasets for a range of artifact types. The on-phone computational resources required are also measured when processing pre-saved data. DAE cleaned EEG shows high correlations with ground-truth clean EEG, with average Correlation Coefficients of 0.96, 0.85, 0.70 and 0.79 for clean EEG reconstruction, and EOG, motion, and EMG artifact removal respectively. On-smartphone tests show the model processes a 4 s EEG window within 5 ms, substantially outperforming a comparison FastICA artifact removal algorithm. The proposed DAE model shows effectiveness in single-channel EEG artifact removal. This is the first demonstration of a low-computational resource deep learning model for mobile EEG in smartphones with hardware/software acceleration. This work enables portable BCIs which require low latency real-time artifact removal, and potentially operation with a small number of EEG channels for wearability. It makes use of the artificial intelligence accelerators found in modern smartphones to improve computational performance compared to previous artifact removal approaches.

Partial Discharge Data Augmentation and Pattern Recognition Method Based on DAE-GAN

Accurate identification of partial discharge (PD) and its types is essential for assessing the operating conditions of electrical equipment. To enhance PD pattern recognition under imbalanced and limited sample conditions, a method based on a Deep Autoencoder-embedded Generative Adversarial Network (DAE-GAN) is proposed. First, the Deep Autoencoder (DAE) is embedded within the Generative Adversarial Network (GAN) to improve the realism of generated samples. Then, complementary PD data samples are introduced during GAN training to address the issue of limited sample size. Lastly, the model’s discriminator is fine-tuned with augmented and balanced training data to enable PD pattern recognition. The DAE-GAN method is used to augment data and recognize patterns in experimental PD signals. The results demonstrate that, under imbalanced and small sample conditions, DAE-GAN generates more authentic PD samples with improved probability distribution fitting compared to other algorithms, leading to varying levels of enhancement in pattern recognition accuracy.

Predicting the Early Detection of Breast Cancer Using Hybrid Machine Learning Systems and Thermographic Imaging

ABSTRACTBreast cancer is a leading cause of mortality among women, emphasizing the critical need for precise early detection and prognosis. However, conventional methods often struggle to differentiate precancerous lesions or tailor treatments effectively. Thermal imaging, capturing subtle temperature variations, presents a promising avenue for non‐invasive cancer detection. While some studies explore thermography for breast cancer detection, integrating it with advanced machine learning for early diagnosis and personalized prediction remains relatively unexplored. This study proposes a novel hybrid machine learning system (HMLS) incorporating deep autoencoder techniques for automated early detection and prognostic stratification of breast cancer patients. By exploiting the temporal dynamics of thermographic data, this approach offers a more comprehensive analysis than static single‐frame approaches. Data processing involves splitting the dataset for training and testing. A predominant infrared image was selected, and matrix factorization was applied to capture temperature changes over time. Integration of convex factor analysis and bell‐curve membership function embedding for dimensionality reduction and feature extraction. The autoencoder deep neural network further reduces dimensionality. HMLS model development included feature selection and optimization of survival prediction algorithms through cross‐validation. Model performance was assessed using accuracy and F‐measure metrics. HMLS, integrating clinical data, achieved 81.6% accuracy, surpassing 77.6% using only convex‐NMF. The best classifier attained 83.2% accuracy on test data. This study demonstrates the effectiveness of thermographic imaging and HMLS for accurate early detection and personalized prediction of breast cancer. The proposed framework holds promise for enhancing patient care and potentially reducing mortality rates.

Deep Autoencoders for Unsupervised Anomaly Detection in Wildfire Prediction

AbstractWildfires pose a significantly increasing hazard to global ecosystems due to the climate crisis. Due to its complex nature, there is an urgent need for innovative approaches to wildfire prediction, such as machine learning. This research took a unique approach, differentiating from classical supervised learning, and addressed the gap in unsupervised wildfire prediction using autoencoders and clustering techniques for anomaly detection. Historical weather and normalized difference vegetation index data sets of Australia for 2005–2021 were utilized. Two main unsupervised approaches were analyzed. The first used a deep autoencoder to obtain latent features, which were then fed into clustering models, isolation forest, local outlier factor and one‐class support vector machines for anomaly detection. The second approach used a deep autoencoder to reconstruct the input data and use reconstruction errors to identify anomalies. Long Short‐Term Memory autoencoders and fully connected (FC) autoencoders were employed in this part, both in an unsupervised way learning only from nominal data. The FC autoencoder outperformed its counterparts, achieving an accuracy of 0.71, an F1‐score of 0.74, and an MCC of 0.42. These findings highlight the practicality of this method, as it effectively predicts wildfires in the absence of ground truth, utilizing an unsupervised learning technique.

FDN-SA: Fuzzy deep neural-stacked autoencoder-based phishing attack detection in social engineering

Phishing attacks have emerged as a major social engineering threat that affects businesses, governments, and general internet users. This work proposes a social engineering phishing detection technique based on Deep Learning (DL). Initially, website data is taken from the dataset. Then, the features of Natural Language Processing (NLP) like bag of words, n-gram, hashtags, sentence length, Term Frequency- Inverse Document Frequency of records (TF-IDF), and all caps are extracted and then web feature extraction is carried out. Later, the feature fusion is done using the Neyman similarity with Deep Belief Network (DBN). Afterwards, oversampling is used for data augmentation to enhance the number of training samples. Lastly, the detection of phishing attacks is performed by employing the proposed Fuzzy Deep Neural-Stacked Autoencoder (FDN-SA). Here, the proposed FDN-SA is developed by combining a Deep Neural Network (DNN), and Deep Stacked Autoencoder (DSA). Further, the investigation of FDN-SA is accomplished based on the accuracy, True Positive Rate (TPR), and True Negative Rate (TNR) and is observed to compute values of 0.920, 0.925, and 0.921, respectively.

XAI-Based Accurate Anomaly Detector That Is Robust Against Black-Box Evasion Attacks for the Smart Grid

In the realm of smart grids, machine learning (ML) detectors—both binary (or supervised) and anomaly (or unsupervised)—have proven effective in detecting electricity theft (ET). However, binary detectors are designed for specific attacks, making their performance unpredictable against new attacks. Anomaly detectors, conversely, are trained on benign data and identify deviations from benign patterns as anomalies, but their performance is highly sensitive to the selected threshold values. Additionally, ML detectors are vulnerable to evasion attacks, where attackers make minimal changes to malicious samples to evade detection. To address these limitations, we introduce a hybrid anomaly detector that combines a Deep Auto-Encoder (DAE) with a One-Class Support Vector Machine (OCSVM). This detector not only enhances classification performance but also mitigates the threshold sensitivity of the DAE. Furthermore, we evaluate the vulnerability of this detector to benchmark evasion attacks. Lastly, we propose an accurate and robust cluster-based DAE+OCSVM ET anomaly detector, trained using Explainable Artificial Intelligence (XAI) explanations generated by the Shapley Additive Explanations (SHAP) method on consumption readings. Our experimental results demonstrate that the proposed XAI-based detector achieves superior classification performance and exhibits enhanced robustness against various evasion attacks, including gradient-based and optimization-based methods, under a black-box threat model.