Autoencoder For Classification Research Articles

An Approach Using in Communication Network Apply in Healthcare System Based on the Deep Learning Autoencoder Classification Optimization Metaheuristic Method

An Approach Using in Communication Network Apply in Healthcare System Based on the Deep Learning Autoencoder Classification Optimization Metaheuristic Method

Deep nonparametric estimation of intrinsic data structures by chart autoencoders: Generalization error and robustness

Autoencoders have demonstrated remarkable success in learning low-dimensional latent features of high-dimensional data across various applications. Assuming that data are sampled near a low-dimensional manifold, we employ chart autoencoders, which encode data into low-dimensional latent features on a collection of charts, preserving the topology and geometry of the data manifold. Our paper establishes statistical guarantees on the generalization error of chart autoencoders, and we demonstrate their denoising capabilities by considering n noisy training samples, along with their noise-free counterparts, on a d-dimensional manifold. By training autoencoders, we show that chart autoencoders can effectively denoise the input data with normal noise. We prove that, under proper network architectures, chart autoencoders achieve a squared generalization error in the order of n−2d+2log4⁡n, which depends on the intrinsic dimension of the manifold and only weakly depends on the ambient dimension and noise level. We further extend our theory on data with noise containing both normal and tangential components, where chart autoencoders still exhibit a denoising effect for the normal component. As a special case, our theory also applies to classical autoencoders, as long as the data manifold has a global parametrization. Our results provide a solid theoretical foundation for the effectiveness of autoencoders, which is further validated through several numerical experiments.

The application value of Gd-EOB-DTPA enhanced MRI based radiomics in the differential diagnosis of iso-or hyperintensity HCC and focal nodular hyperplasia in hepatobiliary phase

Objective: To investigate the application value of Gd-EOB-DTPA enhanced MRI based radiomics model in the differential diagnosis of iso-or hyperintensity HCC and focal nodular hyperplasia (FNH) in hepatobiliary phase. Methods: A total of 88 patients with HCC or FNH confirmed by surgical or puncture pathology who underwent preoperative Gd-EOB-DTPA enhanced MRI (all lesions showed iso-or hyperintensity in hepatobiliary phase) between January 2015 and February 2023 in The First Affiliated Hospital of Soochow University and Nantong No.3 People's Hospital were retrospectively evaluated, which including 58 males and 30 females, aged [M(Q1, Q3)]56 (40, 67) years, including 61 patients with HCC and 27 patients with FNH. The included cases were divided into training (43 cases of HCC, 19 cases of FNH) and validation cohort (18 cases of HCC, 8 cases of FNH) in the ratio of 7∶3 using the random seeding method. A total of 1 781 radiomics features were extracted from Gd-EOB-DTPA enhanced MRI in the arterial, portal and hepatobiliary phases, respectively. The independent three phase models, combined three phases model and combined clinical-radiomics model was established using Auto-Encoder (AE) and Native Bayes (NB) classifier, respectively. The receiver operating characteristic (ROC) curve was used to evaluate the diagnostic performance of these models. DeLong test was used to compare the areas under curve (AUC). Results: In the validation cohort, the combined clinical-radiomics model had the highest AUC (AUC=0.938, 95%CI: 0.828-1.000). The AUC, accuracy, sensitivity, specificity of the clinical-radiomics combined model using AE classifier in the validation cohort were 0.896 (95%CI: 0.760-1.000), 88.5%, 88.9%, 87.5%. The AUC of the clinical-radiomics combined model using NB classifier in the validation cohort were 0.938 (95%CI: 0.828-1.000), 92.3%, 88.9%, 100.0%. Conclusion: Gd-EOB-DTPA enhanced MRI combined clinical-radiomics model has certain value in preoperative differentiation of iso-or hyperintensity in the hepatobiliary phase HCC and FNH, with a high accuracy, sensitivity and specificity.

An automated hybrid attention based deep convolutional capsule with weighted autoencoder approach for skin cancer classification

ABSTRACT Skin cancer is a serious cancer caused by the uncontrollable growth of damaged DNA that leads to death. It is essential to identify the disease at the initial stage and eliminate it from spreading. Hence, this research introduces an automated hybrid deep learning (DL) technique for improving the accuracy of cancer diagnostic systems. In the pre-processing, histogram stretching, colour constancy, hair removal and noise elimination process are undertaken. Then, the Adaptive fuzzy c-means clustering (AFC) is introduced for segmenting the tumour portion. Then, the feature extraction and classification stage is performed using an attention-based deep convolutional capsule weighted auto-encoder classifier network (A-DCCN-WAE) technique. For experimentation, the dataset is collected from International Skin Imaging Collaboration (ISIC) 2019 dataset and is implemented in the PYTHON platform. The proposed method obtains an accuracy of 97%, Precision of 95.6%, F-measure of 97.2%, Mathew's correlation coefficient (MCC) of 90.6% and specificity of 96.9%.

Detection of Cyber Attacks using Machine Learning ‎based Intrusion Detection System for IoT Based Smart ‎Cities

The world’s dynamics is evolving with artificial intelligence (AI) and the results are smart products. A smart city has smart city is collection of smart innovations powered with AI and internet of things (IoTs). Along with the ease and comfort that the concept of a smart city pointed at, many security concerns are being raised that hinders the path of its flourishment. An Intrusion Detection System (IDS) monitors the whole network traffic and alerts in case of any anomaly. A Machine Learning-based IDS intelligently senses the network threats, takes decisions about data packet legibility and alarm the user. Researchers have deployed various ML techniques to IDS to improve the detection accuracy. This work presents a comparative analysis of various ML algorithms trained over UNSW-NB15 dataset. ADA Boost, Linear Support Vector Machine (LSVM), Auto Encoder Classifier, ‎Quadratic Support Vector Machine (QSVM) and Multi-Layer Perceptron algorithms are being employed in the stimulation. ADA Boost showed an excellent accuracy of 98.3% in the results.

Measure of Similarity between GMMs Based on Autoencoder-Generated Gaussian Component Representations

A novel similarity measure between Gaussian mixture models (GMMs), based on similarities between the low-dimensional representations of individual GMM components and obtained using deep autoencoder architectures, is proposed in this paper. Two different approaches built upon these architectures are explored and utilized to obtain low-dimensional representations of Gaussian components in GMMs. The first approach relies on a classical autoencoder, utilizing the Euclidean norm cost function. Vectorized upper-diagonal symmetric positive definite (SPD) matrices corresponding to Gaussian components in particular GMMs are used as inputs to the autoencoder. Low-dimensional Euclidean vectors obtained from the autoencoder’s middle layer are then used to calculate distances among the original GMMs. The second approach relies on a deep convolutional neural network (CNN) autoencoder, using SPD representatives to generate embeddings corresponding to multivariate GMM components given as inputs. As the autoencoder training cost function, the Frobenious norm between the input and output layers of such network is used and combined with regularizer terms in the form of various pieces of information, as well as the Riemannian manifold-based distances between SPD representatives corresponding to the computed autoencoder feature maps. This is performed assuming that the underlying probability density functions (PDFs) of feature-map observations are multivariate Gaussians. By employing the proposed method, a significantly better trade-off between the recognition accuracy and the computational complexity is achieved when compared with other measures calculating distances among the SPD representatives of the original Gaussian components. The proposed method is much more efficient in machine learning tasks employing GMMs and operating on large datasets that require a large overall number of Gaussian components.

A data analytics framework for anomaly detection in flight operations

In the air transport system, there has been a continuous effort to develop policies, tools, and methodologies that increase and standardize safety levels across the entire commercial aviation market, while also enhancing operational efficiency. Furthermore, there is a current focus on proactive approaches for aviation performance management. Within this context, data mining initiatives such as anomaly detection have become more prominent. Dealing with anomalies is a natural step for achieving the goals regarding operational safety and efficiency, as anomalies are often related to hazardous and inefficient operations. In this work, we propose a systematic flight data analytics framework for anomaly detection in flight operations in order to provide a comprehensive and reusable pipeline for model building, application, and explanation. The solution is designed to be applicable to both online and offline regimes and at multiple scales, while also building on domain-expert analysis. We demonstrate the framework applicability in two scenarios of routine flight operations monitoring considering both airline and air traffic management perspectives. In the first one, the framework is applied to aircraft performance data within an unsupervised learning setting with a density-based clustering approach for anomaly detection in landing operations at Minneapolis–Saint Paul International Airport (KMSP). The results are compared with those obtained with exceedance-based methods used in the current practice, revealing the detection of operationally significant anomalies beyond the benchmark. In the second case study, we apply the framework on flight tracking data within a supervised learning setting with the development of an autoencoder classifier for offline anomaly detection in terminal airspace arrival operations at Sao Paulo/Guarulhos International Airport (SBGR). Additionally, supervised learning models are developed for anomaly explanation. The autoencoder classifier was able to detect operationally significant anomalies, while the explanatory models provided novel insights about contributing factors to the anomalies identified. For instance, we learned that anomalous arrival trajectories are more likely to be associated with landing operations on runway 27 under wind scenarios, with an increase in the odds ratio of 62% and 58% for tailwinds and headwinds, respectively. In addition, we also observed a positive association between anomalies and wind gusts situations.

Plastic Extrusion Process Optimization by Inversion of Stacked Autoencoder Classification Machines

AbstractIn the face of climate change and rising energy prices, lowering energy usage of industrial machines is gaining widespread attention. Αpropriate machine settings could lead to reduced production costs and lower environmental impact, while simultaneously maintaining products' quality. However, defining the complex, nonlinear dependencies between these settings and energy usage or quality in manufacturing is often a challenging task. In the presented work, a method for optimized machine settings recommendation is proposed using inverse classification via autoencoders. The algorithm can suggest operation parameters, based on predefined intervals of energy consumption and product properties. The performance is evaluated on data generated by a digital twin of an extrusion process.

Cardiac arrest prediction in smokers using enhanced Artificial Bee Colony algorithm with stacked autoencoder model

In the recent times, the cardiac arrest is a severe heart disease, which results in millions of annual casualties. In this article, the heart rate variability (HRV) parameters are used for predicting cardiac arrest in smokers based on the deep learning techniques. First, the input data is collected from MITU Skillogies dataset, which consists of 1562 smoker and non-smoker instances with 19 HRV input attributes/features. After data collection, the enhanced Artificial Bee Colony algorithm (EABC) is developed for feature selection. The EABC algorithm includes two new multi-objective functions for decreasing the number of attributes in the MITU Skillogies dataset. This mechanism superiorly reduces the burden of computational complexity and improves classification accuracy. Further, the selected attributes are given to the stacked autoencoder classifier for non-cardiac arrest and cardiac arrest classification in smokers for early diagnosis. The extensive experiment showed that the EABC with stacked autoencoder model obtained 96.26% of classification accuracy, which is better related to the traditional machine learning models.

Auto-Encoder Classification Model for Water Crystals with Fine-Tuning

Water is one of the important, though scarce, resources on earth. The 2021 World Water Resource Report claims that environmental challenges threaten the sustainability of water resources. Therefore, it is vital to screen water quality to sustain water resources. Water quality is related to water crystal structure in its solid state. Intelligent models classify water crystals to predict their quality. Methods to analyze water crystals can aid in predicting water quality. Therefore, the major contribution of our research is the prediction of water crystal classes. The proposed model analyzes water crystals in solid states, employing image analysis and the deep learning method. The model specifies several feature groups, including crystal shape factors, solid-state features, crystal geometry and discrete cosine transform coefficients. The model utilizes feature fusion for better training. The proposed model utilized the EP water crystal dataset from the WC image depository and its accuracy was tested with the multi-feature Validation technique. The nature of our data inclined us to utilize F-Measure and sensitivity for the testing phase. Our proposed model outperformed other state of the art water crystal classification models by more than 6% in accuracy and 7% in f-measures, with performance exceeding 11% for triple feature fusion. Furthermore, our model was faster in training time (10% of the training time of the comparative models) and had 1.42 s classification time.

Bagging Supervised Autoencoder Classifier for credit scoring

Automatic credit scoring, a crucial risk management tool for banks and financial institutes, has attracted much attention in the past few decades. As such, various approaches have been developed to accurately and efficiently estimate defaults in loan applicants and seamlessly improve and facilitate decision-making in the lending process. However, the imbalanced nature of credit scoring datasets, as well as the heterogeneous nature of features in credit scoring task pose many challenges in developing and implementing effective credit scoring models, targeting the generalization power of classification models on unseen data. To mitigate these challenges, in this paper, we propose the Bagging Supervised Autoencoder Classifier (BSAC). BSAC is a learning model which simultaneously leverages the superior power of supervised autoencoders and representation learning in classification, as well as the Bagging mechanism to handle the irregularities in feature space. Supervised autoencoder has been exploited to learn an optimal latent space from heterogeneous features and perform classification on top of the learned latent space. In particular, the Bagging mechanism has been employed in the learning process to construct various samples of original data to tackle the problem that arises from imbalanced data and irregularities of features in latent space. Extensive experiments on various real-world and benchmark datasets validate the superiority and robustness of the proposed method in predicting the outcome of loan applications.

Botnet Detection Employing a Dilated Convolutional Autoencoder Classifier with the Aid of Hybrid Shark and Bear Smell Optimization Algorithm-Based Feature Selection in FANETs

Flying ad hoc networks (FANETs) or drone technologies have attracted great focus recently because of their crucial implementations. Hence, diverse research has been performed on establishing FANET implementations in disparate disciplines. Indeed, civil airspaces have progressively embraced FANET technology in their systems. Nevertheless, the FANETs’ distinct characteristics can be tuned and reinforced for evolving security threats (STs), specifically for intrusion detection (ID). In this study, we introduce a deep learning approach to detect botnet threats in FANET. The proposed approach uses a hybrid shark and bear smell optimization algorithm (HSBSOA) to extract the essential features. This hybrid algorithm allows for searching different feature solutions within the search space regions to guarantee a superior solution. Then, a dilated convolutional autoencoder classifier is used to detect and classify the security threats. Some of the most common botnet attacks use the N-BaIoT dataset, which automatically learns features from raw data to capture a malicious file. The proposed framework is named the hybrid shark and bear smell optimized dilated convolutional autoencoder (HSBSOpt_DCA). The experiments show that the proposed approach outperforms existing models such as CNN-SSDI, BI-LSTM, ODNN, and RPCO-BCNN. The proposed HSBSOpt_DCA can achieve improvements of 97% accuracy, 89% precision, 98% recall, and 98% F1-score as compared with those existing models.

Hybrid classical-quantum autoencoder for anomaly detection

We propose a Hybrid classical-quantum Autoencoder (HAE) model, which is a synergy of a classical autoencoder (AE) and a parametrized quantum circuit (PQC) that is inserted into its bottleneck. The PQC augments the latent space, on which a standard outlier detection method is applied to search for anomalous data points within a classical dataset. Using this model and applying it to both standard benchmarking datasets, and a specific use-case dataset which relates to predictive maintenance of gas power plants, we show that the addition of the PQC leads to a performance enhancement in terms of precision, recall, and F1 score. Furthermore, we probe different PQC Ans\"atze and analyse which PQC features make them effective for this task.

Serial Decoders-Based Auto-Encoders for Image Reconstruction

Auto-encoders are composed of coding and decoding units; hence, they hold an inherent potential of being used for high-performance data compression and signal-compressed sensing. The main disadvantages of current auto-encoders comprise the following aspects: the research objective is not to achieve lossless data reconstruction but efficient feature representation; the evaluation of data recovery performance is neglected; it is difficult to achieve lossless data reconstruction using pure auto-encoders, even with pure deep learning. This paper aims at performing image reconstruction using auto-encoders, employs cascade decoders-based auto-encoders, perfects the performance of image reconstruction, approaches gradually lossless image recovery, and provides a solid theoretical and applicational basis for auto-encoders-based image compression and compressed sensing. The proposed serial decoders-based auto-encoders include the architectures of multi-level decoders and their related progressive optimization sub-problems. The cascade decoders consist of general decoders, residual decoders, adversarial decoders, and their combinations. The effectiveness of residual cascade decoders for image reconstruction is proven in mathematics. Progressive training can efficiently enhance the quality, stability, and variation of image reconstruction. It has been shown by the experimental results that the proposed auto-encoders outperform classical auto-encoders in the performance of image reconstruction.

Unsupervised detection of decoupled subspaces: Many-body scars and beyond

Highly excited eigenstates of quantum many-body systems are typically featureless thermal states. Some systems, however, possess a small number of special, low-entanglement eigenstates known as quantum scars. We introduce a quantum-inspired machine-learning platform based on a quantum variational autoencoder (QVAE) that detects families of scar states in spectra of many-body systems. Unlike a classical autoencoder, QVAE performs a parametrized unitary operation, allowing us to compress a single eigenstate into a smaller number of qubits. We demonstrate that the autoencoder trained on a scar state is able to detect the whole family of scar states sharing common features with the input state. We identify families of quantum many-body scars in the PXP model beyond the ${\mathbb{Z}}_{2}$ and ${\mathbb{Z}}_{3}$ families and find dynamically decoupled subspaces in the Hilbert space of disordered, interacting spin-ladder model. The possibility of an automatic detection of subspaces of scar states opens new pathways in studies of models with a weak breakdown of ergodicity and fragmented Hilbert spaces.

Anomaly detection in high-energy physics using a quantum autoencoder

The lack of evidence for new interactions and particles at the Large Hadron Collider (LHC) has motivated the high-energy physics community to explore model-agnostic data-analysis approaches to search for new physics. Autoencoders are unsupervised machine learning models based on artificial neural networks, capable of learning background distributions. We study quantum autoencoders based on variational quantum circuits for the problem of anomaly detection at the LHC. For a QCD $t\overline{t}$ background and resonant heavy-Higgs signals, we find that a simple quantum autoencoder outperforms classical autoencoders for the same inputs and trains very efficiently. Moreover, this performance is reproducible on present quantum devices. This shows that quantum autoencoders are good candidates for analysing high-energy physics data in future LHC runs.

A machine learning classification methodology for Raman Hyperspectral imagery based on auto‐encoders

AbstractRaman spectroscopy is a very sensitive, non‐destructive optical technique that is able to characterize the composition of a material. Although it is a very useful technique, its signals are rather complex and therefore require a lot of knowledge and skills to understand their meaning. This complexity increases further, when it is combined with spatial mapping, creating a three‐dimensional dataset named Raman hyperspectral imagery. The common methodology of analysing such a dataset is to use a generic supervised classification algorithm (i.e., non‐negative least square) combined with reference data. If reference data are not available, unsupervised algorithms are used to perform spectral clusterization of similar areas. This is typically achieved using standard PCA and k‐means methods. The most common problem of processing that significantly affects the results of analysis is that a raw Raman Signal consists of three sources: Raman scattering, photoluminescence and noise. The contribution weights of those components are not standard and are changing based on the sample's chemistry, which makes it very difficult to be solved. This creates the necessity for pre‐processing methods that can split this raw signal to its components. The goals of this project are to improve the unmixing performance using AI and to achieve a successful split of the raw signal into its three components. Through this study, various unsupervised methods are explored to identify the method that fits best to our microscopic RHSI data sets. Our focus is on using autoencoders for the data analysis and on a Gaussian smoothing filter combined with polynomial fittings to split the data. Results show that the selection of a classification methodology depends greatly on the application, but when a large RHSI dataset is available, the newly proposed splitting technique combined with an autoencoder classification presents the best solution.

A novel self-training semi-supervised deep learning approach for machinery fault diagnosis

Fault diagnosis is an indispensable basis for the collaborative maintenance in prognostic and health management. Most of existing data-driven fault diagnosis approaches are designed in the framework of supervised learning, which requires a large number of labelled samples. In this paper, a novel self-training semi-supervised deep learning (SSDL) approach is proposed to train a fault diagnosis model together with few labelled and abundant unlabelled samples. The addressed SSDL approach is realised by initialising a stacked sparse auto-encoder classifier using the labelled samples, and subsequently updating the classifier via sampling a few candidates with most reliable pseudo labels from the unlabelled samples step by step. Unlike the commonly used static sampling strategy in existing self-training semi-supervised frameworks, a gradually exploiting mechanism is proposed in SSDL to increase the number of selected pseudo-labelled candidates gradually. In addition, instead of using the prediction accuracy as the confidence estimation for pseudo-labels, a distance-based sampling criterion is designed to assign the label for each unlabelled sample by its nearest labelled sample based on their Euclidean distances in the deep feature space. The experimental results show that the proposed SSDL approach can achieve good prediction accuracy compared to other self-training semi-supervised learning algorithms.

Robustness of autoencoders for establishing psychometric properties based on small sample sizes: results from a Monte Carlo simulation study and a sports fan curiosity study.

BackgroundThe principal component analysis (PCA) is known as a multivariate statistical model for reducing dimensions into a representation of principal components. Thus, the PCA is commonly adopted for establishing psychometric properties, i.e., the construct validity. Autoencoder is a neural network model, which has also been shown to perform well in dimensionality reduction. Although there are several ways the PCA and autoencoders could be compared for their differences, most of the recent literature focused on differences in image reconstruction, which are often sufficient for training data. In the current study, we looked at details of each autoencoder classifier and how they may provide neural network superiority that can better generalize non-normally distributed small datasets.MethodologyA Monte Carlo simulation was conducted, varying the levels of non-normality, sample sizes, and levels of communality. The performances of autoencoders and a PCA were compared using the mean square error, mean absolute value, and Euclidian distance. The feasibility of autoencoders with small sample sizes was examined.ConclusionsWith extreme flexibility in decoding representation using linear and non-linear mapping, this study demonstrated that the autoencoder can robustly reduce dimensions, and hence was effective in building the construct validity with a sample size as small as 100. The autoencoders could obtain a smaller mean square error and small Euclidian distance between original dataset and predictions for a small non-normal dataset. Hence, when behavioral scientists attempt to explore the construct validity of a newly designed questionnaire, an autoencoder could also be considered an alternative to a PCA.

Machine learning to differentiate small round cell malignant tumors and non-small round cell malignant tumors of the nasal and paranasal sinuses using apparent diffusion coefficient values

ObjectiveWe used radiomics feature–based machine learning classifiers of apparent diffusion coefficient (ADC) maps to differentiate small round cell malignant tumors (SRCMTs) and non-SRCMTs of the nasal and paranasal sinuses.MaterialsA total of 267 features were extracted from each region of interest (ROI). Datasets were randomized into two sets, a training set (∼70%) and a test set (∼30%). We performed dimensional reductions using the Pearson correlation coefficient and feature selection analyses (analysis of variance [ANOVA], relief, recursive feature elimination [RFE]) and classifications using 10 machine learning classifiers. Results were evaluated with a leave-one-out cross-validation analysis.ResultsWe compared the AUC for all the pipelines in the validation dataset using FeAture Explorer (FAE) software. The pipeline using RFE feature selection and Gaussian process classifier yielded the highest AUCs with ten features. When the “one-standard error” rule was used, FAE produced a simpler model with eight features, including Perc.01%, Perc.10%, Perc.90%, Perc.99%, S(1,0) SumAverg, S(5,5) AngScMom, S(5,5) Correlat, and WavEnLH_s-2. The AUCs of the training, validation, and test datasets achieved 0.995, 0.902, and 0.710, respectively. For ANOVA, the pipeline with the auto-encoder classifier yielded the highest AUC using only one feature, Perc.10% (training/validation/test datasets: 0.886/0.895/0.809, respectively). For the relief, the AUCs of the training, validation, and test datasets that used the LRLasso classifier using five features (Perc.01%, Perc.10%, S(4,4) Correlat, S(5,0) SumAverg, S(5,0) Contrast) were 0.892, 0.886, and 0.787, respectively. Compared with the RFE and relief, the results of all algorithms of ANOVA feature selection were more stable with the AUC values higher than 0.800.ConclusionsWe demonstrated the feasibility of combining artificial intelligence with the radiomics from ADC values in the differential diagnosis of SRCMTs and non-SRCMTs and the potential of this non-invasive approach for clinical applications.Key Points• The parameter with the best diagnostic performance in differentiating SRCMTs from non-SRCMTs was the Perc.10% ADC value.• Results of all the algorithms of ANOVA feature selection were more stable and the AUCs were higher than 0.800, as compared with RFE and relief.• The pipeline using RFE feature selection and Gaussian process classifier yielded the highest AUC.