Deep Stacked Autoencoder Research Articles

Social bat optimisation dependent deep stacked auto‐encoder for skin cancer detection

Nowadays, skin cancer is one of the most dangerous forms of cancer found in humans. There are various types of skin cancer, like basal, melanoma, carcinoma, and the squamous cell from which the melanoma is unpredictable. Thus, skin cancer detection in the early stage is very useful to treat it successfully. Hence, this study introduces a new algorithm called social bat optimisation algorithm for skin cancer detection. Initially, the pre-processing is done for the input image to eliminate the noise and artefacts present in the image. Then, the pre-processed image is fed to the feature extraction step where the features are extracted based on convolutional neural network features, and the local pixel pattern-based texture feature (local PPBTF). Here, the PPBTF is the combination of texture features and pixel pattern-based features in which the equation of PPBTF is modified based on the local binary pattern. Subsequently, the classification is done based on the extracted features using a deep stacked auto-encoder, which is trained by the proposed social bat optimisation. The performance of skin cancer detection based on the proposed model is evaluated based on accuracy, sensitivity, and specificity. The proposed model achieves the maximal accuracy of 93.38%, maximal sensitivity of 95%, and the maximal specificity of 96% for K-fold.

Adaptive rag-bull rider: A modified self-adaptive optimization algorithm for epileptic seizure detection with deep stacked autoencoder using electroencephalogram

Electroencephalogram (EEG) signal is widely adapted for monitoring epilepsy to rejuvenate the close-loop brain. Various conventional techniques are devised for identifying seizures that depend on visually analyzing EEG signals that are an expensive and complicated process if there is a rise in numbers of the channel. A new technique, namely Adaptive rag-Rider optimization algorithm (Adaptive rag-ROA) is presented to train deep-stacked autoencoder (Deep SAE) for discovering epileptic seizures. Initially, the EEG signals are given to the pre-processing module, in which noise has been removed by the bandpass filter. Then, the noise removed signals are provided as an input wherein the EEG is divided into various channels and each channel performs the extraction of features. Here, the features such as Taylor-based delta AMS, Holoentropy, fluctuation index, relative energy, tonal power ratio, spectral features, and linear prediction coefficient (LPC) are acquired from each channel. Furthermore, the Probabilistic principal component analysis (PPCA) is adapted to diminish the dimensionality of features. The obtained feature vectors are fed to Deep SAE for epileptic seizure recognition. The Deep SAE training is carried out with Adaptive rag-ROA that is devised by incorporating the Adaptive concept in rag-ROA. Thus, the output generated from the proposed Adaptive rag-ROA-based Deep SAE is adapted to detect seizures from EEG. The proposed Adaptive rag-ROA-based deep SAE outperformed revealing the highest accuracy of 91.5%, the sensitivity of 85.2%, and specificity of 86%.

Unsupervised 3D Motion Summarization Using Stacked Auto-Encoders

In this paper, a deep stacked auto-encoder (SAE) scheme followed by a hierarchical Sparse Modeling for Representative Selection (SMRS) algorithm is proposed to summarize dance video sequences, recorded using the VICON Motion capturing system. SAE’s main task is to reduce the redundant information embedding in the raw data and, thus, to improve summarization performance. This becomes apparent when two dancers are performing simultaneously and severe errors are encountered in the humans’ point joints, due to dancers’ occlusions in the 3D space. Four summarization algorithms are applied to extract the key frames; density based, Kennard Stone, conventional SMRS and its hierarchical scheme called H-SMRS. Experimental results have been carried out on real-life dance sequences of Greek traditional dances while the results have been compared against ground truth data selected by dance experts. The results indicate that H-SMRS being applied after the SAE information reduction module extracts key frames which are deviated in time less than 0.3 s to the ones selected by the experts and with a standard deviation of 0.18 s. Thus, the proposed scheme can effectively represent the content of the dance sequence.

Smartphone Motion Sensor-Based Complex Human Activity Identification Using Deep Stacked Autoencoder Algorithm for Enhanced Smart Healthcare System.

Human motion analysis using a smartphone-embedded accelerometer sensor provided important context for the identification of static, dynamic, and complex sequence of activities. Research in smartphone-based motion analysis are implemented for tasks, such as health status monitoring, fall detection and prevention, energy expenditure estimation, and emotion detection. However, current methods, in this regard, assume that the device is tightly attached to a pre-determined position and orientation, which might cause performance degradation in accelerometer data due to changing orientation. Therefore, it is challenging to accurately and automatically identify activity details as a result of the complexity and orientation inconsistencies of the smartphone. Furthermore, the current activity identification methods utilize conventional machine learning algorithms that are application dependent. Moreover, it is difficult to model the hierarchical and temporal dynamic nature of the current, complex, activity identification process. This paper aims to propose a deep stacked autoencoder algorithm, and orientation invariant features, for complex human activity identification. The proposed approach is made up of various stages. First, we computed the magnitude norm vector and rotation feature (pitch and roll angles) to augment the three-axis dimensions (3-D) of the accelerometer sensor. Second, we propose a deep stacked autoencoder based deep learning algorithm to automatically extract compact feature representation from the motion sensor data. The results show that the proposed integration of the deep learning algorithm, and orientation invariant features, can accurately recognize complex activity details using only smartphone accelerometer data. The proposed deep stacked autoencoder method achieved 97.13% identification accuracy compared to the conventional machine learning methods and the deep belief network algorithm. The results suggest the impact of the proposed method to improve a smartphone-based complex human activity identification framework.

Gated Stacked Target-Related Autoencoder: A Novel Deep Feature Extraction and Layerwise Ensemble Method for Industrial Soft Sensor Application.

These days, data-driven soft sensors have been widely applied to estimate the difficult-to-measure quality variables in the industrial process. How to extract effective feature representations from complex process data is still the difficult and hot spot in the soft sensing application field. Deep learning (DL), which has made great progresses in many fields recently, has been used for process monitoring and quality prediction purposes for its outstanding nonlinear modeling and feature extraction abilities. In this work, deep stacked autoencoder (SAE) is introduced to construct a soft sensor model. Nevertheless, conventional SAE-based methods do not take information related to target values in the pretraining stage and just use the feature representations in the last hidden layer for final prediction. To this end, a novel gated stacked target-related autoencoder (GSTAE) is proposed for improving modeling performance in view of the above two issues. By adding prediction errors of target values into the loss function when executing a layerwise pretraining procedure, the target-related information is used to guide the feature learning process. Besides, gated neurons are utilized to control the information flow from different layers to the final output neuron that take full advantage of different levels of abstraction representations and quantify their contributions. Finally, the effectiveness and feasibility of the proposed approach are verified in two real industrial cases.

Stacked auto-encoder based tagging with deep features for content-based medical image retrieval

Content-based medical image retrieval (CBMIR) is one of the most challenging and ambiguous tasks used to minimize the semantic gap between images and human queries in datasets with rich information content. Similar to the human visual saliency mechanism, CBMIR systems also use the visual features in the images for searching purposes. As a result of this search process, automatically accessing the images is very convenient in large and balanced datasets. Still, it is generally not possible to find such datasets in the medical domain. In this study, a four-step and effective hash code generation technique is presented to reduce the semantic gap between low-level features and high-level semantics for unbalanced medical image datasets. In the first stage, the convolutional neural network (CNN) architecture, the most effective feature representation method available today, is employed to extract discriminative features from images automatically. The features obtained in the last fully connected layer (FCL) at the output of the CNN architecture are used for hash code generation. In the second stage, using the Synthetic Minority Over-sampling Technique (SMOTE), the imbalance between the classes in the dataset is reduced. The solution to the unbalanced problem increases performance by almost 3%. In the third stage, balanced features are converted to a code of 13 symbols by using deep stacked auto-encoder. Finally, this code is translated to the standard 13-character labeling and retrieval code used by the 'Image retrieval in the medical application' (IRMA) dataset, since this is the database with which experiments have been done. IRMA error parameter, classification performance, and retrieval performance of the proposed method are more successful than other state-of-the-art methods.

Stacked isomorphic autoencoder based soft analyzer and its application to sulfur recovery unit

Deep learning is an important and effective tool for process soft sensor modeling in industrial artificial intelligence. Traditional deep learning methods like stacked autoencoder (SAE) usually learn high-level features from their low-level ones progressively by minimizing the reconstruction error for the inputs at each layer. However, the reconstruction cannot be exactly accurate. There is loss cumulation of raw data information from the lowest to the highest levels in SAE. To deal with this problem, a novel deep stacked isomorphic autoencoder (SIAE) is proposed to obtain better feature representation for raw input data in this paper. Different from the original SAE, SIAE aims to extract abstract features at each layer from its previous one by stacking hierarchical isomorphic autoencoders (IAE), in which each IAE reconstructs the same raw input data as well as possible. Thus, SIAE can better describe the complex data patterns and obtain good features for the raw data. Then, SIAE is used to construct soft sensor model for quality prediction. The application on an industrial sulfur recovery unit shows that SIAE can improve the prediction performance for the quality variable.

A Silhouette based Human Action Recognition Technique Using Deep Stacked Auto Encoder

A Silhouette based Human Action Recognition Technique Using Deep Stacked Auto Encoder

A novel stacked sparse denoising autoencoder for mammography restoration to visual interpretation of breast lesion

This paper proposes a deep unsupervised learning based denoising autoencoder model for the restoration of degraded mammogram with visual interpretation of breast lumps or lesion in mammography images (called SSDAE). The proposed model attempts to intensify the underexposed and abnormal structural regions through noise elimination in mammography image. A deep stacked convolutional autoencoder is designed by combining the autoencoder and the deconvolution network which conjointly reduces noisy artifacts and improves image details in mammogram. The proposed SSDAE model takes large noisy mammogram image patches as input and extracts relevant features from target batches. The suggested model can extract relevant features and reduce the dimensionality through sparsity property of the image data while preserving the key features that have been applied to restore image data in feature space. In order to reconstruct a deafening mammogram, the proposed model is carried out through a patched base training on samples to suppress noise thereby preserving structural details in mammography imaging. Experimental results authenticate that the suggested SSDAE model outplays a number of state-of-the-art methods for both X-ray mammogram and ultrasonographic mammogram. The execution speed for target noisy images increases with fine tuning of the network when compared to other algorithms.

Recognizing Polyps in Wireless Endoscopy Images Using Deep Stacked Auto Encoder With Constraint Image Model in Flexible Medical Sensor Platform

In the recent past, Wireless endoscopy (WE) helps physicians to study the digestive tract at the cost of a wide range of images without surgery. The major challenge arises from the complication of robust image characterization in computer-aided WE pictorial diagnostics. The purpose of the research is to provide a biased definition of WE images and to enable medical professionals to automatically identify polyp images. In this paper, the learning feature approach called Deep Stacked Auto Encoder with Constraint Image (DSAECI) for recognizing polyps in WE images has been proposed. This DSAECI differs from the Traditional Method of Auto Encoder (TMAE) due to the introduction of constraint image, created by the nearest neighbor's image and describing inherent object structures. The multiple limitations of images force users to keep images in the same category far away, which share similar learned characteristics and images in various categories and utilized the Flexible medical sensor platform for data analysis. The learned characteristics thus retain large inter-variances and small intra-images. The average total accuracy (OA) of our WE images method is 98.00%. The full results showed that the proposed DSAECI can correctly identify polyps in a WE-image and provide definition characterization for web images. In clinical trials, this approach could be further used to help doctors interpret repetitive images.

Sleep quality of subjects with and without sleep-disordered breathing based on the cyclic alternating pattern rate estimation from single-lead ECG

Objective: The term sleep quality is widely used by researchers and clinicians despite the lack of a definitional consensus, due to different assumptions on quality quantification. It is usually assessed using subject self-reporting, a method that has a major limitation since the subject is a poor self-observer of their sleep behaviors. A more precise method requires the estimation of physiological signals through polysomnography, a procedure that has high costs, is uncomfortable for the subjects and it is unavailable to a large group of the world population. To address these issues, a sleep quality prediction method was developed based on the analysis of the cyclic alternating pattern rate estimated using a single-lead electrocardiogram. Approach: The algorithm analyzes the causality, entropy of the variability and connection of respiratory volume and the N–N interbeat intervals as features for a classifier to assess the cyclic alternating pattern and non-rapid eye movement periods. This information was then combined to estimate the cyclic alternating pattern rate and define the quality of sleep by considering the age-related cyclic alternating pattern rate percentages as a reference threshold. Main results: The best results were achieved using a deep stacked autoencoder as a classifier and employing the minimal-redundancy-maximal-relevance as feature selection algorithm. Data collected from three databases and one hospital were used for training and testing the algorithms, achieving an average accuracy of, respectively, 76% and 77% for the cyclic alternating pattern and non-rapid eye movement sleep classification. The predicted sleep quality achieved a high agreement when considering either the cyclic alternating pattern rate, the arousal index, apnea-hypopnea index or the sleep efficiency as quantification for sleep quality. A moderate correlation was achieved with the Epworth sleepiness score and Pittsburgh sleep quality index. Total sleep time presented a higher variation on the correlation analysis. Significance: The developed method is capable of estimating the sleep quality and is characterized by a low intra-individual variability. It only requires a small number of sensors that can easily be self-assembled, and could possibly lead to new developments in sleep quality estimation by home monitoring devices.

An integrated feature learning approach using deep learning for travel time prediction

Travel time data is a vital factor for numbers of performance measures in transportation systems. Travel time prediction is both a challenging and interesting problem in ITS, because of the underlying traffic and events’ hidden patterns. In this study, we propose a multi-step deep-learning-based algorithm for predicting travel time. Our algorithm starts with data pre-processing. Then, the data is augmented by incorporating external datasets. Moreover, extensive feature learning and engineering such as spatiotemporal feature analysis, feature extraction, and clustering algorithms is applied to improve the feature space. Furthermore, for representing features we used a deep stacked autoencoder with dropout layer as regularizer. Finally, a deep multi-layer perceptron is trained to predict travel times. For testing our predictive accuracy, we used a 5-fold cross validation to test the generalization of our predictive model. As we observed, the performance of the proposed algorithm is on average 4 min better than applying the deep neural network to the initial feature space. Furthermore, we have noticed that representation learning using stacked autoencoders makes our learner robust to overfitting. Moreover, our algorithm is capable of capturing the general dynamics of the traffic, however further works need to be done for some rare events which impact travel time prediction significantly.

An Effective Cell Pathology Image Detection Method Based on Deep Stacked Auto-Encoder Combined with Random Forest

Cytopathology mainly studies the causes and pathogenesis of disease occurrence, as well as the changes in the physiological functions of cells during the process of disease occurrence, providing a basis for the prevention and diagnosis of diseases. Using computational simulation technology to accurately detect cell pathology images can effectively avoid the influence of subjective factors in diagnostic operation by pathologist, and provide reliable technical means for case diagnosis. In this study, we design an effective model to detect cell pathology images based on deep stacked auto-encoder algorithm. Firstly, the cell pathology images are preprocessed by standardization. Secondly, the processed images are sent to the stacked auto-encoder algorithm to extract theirs hidden feature information. Finally, the random forest algorithm is used to quickly and accurately detect theirs category. We introduce cross-validation in our experiments to test the robustness and stability of the results. In addition, we also compared the model based on a k-nearest neighbor classifier and achieved good results. These excellent experiment results indicated that the proposed method can accurately detect cell pathology images and can be used as a reliable tool for pathologists to diagnose rapidly.

Cyber Physical Security Analytics for Transactive Energy Systems

With the significant increase in integration of renewable energy generation into the electric grid, market-based transactive exchanges between energy producers and prosumers will become more common. Transactive energy systems (TESs) employ economic and control mechanisms to dynamically balance the demand and supply across the electrical grid. Emerging transactive control mechanism depends on a large number of distributed edge-computing and Internet of Things (IoT) devices making autonomous/semi-autonomous decisions on energy production, and demand response. However, the electric grid cyber assets and the IoT devices are increasingly vulnerable to attack. TES will likely have similar vulnerabilities and cyber attacks especially with financial interest motives of stakeholders, which could affect the operation of the power grid. Therefore, new analytical methods are needed to continuously monitor these systems operations and detect malicious activity. In this research work, various components of transactive energy systems are modeled and simulated in detail. Various cyber attack models are developed based on threats identified in TES. A deep learning approach called deep stacked autoencoder (SAE) is utilized to detect possible anomalies in the market and physical system measurements. The proposed unsupervised technique is validated for satisfactory performance to detect anomalies and trigger a further investigation for root cause analysis using end-to-end TES testbed and use case.

Learning deep neural networks for node classification

Deep Neural Network (DNN) has made great leaps in image classification and speech recognition in recent years. However, employing DNN for node classification such as in social network remains to be a non-trivial problem. Moreover, the current advanced method of implementing node classification tasks usually takes two steps, i.e. firstly, the embedding vector of the node is obtained through network embedding and then the classifier such as SVM is leveraged to do the task. Distinctly, this may only get the suboptimal solution of the problem. To settle the above issues, a novel Deep Neural Network method for node classification named DNNNC is proposed in the framework of Deep Learning. Specifically, we first get the positive pointwise mutual information (PPMI) matrix from the given adjacency matrix. Then, the data is fed to deep neural network composed of deep stacked sparse autoencoders and softmax layer, which could learn the node representation while encoding the rich nonlinear structural and semantic information and could be well trained for node classification under the DNN framework. Extensive experiments are conducted on real-world network datasets for node classification task and have shown that the proposed model DNNNC outperforms the state-of-the-art method in the view of superior performance.

KDSAE: Chronic kidney disease classification with multimedia data learning using deep stacked autoencoder network

In recent times, Chronic Kidney Disease (CKD) has affected more than 10% of the population worldwide and millions of people die every year. So, early-stage detection of CKD could be beneficial for increasing the life expectancy of suffering patients and reducing the treatment cost. It is required to build such a multimedia driven model which can help to diagnose the disease efficiently with higher accuracy before leading to worse conditions. Various techniques related to conventional machine learning models have been used by researchers in the past time without involvement of multimodal data-driven learning. This research paper offers a novel deep learning framework for chronic kidney disease classification using stacked autoencoder model utilizing multimedia data with a softmax classifier. The stacked autoencoder helps to extract the useful features from the dataset and then a softmax classifier is used to predict the final class. It has experimented on UCI dataset which contains early stages of 400 CKD patients with 25 attributes, which is a binary classification problem. Precision, recall, specificity and F1-score were used as evaluation metrics for the assessment of the proposed network. It was observed that this multimodal model outperformed the other conventional classifiers used for chronic kidney disease with a classification accuracy of 100%.

Hexagonal Volume Local Binary Pattern (H-VLBP) with deep stacked autoencoder for Human Action Recognition

Human action recognition plays a significant role in a number of computer vision applications. This work is based on three processing stages. In the first stage, discriminative frames are selected as representative frames per action to minimize the computational cost and time. In the second stage, novel neighbourhood selection approaches based on geometric shapes including triangle, quadrilateral, pentagon, hexagon, octagon and heptagon are used in Volumetric Local Binary Pattern (VLBP) to extract the features from frame sequences based on motion and appearance information. Hexagonal Volume Local Binary Pattern (H-VLBP) descriptor has been found to produce better results among all other novel geometric shape based neighbourhood selection approaches for human action recognition. However, the dimensionality of extracted feature from H-VLBP is too large. Therefore, the deep stacked autoencoder is used for dimensionality reduction with the decoder layer replaced by softmax layer for performing multi-class recognition. The developed approach is applied to four publicly available benchmark datasets, namely KTH, Weizmann, UCF11 dataset and IXMAS dataset for human action recognition. The results obtained show that the proposed approach outperforms the state-of-art techniques. Moreover, the approach has been tested with a synthetic dataset and better results have been obtained. This illustrates the effectiveness of the approach in real time environment.

Unsupervised tumor detection in Dynamic PET/CT imaging of the prostate.

Early detection and localization of prostate tumors pose a challenge to the medical community. Several imaging techniques, including PET, have shown some success. But no robust and accurate solution has yet been reached. This work aims to detect prostate cancer foci in Dynamic PET images using an unsupervised learning approach. The proposed method extracts three feature classes from 4D imaging data that include statistical, kinetic biological and deep features that are learned by a deep stacked convolutional autoencoder. Anomalies, which are classified as tumors, are detected in feature space using density estimation. The proposed algorithm generates promising results for sufficiently large cancer foci in real PET scans imaging where the foci is not viewed by the tomographic devices used for detection.

Deep stacked sparse auto-encoders for prediction of post-operative survival expectancy in thoracic lung cancer surgery.

Lung cancer is the leading cause of cancer death in men and women. The prognostic value of survival after lung cancer surgery has an important role in decision-making for surgeons and patients. The combination of clinical features and CT scan information for diagnosis, treatment and survival of patients with lung cancer increases the accuracy of prediction using machine learning. Therefore, creating a computer intelligent method with low error and high accuracy to predict survival is an important challenge, and it is beneficial for decreasing mortality from lung cancer, and for planning treatment. In this work, we implemented a deep stacked sparse auto-encoder (DSSAE) approach on a thoracic surgery data set for 470 patients, and our results contributing to deep learning based on 16 features were more precise than other suggested techniques for predicting post-operative survival expectancy in thoracic lung cancer surgery. The proposed method achieved a sensitivity of 94%, specificity of 82.86% and g-mean of 88.25%.

Electric Load Data Compression and Classification Based on Deep Stacked Auto-Encoders

With the development of advanced metering infrastructure (AMI), electrical data are collected frequently by smart meters. Consequently, the load data volume and length increase dramatically, which aggravates the data storage and transmission burdens in smart grids. On the other hand, for event detection or market-based demand response applications, load service entities (LSEs) want smart meter readings to be classified in specific and meaningful types. Considering these challenges, a stacked auto-encoder (SAE)-based load data mining approach is proposed. First, an innovative framework for smart meter data flow is established. On the user side, the SAEs are utilized to compress load data in a distributed way. Then, centralized classification is adopted at remote data center by softmax classifier. Through the layer-wise feature extracting of SAE, the sparse and lengthy raw data are expressed in compact forms and then classified based on features. A global fine-tuning strategy based on a well-defined labeled subset is embedded to improve the extracted features and the classification accuracy. Case studies in China and Ireland demonstrate that the proposed method is more capable to achieve the minimum of error and satisfactory compression ratios (CR) than benchmark compressors. It also significantly improves the classification accuracy on both appliance and house level datasets.