Stacked Convolutional Autoencoder Research Articles

Enhancing Taxonomic Categorization of DNA Sequences with Deep Learning: A Multi-Label Approach.

The application of deep learning for taxonomic categorization of DNA sequences is investigated in this study. Two deep learning architectures, namely the Stacked Convolutional Autoencoder (SCAE) with Multilabel Extreme Learning Machine (MLELM) and the Variational Convolutional Autoencoder (VCAE) with MLELM, have been proposed. These designs provide precise feature maps for individual and inter-label interactions within DNA sequences, capturing their spatial and temporal properties. The collected features are subsequently fed into MLELM networks, which yield soft classification scores and hard labels. The proposed algorithms underwent thorough training and testing on unsupervised data, whereby one or more labels were concurrently taken into account. The introduction of the clade label resulted in improved accuracy for both models compared to the class or genus labels, probably owing to the occurrence of large clusters of similar nucleotides inside a DNA strand. In all circumstances, the VCAE-MLELM model consistently outperformed the SCAE-MLELM model. The best accuracy attained by the VCAE-MLELM model when the clade and family labels were combined was 94%. However, accuracy ratings for single-label categorization using either approach were less than 65%. The approach's effectiveness is based on MLELM networks, which record connected patterns across classes for accurate label categorization. This study advances deep learning in biological taxonomy by emphasizing the significance of combining numerous labels for increased classification accuracy.

Noise-Tolerant Data Reconstruction Based on Convolutional Autoencoder for Wireless Sensor Network

Maintaining data dependability within wireless sensor network (WSN) systems has significant importance. Nevertheless, the deployment of systems in unattended and hostile areas poses a major challenge in dealing with noise. Consequently, several investigations have been conducted to address the issue of noise-affected data recovery. Nevertheless, previous research has primarily focused on the internal noise of the system. Neglecting to include external factors that impact the WSN system in the study might lead to findings that are not true to reality. Hence, this research takes into account both internal and external noise factors, such as rain, fog, or snow conditions. Moreover, in order to maintain the temporal characteristics and intersensor relationships, the data from multiple sensor nodes are consolidated into a two-dimensional matrix format. The stacked convolutional autoencoder (SCAE) model is proposed, which has the capability to extract data features. The stack design of the SCAE enables it to effectively mitigate the issue of vanishing gradients. Moreover, the weight sharing approach used between the two subnetworks also enhances the efficiency of the weight initialization procedure. Thorough experiments are conducted using both simulated WSN systems and real-world sensing data. Experimental results demonstrate that the SCAE outperforms existing methods for reconstructing noisy data.

Improving diversity and quality of adversarial examples in adversarial transformation network

This paper proposes PatternAttack to mitigate two major issues of Adversarial Transformation Network (ATN) including the low diversity and the low quality of adversarial examples. In order to deal with the first issue, this research proposes a stacked convolutional autoencoder based on patterns to generalize ATN. This proposed autoencoder could support different patterns such as all-pixel pattern, object boundary pattern, and class model map pattern. In order to deal with the second issue, this paper presents an algorithm to improve the quality of adversarial examples in terms of \(L_0\)-norm and \(L_2\)-norm. This algorithm employs adversarial pixel ranking heuristics such as JSMA and COI to prioritize adversarial pixels. To demonstrate the advantages of the proposed method, comprehensive experiments have been conducted on the MNIST dataset and the CIFAR-10 dataset. For the first issue, the proposed autoencoder generates diverse adversarial examples. For the second issue, the proposed algorithm significantly improves the quality of adversarial examples. In terms of \(L_0\)-norm, the proposed algorithm decreases from hundreds of adversarial pixels to one adversarial pixel. In terms of \(L_2\)-norm, the proposed algorithm reduces the average distance considerably. These results show that the proposed method can generate high-quality and diverse adversarial examples in practice.

A New SCAE-MT Classification Model for Hyperspectral Remote Sensing Images.

Hyperspectral remote sensing images (HRSI) have the characteristics of foreign objects with the same spectrum. As it is difficult to label samples manually, the hyperspectral remote sensing images are understood to be typical "small sample" datasets. Deep neural networks can effectively extract the deep features from the HRSI, but the classification accuracy mainly depends on the training label samples. Therefore, the stacked convolutional autoencoder network and transfer learning strategy are employed in order to design a new stacked convolutional autoencoder network model transfer (SCAE-MT) for the purposes of classifying the HRSI in this paper. In the proposed classification method, the stacked convolutional au-to-encoding network is employed in order to effectively extract the deep features from the HRSI. Then, the transfer learning strategy is applied to design a stacked convolutional autoencoder network model transfer under the small and limited training samples. The SCAE-MT model is used to propose a new HRSI classification method in order to solve the small samples of the HRSI. In this study, in order to prove the effectiveness of the proposed classification method, two HRSI datasets were chosen. In order to verify the effectiveness of the methods, the overall classification accuracy (OA) of the convolutional self-coding network classification method (CAE), the stack convolutional self-coding network classification method (SCAE), and the SCAE-MT method under 5%, 10%, and 15% training sets are calculated. When compared with the CAE and SCAE models in 5%, 10%, and 15% training datasets, the overall accuracy (OA) of the SCAE-MT method was improved by 2.71%, 3.33%, and 3.07% (on average), respectively. The SCAE-MT method is, thus, clearly superior to the other methods and also shows a good classification performance.

Multi-Label Extreme Learning Machine (MLELMs) for Bangla Regional Speech Recognition

Extensive research has been conducted in the past to determine age, gender, and words spoken in Bangla speech, but no work has been conducted to identify the regional language spoken by the speaker in Bangla speech. Hence, in this study, we create a dataset containing 30 h of Bangla speech of seven regional Bangla dialects with the goal of detecting synthesized Bangla speech and categorizing it. To categorize the regional language spoken by the speaker in the Bangla speech and determine its authenticity, the proposed model was created; a Stacked Convolutional Autoencoder (SCAE) and a Sequence of Multi-Label Extreme Learning machines (MLELM). SCAE creates a detailed feature map by identifying the spatial and temporal salient qualities from MFEC input data. The feature map is then sent to MLELM networks to generate soft labels and then hard labels. As aging generates physiological changes in the brain that alter the processing of aural information, the model took age class into account while generating dialect class labels, increasing classification accuracy from 85% to 95% without and with age class consideration, respectively. The classification accuracy for synthesized Bangla speech labels is 95%. The proposed methodology works well with English speaking audio sets as well.

Deep Non-Cooperative Spectrum Sensing Over Rayleigh Fading Channel

In this paper, we propose a robust non-cooperative spectrum sensing algorithm based on deep learning over Rayleigh fading channel. We conduct noise cancellation on the received sensing data using the stacked convolutional auto-encoder (SCAE) as a pre-processing step. The series of the denoised signal in the time domain is then fed into the proposed Hybrid CNN-SA-GRU (H-CSG) network. The proposed network combines convolutional neural network (CNN), self-attention (SA) modules and gate recurrent unit (GRU). It can extract input features from spatial and temporal domains. The proposed algorithm has been shown to be effective and robust in detecting weak signals at the low signal-to-noise ratio (SNR) level.

Medical image denoising system based on stacked convolutional autoencoder for enhancing 2-dimensional gel electrophoresis noise reduction

Image denoising is the technique of removing noise or distortions from an image. During medical image acquisition, random noise is added, which results in a lower contrast in those images. For that, image denoising is a crucial task for medical imaging analysis. In this study, a denoising system using three heterogeneous medical datasets is proposed based on stacked convolutional autoencoder (SCAE) technique. To validate its efficiency, different evaluation metrics are used, such as mean squared error (MSE), Peak signal-to-noise ratio (PSNR), contrast-to-noise ratio (CNR), structural similarity index measure (SSIM) and cross correlation (CC). The proposed denoising system gives good results among the medical and microscopic datasets that are used for training. The best average results obtained are 0.0039 for MSE, 24.07 for PSNR, 0.1220 for CNR, 0.85 for SSIM, and 0.6358 for CC. Then, the proposed SCAE denoising system was applied to the LECB 2-D PAGE database for denoising real 2-DGE images. The results of denoising 2-DGE images are evaluated by MSE, spot efficiency, false discovery rate (FDR), and signal-to-noise ratio (SNR). The best average results for 2-DGE images are 0.014 for MSE, 75 spot efficiency, 36.3 for FDR and 18.41 for SNR. The proposed system has enhanced the denoising of 2-DGE images by 0.9% to 17.6% when compared to other techniques.

Robust Feature Extraction for Geochemical Anomaly Recognition Using a Stacked Convolutional Denoising Autoencoder

Deep neural networks perform very well in learning high-level representations in support of multivariate geochemical anomaly recognition. Geochemical exploration data typically contain a proportion of large variations and missing values, which motivated us to construct a network architecture optimized to deal with these data. Our approach adopted a stacked convolutional denoising autoencoder (SCDAE) to extract robust features and decreased the level of sensitivity to partially corrupted data, that is, input data that are partially missing. SCDAE parameters, which include the network depth, number of convolution layers, number of convolution kernels, and convolution kernel size, were optimized using trial-and-error experiments. The optimal SCDAE architecture was then used to recognize multivariate geochemical anomalies related to mineralization in a case study in southwestern Fujian Province, based on the differences in the reconstruction errors between sample populations. The spatial distribution of high reconstruction errors in the anomaly map was closely related to most known Fe deposits, indicating the effectiveness of the SCDAE at recognizing geochemical anomalies related to Fe mineralization. A comparative study between the SCDAE and a stacked convolutional autoencoder (SCAE) with different corruption levels showed that the SCDAE exhibited reduced sensitivity to stochastic disturbances with different corruption proportions, and had an enhanced ability to recognize geochemical anomalies varying in a reasonable range. The robustness of the SCDAE makes it applicable to a wide variety of geochemical exploration scenarios, particularly in areas with incomplete or missing data.

Fault Diagnosis of a Rotor-Bearing System Under Variable Rotating Speeds Using Two-Stage Parameter Transfer and Infrared Thermal Images

Current fault diagnosis methods for rotor-bearing system are mostly based on analyzing the vibration signals collected at steady rotating speeds. In those methods, the data collected under one operating condition cannot be accurately used for diagnosis under a different condition. Moreover, in vibration monitoring, installing the necessary sensors will affect the equipment structure and hence the vibration response itself. The present paper proposes a new method based on two-stage parameter transfer and infrared thermal images for fault diagnosis of rotor-bearing system under variable rotating speeds. The method of parameter transfer enables the use of data (or parameters) acquired under one operating condition (called the source domain) to be extended for use in a different operating condition (called the target domain). First, scaled exponential linear unit (SELU) and modified stochastic gradient descent (MSGD) are used to construct an enhanced convolutional neural network (ECNN). Second, a stacked convolutional auto-encoder (CAE) trained based on unlabeled source-domain thermal images is employed to initialize a source-domain ECNN. Third, model parameters from the pre-trained source-domain ECNN are transferred to the target-domain ECNN to adapt to the characteristics of the target domain. The collected thermal images for a rotor-bearing system under variable speeds are used to test the transfer diagnosis performance of the proposed method. The experimental results demonstrate the performance improvement and the advantages of the proposed method.

Gait Recognition Method of Underground Coal Mine Personnel Based on Densely Connected Convolution Network and Stacked Convolutional Autoencoder

Biological recognition methods often use biological characteristics such as the human face, iris, fingerprint, and palm print; however, such images often become blurred under the limitation of the complex environment of the underground, which leads to low identification rates of underground coal mine personnel. A gait recognition method via similarity learning named Two-Stream neural network (TS-Net) is proposed based on a densely connected convolution network (DenseNet) and stacked convolutional autoencoder (SCAE). The mainstream network based on DenseNet is mainly used to learn the similarity of dynamic deep features containing spatiotemporal information in the gait pattern. The auxiliary stream network based on SCAE is used to learn the similarity of static invariant features containing physiological information. Moreover, a novel feature fusion method is adopted to achieve the fusion and representation of dynamic and static features. The extracted features are robust to angle, clothing, miner hats, waterproof shoes, and carrying conditions. The method was evaluated on the challenging CASIA-B gait dataset and the collected gait dataset of underground coal mine personnel (UCMP-GAIT). Experimental results show that the method is effective and feasible for the gait recognition of underground coal mine personnel. Besides, compared with other gait recognition methods, the recognition accuracy has been significantly improved.

BIRADS features-oriented semi-supervised deep learning for breast ultrasound computer-aided diagnosis

We propose a novel BIRADS-SSDL network that integrates clinically-approved breast lesion characteristics (BIRADS features) into task-oriented semi-supervised deep learning (SSDL) for accurate diagnosis of ultrasound (US) images with a small training dataset. Breast US images are converted to BIRADS-oriented feature maps (BFMs) using a distance-transformation coupled with a Gaussian filter. Then, the converted BFMs are used as the input of an SSDL network, which performs unsupervised stacked convolutional auto-encoder (SCAE) image reconstruction guided by lesion classification. This integrated multi-task learning allows SCAE to extract image features with the constraints from the lesion classification task, while the lesion classification is achieved by utilizing the SCAE encoder features with a convolutional network. We trained the BIRADS-SSDL network with an alternative learning strategy by balancing the reconstruction error and classification label prediction error. To demonstrate the effectiveness of our approach, we evaluated it using two breast US image datasets. We compared the performance of the BIRADS-SSDL network with conventional SCAE and SSDL methods that use the original images as inputs, as well as with an SCAE that use BFMs as inputs. The experimental results on two breast US datasets show that BIRADS-SSDL ranked the best among the four networks, with a classification accuracy of around 94.23 ± 3.33% and 84.38 ± 3.11% on two datasets. In the case of experiments across two datasets collected from two different institutions/and US devices, the developed BIRADS-SSDL is generalizable across the different US devices and institutions without overfitting to a single dataset and achieved satisfactory results. Furthermore, we investigate the performance of the proposed method by varying the model training strategies, lesion boundary accuracy, and Gaussian filter parameters. The experimental results showed that a pre-training strategy can help to speed up model convergence during training but with no improvement of the classification accuracy on the testing dataset. The classification accuracy decreases as the segmentation accuracy decreases. The proposed BIRADS-SSDL achieves the best results among the compared methods in each case and has the capacity to deal with multiple different datasets under one model. Compared with state-of-the-art methods, BIRADS-SSDL could be promising for effective breast US computer-aided diagnosis using small datasets.

Human Behavior Recognition from Multiview Videos

With the proliferation of deep learning techniques, a significant number of applications related to home care systems have emerged recently. In particular, detecting abnormal events in a smart home environment has been extensively studied. In this paper, we adopt deep learning techniques, including convolutional neural networks (CNNs) and long short-term memory (LSTM) networks, to construct deep networks to learn the long-term dependencies from videos for human behavior recognition in a multiview framework. We adopt two cameras as our sensors to efficiently overcome the problem of occlusions and contour ambiguity for improving the accuracy performance of the multiview framework. After performing a series of image preprocessing on the raw data, we obtain human silhouette images as the input to our training model. In addition, because real-world datasets are complicated for analysis, labeling data is time consuming. Therefore, we present an image clustering method based on a stacked convolutional autoencoder (SCAE), which generates clustering labels for autolabeling. Finally, we set up our experimental environment as a normal residence to collect a large dataset, and the experimental results demonstrate the novelty of our proposed models.

A novel stacked sparse denoising autoencoder for mammography restoration to visual interpretation of breast lesion

This paper proposes a deep unsupervised learning based denoising autoencoder model for the restoration of degraded mammogram with visual interpretation of breast lumps or lesion in mammography images (called SSDAE). The proposed model attempts to intensify the underexposed and abnormal structural regions through noise elimination in mammography image. A deep stacked convolutional autoencoder is designed by combining the autoencoder and the deconvolution network which conjointly reduces noisy artifacts and improves image details in mammogram. The proposed SSDAE model takes large noisy mammogram image patches as input and extracts relevant features from target batches. The suggested model can extract relevant features and reduce the dimensionality through sparsity property of the image data while preserving the key features that have been applied to restore image data in feature space. In order to reconstruct a deafening mammogram, the proposed model is carried out through a patched base training on samples to suppress noise thereby preserving structural details in mammography imaging. Experimental results authenticate that the suggested SSDAE model outplays a number of state-of-the-art methods for both X-ray mammogram and ultrasonographic mammogram. The execution speed for target noisy images increases with fine tuning of the network when compared to other algorithms.

Land Cover Change Detection With VHR Satellite Imagery Based on Multi-Scale SLIC-CNN and SCAE Features

Change detection with very high resolution (VHR) satellite images is of great application values when evaluating and monitoring land use changes. However, intrinsic complexity of satellite images will introduce more difficulties to change detection tasks. In this study, a new change detection method is proposed by combining multi-scale simple linear iterative clustering-convolutional neural network (SLIC-CNN) with stacked convolutional auto-encoder (SCAE) features to improve change detection capabilities with VHR satellite images. First, the multi-scale SLIC-based image segmentation is performed on multi-temporal images to generate segment objects while keeping their edge information as much as possible. Second, the convolutional layers in a CNN architecture are used to generate change map, then, an SCAE feature-based classification procedure is performed to generate “from-to” change information. Finally, a Bayesian information criterion is used to optimize the results of change detection. In this study, the experiments carried out reveal that the multi-scale SLIC image segmentation algorithm affects the integrity of change regions; CNN features have an effect on the consistency of change regions; and SCAE features influence the performance of support vector machine (SVM) classifiers. And, features extracted from the architectures enhance the ability of information extraction from ground objects. Comparison results also show the superiority to other change detection methods.

Artifact removal from neurophysiological signals: impact on intracranial and arterial pressure monitoring in traumatic brain injury.

Monitoring intracranial and arterial blood pressure (ICP and ABP, respectively) provides crucial information regarding the neurological status of patients with traumatic brain injury (TBI). However, these signals are often heavily affected by artifacts, which may significantly reduce the reliability of the clinical determinations derived from the signals. The goal of this work was to eliminate signal artifacts from continuous ICP and ABP monitoring via deep learning techniques and to assess the changes in the prognostic capacities of clinical parameters after artifact elimination. The first 24 hours of monitoring ICP and ABP in a total of 309 patients with TBI was retrospectively analyzed. An artifact elimination model for ICP and ABP was constructed via a stacked convolutional autoencoder (SCAE) and convolutional neural network (CNN) with 10-fold cross-validation tests. The prevalence and prognostic capacity of ICP- and ABP-related clinical events were compared before and after artifact elimination. The proposed SCAE-CNN model exhibited reliable accuracy in eliminating ABP and ICP artifacts (net prediction rates of 97% and 94%, respectively). The prevalence of ICP- and ABP-related clinical events (i.e., systemic hypotension, intracranial hypertension, cerebral hypoperfusion, and poor cerebrovascular reactivity) all decreased significantly after artifact removal. The SCAE-CNN model can be reliably used to eliminate artifacts, which significantly improves the reliability and efficacy of ICP- and ABP-derived clinical parameters for prognostic determinations after TBI.

Unsupervised tumor detection in Dynamic PET/CT imaging of the prostate.

Early detection and localization of prostate tumors pose a challenge to the medical community. Several imaging techniques, including PET, have shown some success. But no robust and accurate solution has yet been reached. This work aims to detect prostate cancer foci in Dynamic PET images using an unsupervised learning approach. The proposed method extracts three feature classes from 4D imaging data that include statistical, kinetic biological and deep features that are learned by a deep stacked convolutional autoencoder. Anomalies, which are classified as tumors, are detected in feature space using density estimation. The proposed algorithm generates promising results for sufficiently large cancer foci in real PET scans imaging where the foci is not viewed by the tomographic devices used for detection.

CLASSIFICATION OF THE STRUCTURE OF CITIES THROUGH MID-RESOLUTION SATELLITE IMAGERY AND PATCH BASED NEURAL NETWORKS

Abstract. The studies in the classification of the urban spatial structure have been essential in deriving insights into the land cover and the built typology which helped in the estimation of energy consumption patterns, urban density, compactness, and hierarchy of settlements. However, the analysis and comparison of the physical forms of the cities have been attempted in a piecemeal fashion where the requirement of datasets and the computation power for analysis has been a major hindrance. With the advancement in machine learning based techniques, large datasets such as satellite imagery can be studied with advanced computer vision methods. These solutions may help in studying the intricate nature of human habitats in large extents of geographical areas including various urban areas. This study utilizes smaller patches of medium resolution Sentinel-2B Imagery of ten different cities in India to explore the urban forms present in these cities. This study uses Stacked Convolutional Autoencoder (CAE) to reduce the dimensionality of satellite imagery patches and unsupervised clustering techniques such as t-SNE and K-means to study the characteristics of similar patches. On analyzing the clusters through visual exploration, similar patches are delineated and provided with corresponding labels representing urban forms. Individual clusters are then studied with respect to each city. The motive of the study is to gain insights into the different types of morphological patterns present within and among cities.

Deep semantic preserving hashing for large scale image retrieval

Hashing approaches have got a great attention because of its efficient performance for large-scale images. This paper, aims to propose a deep hashing method which can combines stacked convolutional autoencoder with hashing learning, where the input image hierarchically maps to the low dimensional space. The proposed method DCAH contains encoder-decoder, and supervisory sub-network, that generates a low dimensional binary code in a layer-wised manner of the deep conventional neural network. To optimizing the hash algorithm, we added some extra relaxations constraint to the objective function. In our extensive experiments on ultra-high dimensional image datasets, our results demonstrate that the decoder structure can improve the hashing method to preserve the similarities in hashing codes; also, DCAH achieves the best performance comparing to other states of the art approaches.

Reconstruction of undersampled radial free-breathing 3D abdominal MRI using stacked convolutional auto-encoders.

Free-breathing three-dimensional (3D) abdominal imaging is a challenging task for MRI, as respiratory motion severely degrades image quality. One of the most promising self-navigation techniques is the 3D golden-angle radial stack-of-stars (SOS) sequence, which has advantages in terms of speed, resolution, and allowing free breathing. However, streaking artifacts are still clearly observed in reconstructed images when undersampling is applied. This work presents a novel reconstruction approach based on a stacked convolutional auto-encoder (SCAE) network to solve this problem. Thirty healthy volunteers participated in our experiment. To build the dataset, reference and artifact-affected images were reconstructed using 451 golden-angle spokes and the first 20, 40, or 90 golden-angle spokes corresponding to acceleration rates of 31.4, 15.7, and 6.98, respectively. In the training step, we trained the SCAE by feeding it with patches from artifact-affected images. The SCAE outputs patches in the corresponding reference images. In the testing step, we applied the trained SCAE to map each input artifact-affected patch to the corresponding reference image patch. The SCAE-based reconstruction images with acceleration rates of 6.98 and 15.7 show nearly similar quality as the reference images. Additionally, the calculation time is below 1 s. Moreover, the proposed approach preserves important features, such as lesions not presented in the training set. The preliminary results demonstrate the feasibility of the proposed SCAE-based strategy for correcting the streaking artifacts of undersampled free-breathing 3D abdominal MRI with a negligible reconstruction time.

Automatic feature extraction of time-series applied to fault severity assessment of helical gearbox in stationary and non-stationary speed operation

Signals captured in rotating machines to obtain the status of their components can be considered as a source of massive information. In current methods based on artificial intelligence to fault severity assessment, features are first generated by advanced signal processing techniques. Then feature selection takes place, often requiring human expertise. This approach, besides time-consuming, is highly dependent on the machinery configuration as in general the results obtained for a mechanical system cannot be reused by other systems. Moreover, the information about time events is often lost along the process, preventing the discovery of faulty state patterns in machines operating under time-varying conditions. In this paper a novel method for automatic feature extraction and estimation of fault severity is proposed to overcome the drawbacks of classical techniques. The proposed method employs a Deep Convolutional Neural Network pre-trained by a Stacked Convolutional Autoencoder. The robustness and accuracy of this new method are validated using a dataset with different severity conditions on failure mode in a helical gearbox, working in both constant and variable speed of operation. The results show that the proposed unsupervised feature extraction method is effective for the estimation of fault severity in helical gearbox, and it has a consistently better performance in comparison with other reported feature extraction methods.