Deep Sparse Autoencoder Research Articles

A Hybrid Detection System for DDoS Attacks Based on Deep Sparse Autoencoder and Light Gradient Boost Machine

In the internet era, network-based services and connected devices are growing with many users, thus it became an increase in the number of cyberattacks. Distributed Denial of Service (DDoS) attacks are the type of cyberattacks increasing their strength and impact on the victim. Effective detection of such attacks through a DDoS Detection System is relatively essential research. Although machine learning techniques have grown in popularity in the field of cybersecurity over the last several years, the change in the attack patterns in recent days shows the need for developing a robust DDoS prediction model. Therefore, we suggested a DDoS prediction system using a two-stage hybrid methodology. Initially, features are extracted by the unsupervised Deep Sparse Autoencoder (DSAE) using Elastic Net regularisation with optimum hyperparameters. Further, several learning models are tuned to classify attacks based on the extracted feature sets. Finally, the models’ performance is analysed with extracted features in balanced and imbalanced data scenarios. The experimental outcomes show that the suggested model outperforms current approaches. The model was evaluated on the CICIDS-2017 and CICDDoS-2019 datasets and achieved an accuracy of 99.98% and 99.99%, respectively.

Deep fake detection using cascaded deep sparse auto-encoder for effective feature selection.

In the recent research era, artificial intelligence techniques have been used for computer vision, big data analysis, and detection systems. The development of these advanced technologies has also increased security and privacy issues. One kind of this issue is Deepfakes which is the combined word of deep learning and fake. DeepFake refers to the formation of a fake image or video using artificial intelligence approaches which are created for political abuse, fake data transfer, and pornography. This paper has developed a Deepfake detection method by examining the computer vision features of the digital content. The computer vision features based on the frame change are extracted using a proposed deep learning model called the Cascaded Deep Sparse Auto Encoder (CDSAE) trained by temporal CNN. The detection process is performed using a Deep Neural Network (DNN) to classify the deep fake image/video from the real image/video. The proposed model is implemented using Face2Face, FaceSwap, and DFDC datasets which have secured an improved detection rate when compared to the traditional deep fake detection approaches.

Deep sparse autoencoder integrated with three‐stage framework for glaucoma diagnosis

Recently, end-to-end deep neural networks-based glaucoma diagnosis approaches have been gaining much attention. However, the feature extractor and classier in these approaches are trained together, which is known as coadaptation. Therefore, the feature distribution in them should adapt to particular decision boundaries. To learn generic data representations and improve the generalization ability of the model, this paper designs a three-stage framework for glaucoma diagnosis. In the first stage, preprocessing is utilized to extract the Region of Interesting around the Optic Disc to reduce the computational cost and nonobjective interference. In the second stage, Deep Sparse Autoencoder is designed to learn hybrid features between the deep features and the original features, which could improve the effectiveness of final high-level feature expression. Meanwhile, L1 regularization is introduced and applied on the hybrid features to obtain deep features with high complementarity under small sample problem. In the third stage, the obtained generic feature representations are fed into different classifiers, in which Support Vector Machine classifier achieves the best diagnosis performance. The proposed approach is evaluated on two publicly available databases. Extensive experimental results indicate that our approach outperforms the state-of-the-art approaches with the accuracy of 96.00%, 97.00% and Area Under Curve of 96.94%, 98.28% for REFUGE and Drishti-GS1 databases, respectively.

Representation learning with deep sparse auto-encoder for multi-task learning

We demonstrate an effective framework to achieve a better performance based on Deep Sparse auto-encoder for Multi-task Learning, called DSML for short. To learn the reconstructed and higher-level features on cross-domain instances for multiple tasks, we combine the labeled and unlabeled data from all tasks to reconstruct the feature representations. Furthermore, we propose the model of Stacked Reconstruction Independence Component Analysis (SRICA for short) for the optimization of feature representations with a large amount of unlabeled data, which can effectively address the redundancy of image data. Our proposed SRICA model is developed from RICA and is based on deep sparse auto-encoder. In addition, we adopt a Semi-Supervised Learning method (SSL for short) based on model parameter regularization to build a unified model for multi-task learning. There are several advantages in our proposed framework as follows: 1) The proposed SRICA makes full use of a large amount of unlabeled data from all tasks. It is used to pursue an optimal sparsity feature representation, which can overcome the over-fitting problem effectively. 2) The deep architecture used in our SRICA model is applied for higher-level and better representation learning, which is designed to train on patches for sphering the input data. 3) Training parameters in our proposed framework has lower computational cost compared to other common deep learning methods such as stacked denoising auto-encoders. Extensive experiments on several real image datasets demonstrate our proposed framework outperforms the state-of-the-art methods.

Seismic Horizon Estimation Based on Deep Learning Technique

The seismic horizons estimation from the seismic data is essential for structural and stratigraphic modeling of reservoirs. Until now, manual interpretation and semi-automated techniques were used to estimate the seismic horizon from the seismic data. However, the seismic horizon estimation takes more time and needs a human expert for analysis. To overcome those limitations, we propose a novel method to estimate the seismic horizon using a deep sparse convolutional autoencoder (DSCA). The convolutional layers in the DSCA network extract the complex features from the seismic data which improves the accuracy of seismic horizon estimation. The DSCA network is trained in a supervised way with the labeled seismic data. The performance of the proposed method is tested and compared with an existing method based on quantitative metrics such as Mean Squared Error (MSE), Coefficient of Determination (r2), and Pearson Correlation Coefficient (PCC). Experimental results prove that the proposed method has shown better results compared to the existing method.

Fault diagnosis of planetary gear backlash based on motor current and Fisher criterion optimized sparse autoencoder

Planetary gear reducer is widely applied in various transmission equipment, and its performance highly affects the operation of a machine. The appearance of unreasonable backlash in planetary gear reducer may lead to undesirable vibration, which may accelerate the degradation of equipment and eventually cause premature failure. In traditional condition-based monitoring (CBM), sensors such as accelerometers have been utilized to detect the fault of planetary gear. However, the complexity and integration of planetary gear limit the installation and application of vibration sensors in practice. In this case, the current signal from motor, as a convenient real-time monitoring approach, is introduced into the CBM of the planetary gear. In this paper, a fault diagnosis method based on drive motor current signal analysis (MCSA) is presented to identify the backlash fault in a planetary gear reducer. In this method, frequency domain data of the original current signal is found and used to automatically extract fault features. A deep sparse autoencoder (DSAE) extracts the required features. In particular, the Fisher criterion is introduced to evaluate the sensitivity of these features. It then selects the most effective ones for improving diagnostic accuracy as well as diagnostic efficiency. Experimental test data shows that under different load conditions, this method outperforms other typical fault diagnosis methods and exhibits the best performance.

A reliable deep representation learning to improve trust-aware recommendation systems

Deep neural networks have been extensively employed in many applications such as natural language processing and computer vision. They have attracted a lot of attention in designing recommender systems due to their recent significant achievements. These models are mainly used in recommender systems to generate the users’ latent features according to the input data such as user-item rating and trust matrixes. However, such input data are significantly sparse leading to reduce the effectiveness of deep learning models in generating the latent features. In addition, the existing deep learning based recommendation models ignore the reliability measure in their recommendation process. To address these challenges, in this paper, a trust-aware recommendation method based on deep sparse autoencoder is proposed. Specifically, an effective probabilistic model is proposed to determine how many ratings are required for each user to produce an accurate prediction. Moreover, for enhancing the rating profiles of users with insufficient ratings, an efficient mechanism is developed based on the utilization of implicit ratings. In this mechanism, the implicit ratings are selected based on a reliability measure to guarantee their effectiveness. Then, the reliable enhanced rating profiles and trust statements are considered as the input data of deep sparse autoencoder to generate the users’ latent features. Finally, a similarity function that utilizes the generated latent features is developed to obtain similarity values between users and make recommendations. Massive experiments are carried out on two datasets to assess the effectiveness of the proposed method in comparison to the-state-of-the-art models. The results of experiments demonstrate that the proposed method can significantly beat other models in producing more accurate recommendations.

Evolutional Neural Architecture Search for Optimization of Spatiotemporal Brain Network Decomposition.

Using deep neural networks (DNNs) to explore spatial patterns and temporal dynamics of human brain activities has been an important yet challenging problem because the artificial neural networks are hard to be designed manually. There have been several promising deep learning methods, e.g., deep belief network (DBN), convolutional neural network (CNN), and deep sparse recurrent auto-encoder (DSRAE), that can decompose neuroscientific and meaningful spatiotemporal patterns from 4D functional Magnetic Resonance Imaging (fMRI) data. However, those previous studies still depend on hand-crafted neural network architectures and hyperparameters, which are not optimal in various senses. In this paper, we employ the evolutionary algorithms (EA) to optimize the deep neural architecture of DSRAE by minimizing the expected loss of initialized models, named eNAS-DSRAE (evolutionary Neural Architecture Search on Deep Sparse Recurrent Auto-Encoder). Also, validation experiments are designed and performed on the publicly available human connectome project (HCP) 900 datasets, and the results achieved by the optimized eNAS-DSRAE suggested that our framework can successfully identify the spatiotemporal features and perform better than the hand-crafted neural network models. To our best knowledge, the proposed eNAS-DSRAE is not only among the earliest NAS models that can extract connectome-scale meaningful spatiotemporal brain networks from 4D fMRI data, but also is an effective framework to optimize the RNN-based models.

Alz-SAENet: A Deep Sparse Autoencoder based Model for Alzheimer’s Classification

Precise identification of Alzheimer's Disease (AD) is vital in health care, especially at an early stage, since recognizing the likelihood of incidence and progression allows patients to adopt preventive measures before irreparable brain damage occurs. Magnetic Resonance Imaging is an effective and common clinical strategy to diagnose AD due to its structural details. we built an advanced deep sparse autoencoder-based architecture, named Alz-SAENet for the identification of diseased from typical control subjects using MRI volumes. We focused on a novel optimal feature extraction procedure using the combination of a 3D Convolutional Neural Network (CNN) and deep sparse autoencoder (SAE). Optimal features derived from the bottleneck layer of the hyper-tuned SAE network are subsequently passed via a deep neural network (DNN). This approach results in the improved four-way categorization of AD-prone 3D MRI brain images that prove the capability of this network in AD prognosis to adopt preventive measures. This model is further evaluated using ADNI and Kaggle data and achieved 98.9% and 98.215% accuracy and showed a tremendous response in distinguishing the MRI volumes that are in a transitional phase of AD.

Fog-inspired water resource analysis in urban areas from satellite images

In the context of climate change, the extraction of accurate information on natural resources becomes necessary and is considered one of the most challenging tasks in the field of remote sensing. The identification of water resources has achieved considerable attention in the field of remote sensing to deal with the problem of water scarcity. In the proposed study, a novel Multi-layered Data Integration Technique (MDIT) is proposed for the identification of water resources from satellite imagery. To evaluate the patterns, Deep Convolutional Restrictive Model (DCRM) is proposed to extract deep hierarchical features from the satellite images. Furthermore, the DCRM model is calculating the relationship between the features to evaluate the meaningful patterns. Moreover, Spatial Inferred Features (SIF) and Deep Sparse Auto-encoder (DSA) modules are utilized in MDIT to improve the inferences between the spatial features and to calculate the non-direct relationship between the extracted features. To evaluate the performance, the prediction efficiency of the proposed solution is compared with different state-of-the-art conventional and deep learning approaches such as Normalized Difference Water Index (NDWI), Residual Neural Network (ResNet), Visual Geometry Group (VGG), DeepLab V3, Densely Connected Convolutional Network (DenseNet), and Semantic Segmentation Network (SegNet). The proposed solution outperformed all the state-of-the-art approaches by achieving a higher precision of 0.945% for the extraction of water resources from low-resolution satellite imagery.

An Optimal NIDS for VCN Using Feature Selection and Deep Learning Technique

In this modern era, due to demand for cloud environments in business, the size, complexity, and chance of attacks to virtual cloud network (VCN) are increased. The protection of VCN is required to maintain the faith of the cloud users. Intrusion detection is essential to secure any network. The existing approaches that use the conventional neural network cannot utilize all information for identifying the intrusions. In this paper, the anomaly-based NIDS for VCN is proposed. For feature selection, grey wolf optimization (GWO) is hybridized with a bald eagle search (BES) algorithm. For classification, a deep learning approach - deep sparse auto-encoder (DSAE) is employed. In this way, this paper proposes a NIDS model for VCN named - GWO-DES-DSAE. The proposed system is simulated in the python programming environment. The proposed NIDS model's performance is compared with other recent approaches for both binary and multi-class classification on the considered datasets - NSL-KDD, UNSW-NB15, and CICIDS 2017 and found better than other methods.

Community Detection Based on Graph Representation Learning in Evolutionary Networks

Aiming at analyzing the temporal structures in evolutionary networks, we propose a community detection algorithm based on graph representation learning. The proposed algorithm employs a Laplacian matrix to obtain the node relationship information of the directly connected edges of the network structure at the previous time slice, the deep sparse autoencoder learns to represent the network structure under the current time slice, and the K-means clustering algorithm is used to partition the low-dimensional feature matrix of the network structure under the current time slice into communities. Experiments on three real datasets show that the proposed algorithm outperformed the baselines regarding effectiveness and feasibility.

Deep sparse autoencoder prediction model based on adversarial learning for cross-domain recommendations

Online recommender systems generally suffer from severe data sparsity problems, and this are particularly prevalent in newly launched systems that do not have sufficient amounts of data. Cross-domain recommendations can provide us with some new ideas for assisting with user recommendations in sparse target domains by transferring knowledge from a source domain with rich data. In this paper, a deep sparse autoencoder prediction model based on adversarial learning for cross-domain recommendations (DSAP-AL) is proposed to improve the accuracy of rating predictions in similar cross-domain recommender systems. Specifically, joint matrix factorization and adversarial network learning models are adopted to integrate and align user and item latent factor spaces in a unified pattern. Then, a deep sparse autoencoder is represented and modeled by transferring the latent factors and interlayer weights. Furthermore, a domain factor adaptation algorithm is proposed to capture robust user and item factors, and the learned regularization constraints are added to the objective function, thereby alleviating the data sparsity issue. Experimental results on four real-world datasets demonstrate that, even without overlapping entities (users or items) in the source and target domains, the proposed DSAP-AL method achieves competitive performance relative to other state-of-the-art individual and cross domain approaches. Moreover, the DSAP-AL model is not only effective for scenarios with sparse data but also robust for noise-containing recommendations.

History Matching of Naturally Fractured Reservoirs Using a Deep Sparse Autoencoder

SummaryAlthough researchers have applied many methods to history matching, such as Monte Carlo methods, ensemble-based methods, and optimization algorithms, history matching fractured reservoirs is still challenging. The key challenges are effectively representing the fracture network and coping with large amounts of reservoir-model parameters. With increasing numbers of fractures, the dimension becomes larger, resulting in heavy computational work in the inversion of fractures. This paper proposes a new characterization method for the multiscale fracture network, and a powerful dimensionality-reduction method by means of an autoencoder for model parameters. The characterization method of the fracture network is dependent on the length, orientation, and position of fractures, including large-scale and small-scale fractures. To significantly reduce the dimension of parameters, the deep sparse autoencoder (DSAE) transforms the input to the low-dimensional latent variables through encoding and decoding. Integrated with the greedy layer-wise algorithm, we set up a DSAE and then take the latent variables as optimization variables. The performance of the DSAE with fewer activating nodes is excellent because it reduces the redundant information of the input and avoids overfitting. Then, we adopt the ensemble smoother (ES) with multiple data assimilation (ES-MDA) to solve this minimization problem. We test our proposed method in three synthetic reservoir history-matching problems, compared with the no-dimensionality-reduction method and the principal-component analysis (PCA). The numerical results show that the characterization method integrated with the DSAE could simplify the fracture network, preserve the distribution of fractures during the update, and improve the quality of history matching naturally fractured reservoirs.

A weighted network community detection algorithm based on deep learning

At present, community detection methods are mostly focused on the investigation at unweighted networks. However, real-world networks are always complex, and unweighted networks are not sufficient to reflect the connections among real-world objects. Hence, this paper proposes a community detection algorithm based on a deep sparse autoencoder. First, the second-order neighbors of the nodes are identified, and we can obtain the path weight matrix for the second-order neighbors of the node. We combine the path weight matrix with the weighted adjacent paths of the node to obtain the similarity matrix, which can not only reflect the similarity relationships among connected nodes in the network topology but also the similarity relationships among nodes and second-order neighbors. Then, based on the unsupervised deep learning method, the feature matrix which has a stronger ability to express the features of the network can be obtained by constructing a deep sparse autoencoder. Finally, the K-means algorithm is adopted to cluster the low-dimensional feature matrix and obtain the community structure. The experimental results indicate that compared with 4 typical community detection algorithms, the algorithm proposed here can more accurately identify community structures. Additionally, the results of parameter experiments show that compared with the community structure found by the K-means algorithm, which directly uses the high-dimensional adjacency matrix, the community structure detected by the WCD algorithm in this paper is more accurate.

Bearing Fault Diagnosis Using Deep Sparse Autoencoder

Rolling element bearing is an important component in various machinery. Faulty on bearing cause severe equipment damage that lead to high maintenance cost. The development of deep learning has been paid a considerable amount of attention to fault diagnosis on rolling element bearing. Traditional machine learning such as Artificial Neural Network and Support Vector Machine have problems of lacking expression capacity, existing the curse of dimensionality, require manual feature extraction and require an additional feature selection. Deep learning model has the ability to effectively mine the high dimensional features and accurately recognize the health condition. In consequence, deep learning model has turned into an innovative and promising research in bearing fault diagnosis field. Thus, this paper tends to proposed Deep Sparse Autoencoder (DSAE) with Teager Kaiser Energy Operator (TKEO) to diagnose the bearing condition. DSAE is one of deep learning model which uses the architecture of neural network. During the analysis, the hyperparameter of DSAE model was optimized by Ant Lion Optimization. The analysis results show that the proposed TKEO-DSAE achieved 99.5% accuracy of the fault diagnosis. The comparative study between proposed model and ANN proved that deep learning model outperform traditional machine learning model on bearing fault diagnosis.

Deep Sparse Auto-Encoder Features Learning for Arabic Text Recognition

One of the most recent challenging issues of pattern recognition and artificial intelligence is Arabic text recognition. This research topic is still a pervasive and unaddressed research field, because of several factors. Complications arise due to the cursive nature of the Arabic writing, character similarities, unlimited vocabulary, use of multi-size and mixed-fonts, etc. To handle these challenges, an automatic Arabic text recognition requires building a robust system by computing discriminative features and applying a rigorous classifier together to achieve an improved performance. In this work, we introduce a new deep learning based system that recognizes Arabic text contained in images. We propose a novel hybrid network, combining a Bag-of-Feature (BoF) framework for feature extraction based on a deep Sparse Auto-Encoder (SAE), and Hidden Markov Models (HMMs), for sequence recognition. Our proposed system, termed BoF-deep SAE-HMM, is tested on four datasets, namely the printed Arabic line images Printed KHATT (P-KHATT), the benchmark printed word images Arabic Printed Text Image (APTI), the benchmark handwritten Arabic word images IFN/ENIT, and the benchmark handwritten digits images Modified National Institute of Standards and Technology (MNIST).

Hybrid Feature Embedded Sparse Stacked Autoencoder and Manifold Dimensionality Reduction Ensemble for Mental Health Speech Recognition

Speech feature learning is the key to speech mental health recognition. Deep feature learning can automatically extract the speech features but suffers from the small sample problem. The traditional feature extract method is effective, but cannot find the inter-feature structure to generate the new high-quality features. This paper proposes an embedded hybrid feature deep sparse stacked autoencoder ensemble method to solve this problem. Firstly, the speech features are extracted based on prior knowledge and called original features. Secondly, the original features are embedded into the deep network (Sparse Stacked Autoencoder) to filter the output of the hidden layer, to enhance the complementarity between the deep features and the original features. Thirdly, the L1 regularized feature selection mechanism is designed to reduce the hybrid feature set formed by the combination of deep features and original features. Finally, a manifold projection classifier ensemble is designed to enhance the stability of classification. Besides, this paper for the first time proposes a speech collection scheme for mental health recognition. We construct a large-scale Chinese mental health speech database for verification of the proposed algorithm of mental health. In the experimental section, the proposed algorithm is verified and compared with the representative related algorithms. The experimental results show that the proposed algorithm has better classification accuracy than the other representative algorithms. The proposed method combines the advantages of deep feature learning and traditional feature extraction methods more efficiently to solve the small sample problem.

Diagnosis of Melanoma Based on the Sparse Auto-Encoder for Feature Extraction

Aims: Skin cancer is a fairly critical disease all over the world and especially in Western countries and America. However, if it is perceived and treated early, it is quite often curable. The main risk factors for melanoma are exposure to UV rays, the presence of many moles, and heredity. For this reason, this work focuses on the issue of automatic diagnosis of melanoma. The aim is to extract significant features from pixels of the images based on an unsupervised deep learning technique which is the sparse autoencoder method.
 Methodology: A preprocessing phase is required to remove the artifacts and enhance the contrast of the images before proceeding with the feature extraction. Once the characteristics are extracted automatically, the support vector machine classifier and the k-nearest neighbors are applied for the classification phase. The objective is to differentiate between 3 categories: melanoma, suspected case, and non-melanoma. Finally, the PH2 database is used to test the proposed approaches (200 images are presented in this dataset: 80 atypical nevi, 80 common nevi, and 40 melanoma).
 Results: The obtained results in terms of specificity, accuracy, and sensitivity present noticeable performances with the support vector machine classifier (achieved 94 % overall accuracy) and the k-nearest neighbors (92 %).
 Conclusion: This study's experimental findings showed that the best performance was obtained by the approach based on a deep sparse autoencoder combined with support vector machine.

An Optimal Intrusion Detection System using GWO-CSA-DSAE Model

ABSTRACT In the high demand of the cloud computing environment, intrusion detection in a cloud network playing a big role in maintaining the faith of the client(s). Due to the increasing complexity of the cloud environment, the existing approaches which use the conventional neural networks are not able to utilise the relevant information from the network traffic, which leads to a low detection rate. This reduces the stability of the existing approaches in this changing environment. In this paper, an anomaly-based cloud intrusion detection system (IDS) is proposed for finding the intrusions in a cloud network. The proposed system uses a hybrid metaheuristic algorithm for feature selection and a deep learning approach for classification. For feature selection, grey wolf optimisation (GWO) is hybrid with a crow search algorithm (CSA), which extracts relevant features from the cloud network connection to be processed more effectively in the deep learning classifier section. A deep sparse auto-encoder (DSAE) is employed for the classification purpose. For the performance comparison, the considered metrics are accuracy, precision, recall or detection rate (DR), and F1 Score. Three publically well-known available datasets namely NSL-KDD, UNSW-NB15, and CICIDS 2017 have been considered for analysing the performance of the proposed GWO-CSA-DSAE model for intrusion detection in a cloud network. The experimental results of the proposed model have been compared with the results of existing recent approaches in the case of binary classification and multi-class classification. It is found that GWO-CSA-DSAE model is better for intrusion detection, which is the proposed model for intrusion detection in a cloud network.