Deep Sparse Research Articles

Automatic deep sparse clustering with a dynamic population-based evolutionary algorithm using reinforcement learning and transfer learning

Clustering data effectively remains a significant challenge in machine learning, particularly when the optimal number of clusters is unknown. Traditional deep clustering methods often struggle with balancing local and global search, leading to premature convergence and inefficiency. To address these issues, we introduce ADSC-DPE-RT (Automatic Deep Sparse Clustering with a Dynamic Population-based Evolutionary Algorithm using Reinforcement Learning and Transfer Learning), a novel deep clustering approach. ADSC-DPE-RT builds on Multi-Trial Vector-based Differential Evolution (MTDE), an algorithm that integrates sparse auto-encoding and manifold learning to enable automatic clustering without prior knowledge of cluster count. However, MTDE's fixed population size can lead to either prolonged computation or premature convergence. Our approach introduces a dynamic population generation technique guided by Reinforcement Learning (RL) and Markov Decision Process (MDP) principles. This allows for flexible adjustment of population size, preventing premature convergence and reducing computation time. Additionally, we incorporate Generative Adversarial Networks (GANs) to facilitate dynamic knowledge transfer between MTDE strategies, enhancing diversity and accelerating convergence towards the global optimum. This is the first work to address the dynamic population issue in deep clustering through RL, combined with Transfer Learning to optimize evolutionary algorithms. Our results demonstrate significant improvements in clustering performance, positioning ADSC-DPE-RT as a competitive alternative to state-of-the-art deep clustering methods.

A Continuous Non-Invasive Blood Pressure Prediction Method Based on Deep Sparse Residual U-Net Combined with Improved Squeeze and Excitation Skip Connections.

Arterial blood pressure (ABP) serves as a pivotal clinical metric in cardiovascular health assessments, with the precise forecasting of continuous blood pressure assuming a critical role in both preventing and treating cardiovascular diseases. This study proposes a novel continuous non-invasive blood pressure prediction model, DSRUnet, based on deep sparse residual U-net combined with improved SE skip connections, which aim to enhance the accuracy of using photoplethysmography (PPG) signals for continuous blood pressure prediction. The model first introduces a sparse residual connection approach for path contraction and expansion, facilitating richer information fusion and feature expansion to better capture subtle variations in the original PPG signals, thereby enhancing the network's representational capacity and predictive performance and mitigating potential degradation in the network performance. Furthermore, an enhanced SE-GRU module was embedded in the skip connections to model and weight global information using an attention mechanism, capturing the temporal features of the PPG pulse signals through GRU layers to improve the quality of the transferred feature information and reduce redundant feature learning. Finally, a deep supervision mechanism was incorporated into the decoder module to guide the lower-level network to learn effective feature representations, alleviating the problem of gradient vanishing and facilitating effective training of the network. The proposed DSRUnet model was trained and tested on the publicly available UCI-BP dataset, with the average absolute errors for predicting systolic blood pressure (SBP), diastolic blood pressure (DBP), and mean blood pressure (MBP) being 3.36 ± 6.61 mmHg, 2.35 ± 4.54 mmHg, and 2.21 ± 4.36 mmHg, respectively, meeting the standards set by the Association for the Advancement of Medical Instrumentation (AAMI), and achieving Grade A according to the British Hypertension Society (BHS) Standard for SBP and DBP predictions. Through ablation experiments and comparisons with other state-of-the-art methods, the effectiveness of DSRUnet in blood pressure prediction tasks, particularly for SBP, which generally yields poor prediction results, was significantly higher. The experimental results demonstrate that the DSRUnet model can accurately utilize PPG signals for real-time continuous blood pressure prediction and obtain high-quality and high-precision blood pressure prediction waveforms. Due to its non-invasiveness, continuity, and clinical relevance, the model may have significant implications for clinical applications in hospitals and research on wearable devices in daily life.

A mobile Deep Sparse Wavelet autoencoder for Arabic acoustic unit modeling and recognition

In this manuscript, we introduce a novel methodology for modeling acoustic units within a mobile architecture, employing a synergistic combination of various motivating techniques, including deep learning, sparse coding, and wavelet networks. The core concept involves constructing a Deep Sparse Wavelet Network (DSWN) through the integration of stacked wavelet autoencoders. The DSWN is designed to classify a specific class and discern it from other classes within a dataset of acoustic units. Mel-frequency cepstral coefficients (MFCC) and perceptual linear predictive (PLP) features are utilized for encoding speech units. This approach is tailored to leverage the computational capabilities of mobile devices by establishing deep networks with minimal connections, thereby immediately reducing computational overhead. The experimental findings demonstrate the efficacy of our system when applied to a segmented corpus of Arabic words. Notwithstanding promising results, we will explore the limitations of our methodology. One limitation concerns the use of a specific dataset of Arabic words. The generalizability of the sparse deep wavelet network (DSWN) to various contexts requires further investigation “We will evaluate the impact of speech variations, such as accents, on the performance of our model, for a nuanced understanding.

Drug Repositioning Based on Deep Sparse Autoencoder and Drug-Disease Similarity.

Drug repositioning is critical to drug development. Previous drug repositioning methods mainly constructed drug-disease heterogeneous networks to extract drug-disease features. However, these methods faced difficulty when we are using structurally simple models to deal with complex heterogeneous networks. Therefore, in this study, the researchers introduced a drug repositioning method named DRDSA. The method utilizes a deep sparse autoencoder and integrates drug-disease similarities. First, the researchers constructed a drug-disease feature network by incorporating information from drug chemical structure, disease semantic data, and existing known drug-disease associations. Then, we learned the low-dimensional representation of the feature network using a deep sparse autoencoder. Finally, we utilized a deep neural network to make predictions on new drug-disease associations based on the feature representation. The experimental results show that our proposed method has achieved optimal results on all four benchmark datasets, especially on the CTD dataset where AUC and AUPR reached 0.9619 and 0.9676, respectively, outperforming other baseline methods. In the case study, the researchers predicted the top ten antiviral drugs for COVID-19. Remarkably, six out of these predictions were subsequently validated by other literature sources.

Remaining Useful Life Prediction of Gear Pump Based on Deep Sparse Autoencoders and Multilayer Bidirectional Long–Short–Term Memory Network

Prediction of remaining useful life is crucial for mechanical equipment operation and maintenance. It ensures safe equipment operation, reduces maintenance costs and economic losses, and promotes development. Most of the remaining useful life prediction studies focus on bearings, gearboxes, and engines; however, research on hydraulic pumps remains limited. This study focuses on gear pumps that are commonly used in the hydraulic field and develops a practical method of predicting remaining useful life. The deep sparse autoencoder is used to extract multi–dimensional features. Subsequently, the feature vectors are inputted to the support vector data description to calculate the machine degradation degree at the corresponding time and obtain the health indicator curve of the machine’s life cycle. In building the health state degradation curve, data are processed in an unsupervised manner to avoid the influence of artificial feature selection on the test. The method is validated on the public bearing and self–collected gear pump datasets. The results are better than those of the comparative algorithms: (1) commonly used time–frequency characteristics with principal component analysis and (2) deep sparse autoencoder with self–organizing mapping. Next, the multilayer bidirectional long–short–term memory network is trained as a prediction model using the gear pump health indicator curves obtained previously and applied to the test data. Finally, the proposed method is compared with two others of the same type and the evaluation indexes are calculated based on the prediction results of the three algorithms. From the evaluation indexes, the mean absolute error of the proposed method is reduced by 2.53, and the normalized mean squared error is reduced by 0.36. This result indicates that the prediction results of the method for the remaining useful life of the gear pump are closer to the actual situation.

Authentication and Resource Allocation Strategies during Handoff for 5G IoVs Using Deep Learning

One of the most sought-after applications of cellular technology is transforming a vehicle into a device that can connect with the outside world, similar to smartphones. This connectivity is changing the automotive world. With the speedy growth and densification of vehicles in Internet of Vehicles (IoV) technology, the need for consistency in communication amongst vehicles becomes more significant. This technology needs to be scalable, secure, and flexible when connecting products and services. 5G technology, with its incredible speed, is expected to power the future of vehicular networks. Owing to high mobility and constant change in the topology, cooperative intelligent transport systems ensure real time connectivity between vehicles. For ensuring a seamless connectivity amongst the entities in vehicular networks, a significant alternative to design is support of handoff. This paper proposes a scheme for the best Road Side Unit (RSU) selection during handoff. Authentication and security of the vehicles are ensured using the Deep Sparse Stacked Autoencoder Network (DS2AN) algorithm, developed using a deep learning model. Once authenticated, resource allocation by RSU to the vehicle is accomplished through Deep-Q learning (DQL) techniques. Compared with the existing handoff schemes, Reinforcement Learning based on the MDP (RL-MDP) has been found to have a 13% lesser decision delay for selecting the best RSU. A higher level of security and minimum time requirement for authentication is achieved using DS2AN. The proposed system simulation results demonstrate that it ensures reliable packet delivery, significantly improving system throughput, upholding tolerable delay levels during a change of RSUs.

DS2AN: Deep stacked sparse autoencoder for secure and fast authentication in HetNets

AbstractWith the development of wireless communication technologies, providing seamless communication and secured roaming across devices has gained momentum. With 5G building upon and working together with 4G, it promises up to 100 times faster connectivity than 4G, greater bandwidth and extremely low latency. Though this dramatically enhances user experience, it calls for densification of the network by deploying smaller cells. This solution significantly improves network coverage but results in frequent handoffs. With the adoption of newer network technologies, potential new threats are introduced which becomes quite challenging for the industry. For ensuring seamless mobility, an effective handoff mechanism should ensure a secure and fast authentication process. In this article, Deep Sparse Stacked Autoencoder Network (DS2AN), an unsupervised deep learning model has been used to achieve the said objective. In the proposed method, training is done based on number of hidden layers using back propagation algorithm. Once trained, the cost function is minimized in order to achieve fine tuning of DS2AN.Compared with the existing handoff schemes, DS2AN promises a better level of security and a reduction in time required for handoff. Also, DS2AN is used to identify the presence of malicious entities thereby increasing security in the network. The performance of DS2AN algorithm for authentication delay has been evaluated based on communication and computational cost, and the results shows that DS2AN performs better compared to existing techniques.

Collaborative representation of blur invariant deep sparse features for periocular recognition from smartphones

The periocular region is used for authentication in the recent days under unconstrained acquisition in biometrics. This work presents two new feature extraction techniques to achieve robust and blur invariant biometric verification using periocular images captured using smartphones - (1) Deep Sparse Features (DSF) and (2) Deep Sparse Time Frequency Features (DeSTiFF). Both the approaches are based on extracting features via convolution of periocular images with a set of filters also referred as Deep Sparse Filters. The filters are learnt using natural image patches and sparse filtering approach. The DSF is obtained through convolution via Deep Sparse Filters. Further, convoluted responses are analyzed using Short Term Fourier Transform (STFT) to obtain time and frequency features of the images referred as DeSTIFF. The features obtained from the newly proposed feature extraction techniques are further represented in a collaborative subspace to achieve better verification performance. Both of the proposed feature extraction schemes are evaluated on two publicly available smartphone periocular databases and a new database (Visible Spectrum Periocular Image (VISPI) database) released with this article. The robustness of the proposed feature extraction is exemplified by comparing it with state-of-art approaches along with multiple deep networks where the improvement is evidently seen on large scale database with an average verification accuracy of Genuine Match Rate ≈98% at False Match Rate =0.01%. We further support reproducible research by making the code and the database available for the academic research.

Deep Sparse Coding for Non–Intrusive Load Monitoring

Energy disaggregation is the task of segregating the aggregate energy of the entire building (as logged by the smartmeter) into the energy consumed by individual appliances. This is a single channel (the only channel being the smart-meter) blind source (different electrical appliances) separation problem. The traditional way to address this is via stochastic finite state machines (e.g., factorial hidden Markov model). In recent times, dictionary learning-based approaches have shown promise in addressing the disaggregation problem. The usual technique is to learn a dictionary for every device and use the learned dictionaries as basis for blind source separation during disaggregation. Prior studies in this area are shallow learning techniques, i.e., they learn a single layer of dictionary for every device. In this paper, we propose a deep learning approach—instead of learning one level of dictionary, we learn multiple layers of dictionaries for each device. These multi-level dictionaries are used as a basis for source separation during disaggregation. Results on two benchmark datasets and one actual implementation show that our method outperforms state-of-the-art techniques.

Cancer Characteristic Gene Selection via Sample Learning Based on Deep Sparse Filtering

Identification of characteristic genes associated with specific biological processes of different cancers could provide insights into the underlying cancer genetics and cancer prognostic assessment. It is of critical importance to select such characteristic genes effectively. In this paper, a novel unsupervised characteristic gene selection method based on sample learning and sparse filtering, Sample Learning based on Deep Sparse Filtering (SLDSF), is proposed. With sample learning, the proposed SLDSF can better represent the gene expression level by the transformed sample space. Most unsupervised characteristic gene selection methods did not consider deep structures, while a multilayer structure may learn more meaningful representations than a single layer, therefore deep sparse filtering is investigated here to implement sample learning in the proposed SLDSF. Experimental studies on several microarray and RNA-Seq datasets demonstrate that the proposed SLDSF is more effective than several representative characteristic gene selection methods (e.g., RGNMF, GNMF, RPCA and PMD) for selecting cancer characteristic genes.

IDHSs-PseTNC: Identifying DNase I Hypersensitive Sites with Pseuo Trinucleotide Component by Deep Sparse Auto-encoder

iDHSs-PseTNC: Identifying DNase I Hypersensitive Sites with Pseuo Trinucleotide Component by Deep Sparse Auto-encoder

Developing Iris Recognition System for Smartphone Security

Smartphones have become an important way to store sensitive information; therefore, users’ privacy needs to be highly protected. This can be done by using the most reliable and accurate biometric identification system available today: iris recognition. This paper develops and tests an iris recognition system for smartphones. The system uses eye images that rely on visible wavelength; these images are acquired by the smartphone built-in camera. The development of the system passes through four main phases: the first phase is the iris segmentation phase, which is done in three steps to detect the iris region from the captured image, which contains the eye and part of the face using Haar Cascade Classifier training, pupil localization, and iris localization using a Circular Hough Transform. In the second phase, the system applies normalization using a Rubber Sheet model, which converts the iris image to a fixed size pattern. In the third phase, unique features are extracted from that pattern using a Deep Sparse Filtering algorithm. Finally, in the matching phase, seven different matching techniques are investigated to decide the most appropriate one the system will use to verify the user. Two types of testing are conducted: Offline and Online tests. The BIPLab database and a collected dataset are used to measure the accuracy of the system phases and to calculate the Equal Error Rate (EER) for the whole system. The average EER is 0.18 for the BIPLab database and 0.26 for the collected dataset.

基于深度稀疏学习的土壤近红外光谱分析预测模型

基于深度稀疏学习的土壤近红外光谱分析预测模型