Unsupervised Feature Learning Research Articles

An Analytical Approach for IGBT Life Prediction Using Successive Variational Mode Decomposition and Bidirectional Long Short-Term Memory Networks

The precise estimation of the operational lifespan of insulated gate bipolar transistors (IGBT) holds paramount significance for ensuring the efficient and uncompromised safety of industrial equipment. However, numerous methodologies and models currently employed for this purpose often fall short of delivering highly accurate predictions. The analytical approach that combines the Pattern Optimization Algorithm (POA) with Successive Variational Mode Decomposition (SVMD) and Bidirectional Long Short-term Memory (BiLSTM) network is introduced. Firstly, SVMD is employed as an unsupervised feature learning method to partition the data into intrinsic modal functions (IMFs), which are used to eliminate noise and preserve the essential signal. Secondly, the BiLSTM network is integrated for supervised learning purposes, enabling the prediction of the decomposed sequence. Additionally, the hyperparameters of BiLSTM and the penalty coefficients of SVMD are optimized utilizing the POA technique. Subsequently, the various modal functions are predicted utilizing the trained prediction model, and the individual mode predictions are subsequently aggregated to yield the model’s definitive final life prediction. Through case studies involving IGBT aging datasets, the optimal prediction model was formulated and its lifespan prediction capability was validated. The superiority of the proposed method is demonstrated by comparing it with benchmark models and other state-of-the-art methods.

A novel deep learning framework for rolling bearing fault diagnosis enhancement using VAE-augmented CNN model

In the context of burgeoning industrial advancement, there is an increasing trend towards the integration of intelligence and precision in mechanical equipment. Central to the functionality of such equipment is the rolling bearing, whose operational integrity significantly impacts the overall performance of the machinery. This underscores the imperative for reliable fault diagnosis mechanisms in the continuous monitoring of rolling bearing conditions within industrial production environments. Vibration signals are primarily used for fault diagnosis in mechanical equipment because they provide comprehensive information about the equipment's condition. However, fault data often contain high noise levels, high-frequency variations, and irregularities, along with a significant amount of redundant information, like duplication, overlap, and unnecessary information during signal transmission. These characteristics present considerable challenges for effective fault feature extraction and diagnosis, reducing the accuracy and reliability of traditional fault detection methods. This research introduces an innovative fault diagnosis methodology for rolling bearings using deep convolutional neural networks (CNNs) enhanced with variational autoencoders (VAEs). This deep learning approach aims to precisely identify and classify faults by extracting detailed vibration signal features. The VAE enhances noise robustness, while the CNN improves signal data expressiveness, addressing issues like gradient vanishing and explosion. The model employs the reparameterization trick for unsupervised learning of latent features and further trains with the CNN. The system incorporates adaptive threshold methods, the "3/5" strategy, and Dropout methods. The diagnosis accuracy of the VAE-CNN model for different fault types at different rotational speeds typically reaches more than 90 %, and it achieves a generally acceptable diagnosis result. Meanwhile, the VAE-CNN augmented fault diagnosis model, after experimental validation in various dimensions, can achieve more satisfactory diagnosis results for various fault types compared to several representative deep neural network models without VAE augmentation, significantly improving the accuracy and robustness of rolling bearing fault diagnosis.

Learning sparse feature representation for blind quality assessment of night-time images

Capturing Night-Time Images (NTIs) with high-quality is quite challenging for consumer photography and several practical applications. Thus, addressing the quality assessment of night-time images is urgently needed. Since there is no available reference image for such images, Night-Time image Quality Assessment (NTQA) should be done blindly. Although Blind natural Image Quality Assessment (BIQA) has attracted a great deal of attention for a long time, very little work has been done in the field of NTQA. Due to the capturing conditions, NTIs suffer from various complex authentic distortions that make it a challenging field of research. Therefore, previous BIQA methods, do not provide sufficient correlation with subjective scores in the case of NTIs and special methods of NTQA should be developed. In this paper we conduct an unsupervised feature learning method for blind quality assessment of night-time images. The features are the sparse representation over the data-adaptive dictionaries learned on the image exposure and gradient magnitude maps. Having these features, an ensemble regression model trained using least squares gradient boosting scheme predicts high correlated objective scores on the standard datasets.

A residual autoencoder-based transformer for fault detection of multivariate processes

The complexity of high-dimensional and noisy process signals reduces the effectiveness of conventional fault detection methods in industrial processes. Based on the hypothesis that data collected from normal and faulty processes has different characteristics, unsupervised deep neural networks, e.g., autoencoders, have been widely applied in process fault detection and achieved good performance. Many variants have been proposed to improve feature learning by combining different network structures. In this paper, a new transformer model, residual autoencoder-based transformer, is proposed for process fault detection. Firstly, autoencoder and transformer are integrated for better unsupervised feature learning of process signals. Secondly, linear embedding and attention mechanisms with bias are proposed to generate effective features from process signals. Finally, residual connections are constructed between the encoder and decoder of RATransformer to address overfitting in training. Four industrial cases are used to test the performance of RATransformer for process fault detection. The results show that the fault detection rate of RATransformer is at least 1 % higher than other comparison methods. Moreover, the testing results show that the model structure improves the fault detection performance of RATransformer. The complex models like RATransformer can be used in the industrial process when sufficient normal process data is available. An end-to-end training method can be further developed to improve the applicability of RATransformer in process fault detection in the future.

Examining 22 Years of Ambient Seismic Wavefield at Mount St. Helens

Abstract An increase in seismic activity precedes most volcanic eruptions. Whereas event-based forecasting approaches have been successful, some eruptions remain unanticipated, resulting in casualties and damage. Our study leverages the recent advancements in ambient field seismology. We explore features extracted from continuous ambient fields using traditional methods, for example, peak ground velocity, peak ground acceleration, root mean square, root median square, real-time seismic amplitude measurement, and novel methods (displacement seismic amplitude ratio and spectral width). In addition, we explore unsupervised learning of higher order wavelet features using scattering networks. We find that combining all the methods was necessary to disentangle the effects of seismic sources from structural changes at Mount St. Helens. Although the ambient wavefield-based approach does not yield additional or more significant precursory signals than event-based methods at Mount St. Helens, our study demonstrates that the ambient wavefield provides supplementary information, mainly about structural changes and complements traditional methods. The ambient seismic wavefield offers additional insights into long-lasting processes. We find enhanced wave attenuation correlating with geochemical measurements. We interpret this as ongoing structural changes, such as dome growth or the evolution of the volcanic conduit system. On annual and decadal timescales, we interpret seasonal seismic attenuation in the shallow subsurface as groundwater fluctuations, corroborated by observations at the nearby Spirit Lake level. This multimethod approach at Mount St. Helens sheds light on a volcanic system’s underlying dynamics and structure.

SpaDE: Semantic Locality Preserving Biclustering for Neuroimaging Data.

The most discriminative and revealing patterns in the neuroimaging population are often confined to smaller subdivisions of the samples and features. Especially in neuropsychiatric conditions, symptoms are expressed within micro subgroups of individuals and may only underly a subset of neurological mechanisms. As such, running a whole-population analysis yields suboptimal outcomes leading to reduced specificity and interpretability. Biclustering is a potential solution since subject heterogeneity makes one-dimensional clustering less effective in this realm. Yet, high dimensional sparse input space and semantically incoherent grouping of attributes make post hoc analysis challenging. Therefore, we propose a deep neural network called semantic locality preserving auto decoder (SpaDE), for unsupervised feature learning and biclustering. SpaDE produces coherent subgroups of subjects and neural features preserving semantic locality and enhancing neurobiological interpretability. Also, it regularizes for sparsity to improve representation learning. We employ SpaDE on human brain connectome collected from schizophrenia (SZ) and healthy control (HC) subjects. The model outperforms several state-of-the-art biclustering methods. Our method extracts modular neural communities showing significant (HC/SZ) group differences in distinct brain networks including visual, sensorimotor, and subcortical. Moreover, these bi-clustered connectivity substructures exhibit substantial relations with various cognitive measures such as attention, working memory, and visual learning.

Enhancing surrogate-assisted evolutionary optimization for medium-scale expensive problems: a two-stage approach with unsupervised feature learning and Q-learning

Enhancing surrogate-assisted evolutionary optimization for medium-scale expensive problems: a two-stage approach with unsupervised feature learning and Q-learning

Combining EEG Features and Convolutional Autoencoder for Neonatal Seizure Detection.

Neonatal epilepsy is a common emergency phenomenon in neonatal intensive care units (NICUs), which requires timely attention, early identification, and treatment. Traditional detection methods mostly use supervised learning with enormous labeled data. Hence, this study offers a semi-supervised hybrid architecture for detecting seizures, which combines the extracted electroencephalogram (EEG) feature dataset and convolutional autoencoder, called Fd-CAE. First, various features in the time domain and entropy domain are extracted to characterize the EEG signal, which helps distinguish epileptic seizures subsequently. Then, the unlabeled EEG features are fed into the convolutional autoencoder (CAE) for training, which effectively represents EEG features by optimizing the loss between the input and output features. This unsupervised feature learning process can better combine and optimize EEG features from unlabeled data. After that, the pre-trained encoder part of the model is used for further feature learning of labeled data to obtain its low-dimensional feature representation and achieve classification. This model is performed on the neonatal EEG dataset collected at the University of Helsinki Hospital, which has a high discriminative ability to detect seizures, with an accuracy of 92.34%, precision of 93.61%, recall rate of 98.74%, and F1-score of 95.77%, respectively. The results show that unsupervised learning by CAE is beneficial to the characterization of EEG signals, and the proposed Fd-CAE method significantly improves classification performance.

Exploring tumor heterogeneity in colorectal liver metastases by imaging: Unsupervised machine learning of preoperative CT radiomics features for prognostic stratification

ObjectivesThis study aimed to investigate tumor heterogeneity of colorectal liver metastases (CRLM) and stratify the patients into different risk groups of prognoses following liver resection by applying an unsupervised radiomics machine-learning approach to preoperative CT images. MethodsThis retrospective study retrieved clinical information and CT images of 197 patients with CRLM from The Cancer Imaging Archive (TCIA) database. Radiomics features were extracted from a segmented liver lesion identified at the portal venous phase. Those features which showed high stability, non-redundancy, and indicative information were selected. An unsupervised consensus clustering analysis on these features was adopted to identify subgroups of CRLM patients. Overall survival (OS), disease-free survival (DFS), and liver-specific DFS were compared between the identified subgroups. Cox regression analysis was applied to evaluate prognostic risk factors. ResultsA total of 851 radiomics features were extracted, and 56 robust features were finally selected for unsupervised clustering analysis which identified two distinct subgroups (96 and 101 patients respectively). There were significant differences in the OS, DFS, and liver-specific DFS between the subgroups (all log-rank p < 0.05). The subgroup with worse outcome using the proposed radiomics model was consistently associated with shorter OS, DFS, and liver-specific DFS, with hazard ratios of 1.78 (95 %CI: 1.12–2.83), 1.72 (95 %CI: 1.16–2.54), and 1.59 (95 %CI: 1.10–2.31), respectively. The general performance of this radiomics model outperformed the traditional Clinical Risk Score and Tumor Burden Score in the prognosis prediction after surgery for CRLM. ConclusionRadiomics features derived from preoperative CT images can reveal the heterogeneity of CRLM and stratify the patients with CRLM into subgroups with significantly different clinical outcomes.

Large language models and unsupervised feature learning: implications for log analysis

Large language models and unsupervised feature learning: implications for log analysis

Brain Tumor Detection Using Classification Model

This study aims to develop automated methods for detecting and segmenting brain tumors in MRI scans using neural networks and conventional image processing. Accurately identifying tumor location and boundaries is crucial for diagnosis and determining the most effective treatment approach. Manual review by radiologists is time-consuming and subjective. However, recent advances in deep learning now enable more efficient analysis of large MRI datasets.&#x0D; A variety of neural network models have been explored for tumor segmentation, including recurrent networks, autoencoders and convolutional networks. RNNs are well-suited to capture contextual information across sequential MRI slices. Autoencoders perform unsupervised feature learning from the images. CNNs have shown promise due to their ability to directly learn visual patterns from pixel values. Unsupervised clustering techniques such as fuzzy c-means and k-means are also investigated to group tumor pixels based on intensity similarity without labeled data.&#x0D; Previous research combining these techniques has achieved high segmentation accuracy between 84-97% using additional image processing steps. Automated segmentation provides benefits over manual inspection such as reduced workload and more consistent delineation of tumor margins. It can precisely outline growths to aid radiologists in diagnosis and pathologists in assessing malignancy.&#x0D; The detection and segmentation of brain tumors has applications in both radiology and oncology. Radiologists could benefit from faster scan review and automated report generation. Neurosurgeons would gain insights into a tumor's size, location and infiltration to determine the most suitable surgical approach. Oncologists could utilize quantitative tumor features to select the most targeted radiation therapy or chemotherapy protocols. Overall, continued advances in deep and machine learning hold promise to improve diagnosis and management of brain cancer patients.

Recognizing Bearings’ Degradation Stage Using Multimodal Autoencoder to Learn Features from Different Time Series

Utilizing machine learning technologies to monitor assets’ health conditions can improve the effectiveness of maintenance activities. However, accurately recognizing the current health degradation stages of industrial assets requires a time-consuming manual feature extraction due to the wide range of observable measures (e.g., temperature, vibration) and behaviors characterizing assets’ degradation. To address this issue, feature learning technology can transform minimally processed time series into informative features, i.e., able to simplify the classification task (e.g., recognizing degradation stages) regardless of the specific machine learning classifier employed. In this work, minimally preprocessed time series of vibration and temperature of industrial bearings are exploited by an autoencoder-based architecture to extract degradation-representative features to be used for recognizing their degradation stages. Different autoencoder architectures are employed to compare their data fusion strategies. The effectiveness of the proposed approach is evaluated in terms of recognition performance and the quality of the learned features by using a publicly available real-world dataset and comparing the proposed approach against a state-of-the-art feature learning technology. We tested three different multimodal autoencoder-based feature learning approaches, i.e., shared-input autoencoder (SAE), multimodal autoencoder (MMAE), and partition-based autoencoder (PAE). All the AE-based architecture results in classification performances greater or comparable with the state-of-the-art feature learning technology, despite being trained in an unsupervised fashion. Also, the features provided via PAE correspond to the greatest performances in recognizing bearings’ degradation stage, providing high-quality features both from a classification and clustering perspective. Unsupervised feature learning methodologies based on multimodal autoencoders are capable of learning high-quality features. These result in greater degradation stages recognition performances when compared to supervised state-of-the-art feature learning technology. Also, this enables the correct representation of the expected progressive degradation of the bearing.

Customized Mechanism for Diabetic Risk Prediction: A Hybrid CNN–Autoencoder Approach with Emphasis on Retinal Imaging in the Elderly

Diabetes Mellitus presents a substantial health obstacle on a global scale, with a particular impact on the elderly demographic. Prompt identification is vital for efficient control and avoidance of complications. This study introduces a new Hybrid Convolutional Neural Network (CNN) and Autoencoder model specifically developed for accurately predicting the risk of diabetes at an early stage. The model is specifically designed to analyze retinal images in older individuals. The introduction of this paper presents a comprehensive analysis of the increasing incidence of diabetes in the elderly population and underscores the significance of early identification. Conventional approaches frequently encounter constraints in terms of precision and specificity, which has led to the investigation of sophisticated machine learning models. The Hybrid CNN–Autoencoder model combines the advantageous characteristics of both architectures, utilizing the CNN proficiency in extracting spatial features and the Autoencoder's capability for unsupervised feature learning. The approach we use consists of training and validating the model using a comprehensive dataset of retinal images from elderly individuals. The model attains a remarkable accuracy of 90.92%, outperforming the typical deep learning and machine learning models frequently employed in predicting diabetic risk. The experimental results demonstrate the superior performance of the Hybrid CNN–Autoencoder model in terms of accuracy, sensitivity, and specificity. Comparative analysis shows that it is highly effective in identifying subtle patterns that indicate early signs of diabetes, surpassing traditional models and other modern deep learning methods. The research findings presented in this study make a valuable contribution to the expanding knowledge base on diabetes detection, specifically within the elderly population. The proven precision of the suggested model highlights its capacity as a dependable and tailored instrument for early forecasting, thus enabling prompt interventions and individualized healthcare strategies for individuals susceptible to diabetes.

Enhancing Electric Vehicle Range Prediction through Deep Learning: An Autoencoder and Neural Network Approach

The burgeoning adoption of electric vehicles (EVs) signifies a pivotal shift towards sustainable transportation, necessitated by the global imperative to mitigate climate change impacts. Central to this transition is the resolution of range anxiety, a significant barrier impeding wider EV acceptance. This research introduces a novel deep learning framework combining autoencoders and deep neural networks (DNNs) to predict EV range more accurately and reliably. Leveraging a comprehensive dataset from the "Electric Vehicle Population Data," we embarked on a meticulous process of data cleaning, feature engineering, and preprocessing to prepare the dataset for analysis. The study innovatively applies an autoencoder for unsupervised feature learning, effectively reducing dimensionality and extracting salient features from high-dimensional EV data. Subsequently, a DNN model utilizes these features to predict the EV range, offering insights into the vehicle's performance across various conditions. Employing a 10-fold cross-validation approach, the model's efficacy is rigorously evaluated, ensuring robustness and generalizability of the predictions. Our methodology demonstrates a significant enhancement in prediction accuracy compared to conventional machine learning models, as evidenced by the Mean Squared Error (MSE) metric. This research not only contributes to the academic discourse on sustainable transportation and deep learning applications but also provides practical insights for manufacturers, policymakers, and consumers aiming to navigate the complexities of EV adoption and infrastructure development. By addressing the critical challenge of range prediction, this study paves the way for advancing EV analytics, ultimately supporting the transition to a more sustainable and efficient transportation ecosystem.

Prediction of cervical lymph nodes recurrence after radiotherapy for early nasopharyngeal carcinoma via unsupervised diagnostic feature learning and supervised ensemble classifier learning

Purpose:Nasopharyngeal carcinoma is one of the most prevalent malignant tumors in Guangdong, China. In the field of medicine, predicting the recurrence of early-stage nasopharyngeal carcinoma after radiotherapy holds significant importance. Our objective was to develop a novel classification model aimed at predicting recurrence following radiotherapy for early nasopharyngeal carcinoma. Methods and Materials:This paper introduces an innovative approach by combining unsupervised diagnostic feature learning (biclustering algorithm) with Adaboost ensemble learning to create a novel classification model. Notably, the nasopharyngeal carcinoma dataset underwent biclustering algorithm application for the first time. Initially, the Borderline-SMOTE oversampling algorithm was employed to address the dataset’s imbalance issue. Subsequently, the biclustering algorithm, which is based on an improved multi-objective genetic algorithm (NSGA-II), was utilized to seek bicluster outcomes exhibiting consistent representation patterns. The attributes of these bicluster outcomes were assessed and employed as diagnostic rules for constructing weak classifiers. Ultimately, the Adaboost ensemble learning technique was employed to amalgamate the weak classifiers into a robust classifier. Results:Following the application of 10-fold cross-validation, the model exhibited an accuracy of 80.33%, sensitivity of 79.55%, specificity of 80.43%, and GMean of 79.99%, accompanied by an AUC value of 0.904. Conclusions:The presented classification model outperformed alternative classification models in terms of classification accuracy and generalization when applied to the nasopharyngeal carcinoma dataset. Consequently, it serves as a valuable tool to aid medical professionals in predicting the likelihood of recurrence in the cervical lymph nodes following radiotherapy for early-stage nasopharyngeal carcinoma.

Unsupervised Deep Feature Learning for Icequake Discrimination at Neumayer Station, Antarctica

Abstract Unsupervised machine learning methods are gaining attention in the seismological community as more and larger datasets of continuous waveforms are collected. Recently, contrastive learning for unsupervised feature learning has shown great success in the field of computer vision and other domains, and we aim to transfer these methods to the domain of seismology. Contrastive learning algorithms use data augmentation to implement an instance-level discrimination task: The feature representations of two augmented versions of the same data example are trained to be similar, when at the same time dissimilar to other data examples. In particular, we use the popular contrastive learning method SimCLR. We test data augmentation strategies varying amplitude and frequency of seismological signals, and apply contrastive learning methods to automatically learn features. We use a dataset containing various mostly cryogenic waveforms detected by an STA/LTA short-term average/long-term average algorithm on continuous waveform recordings from the geophysical observatory at Neumayer station, Antarctica. The quality of the features is evaluated on a hand-labeled dataset that includes icequakes, earthquakes, and spikes, and on a larger unlabeled dataset using a classical clustering method, k-means. Results show that the approach separates the different hand-labeled groups with an accuracy of up to 88% and separates meaningful groups within the unlabeled data. Thus, we provide an effective tool for the unsupervised exploration of large seismological datasets and the automated compilation of event catalogs.

Contrastive Unsupervised Representation Learning With Optimize-Selected Training Samples.

Contrastive unsupervised representation learning (CURL) is a technique that seeks to learn feature sets from unlabeled data. It has found widespread and successful application in unsupervised feature learning, with the design of positive and negative pairs serving as the type of data samples. While CURL has seen empirical successes in recent years, there is still room for improvement in terms of the pair data generation process. This includes tasks such as combining and re-filtering samples, or implementing transformations among positive/negative pairs. We refer to this as the sample selection process. In this article, we introduce an optimized pair-data sample selection method for CURL. This method efficiently ensures that the two types of sampled data (similar pair and dissimilar pair) do not belong to the same class. We provide a theoretical analysis to demonstrate why our proposed method enhances learning performance by analyzing its error probability. Furthermore, we extend our proof into PAC-Bayes generalization to illustrate how our method tightens the bounds provided in previous literature. Our numerical experiments on text/image datasets show that our method achieves competitive accuracy with good generalization bounds.

Unsupervised learning of local features for person re-identication with loss funciton

Unsupervised learning of local features for person re-identication with loss funciton

Unsupervised Multiview Graph Contrastive Feature Learning for Hyperspectral Image Classification

Unsupervised Multiview Graph Contrastive Feature Learning for Hyperspectral Image Classification

Update Algorithm of Secure Computer Database Based on Deep Belief Network

In order to ensure the security of large-scale data transmission in a short time and in a wide range during online database updating, this paper presents a secure computer database updating algorithm based on DBN (Deep Belief Network). In this paper, the model adopts multi-layer depth structure for unsupervised feature learning, maps high-dimensional and nonlinear intrusion data to low-dimensional space, establishes the relationship mapping between high-dimensional and low-dimensional, and then uses fine-tuning algorithm to transform the model to achieve the best expression of features. At the same time, this method improves the data processing and method model without destroying the learned knowledge of the model and seriously affecting the real-time performance of detection. In order to overcome the problem of system instability caused by fixed empirical learning rate, this paper proposes a learning rate optimization strategy based on energy change. In the process of feature extraction, the features of different hidden layers are extracted to form combined features. Experiments show that the detection rate of this method can reach 95.31%, and the false alarm rate is 2.14%. This verifies the effectiveness of the secure computer database updating algorithm in this paper. Which can ensure the online update of the secure computer database.