Unsupervised Deep Learning Algorithm Research Articles

Scan-Specific Unsupervised Highly Accelerated Non-Cartesian CEST Imaging Using Implicit Neural Representation and Explicit Sparse Prior.

Chemical exchange saturation transfer (CEST) is a promising magnetic resonance imaging (MRI) technique. CEST imaging usually requires a long scan time, and reducing acquisition time is highly desirable for clinical applications. A novel scan-specific unsupervised deep learning algorithm is proposed to accelerate steady-state pulsed CEST imaging with golden-angle stack-of-stars trajectory using hybrid-feature hash encoding implicit neural representation. Additionally, imaging quality is further improved by using the explicit prior knowledge of low rank and weighted joint sparsity in the spatial and Z-spectral domain of CEST data. In the retrospective acceleration experiment, the proposed method outperforms other state-of-the-art algorithms (TDDIP, LRTES, kt-SLR, NeRP, CRNN, and PBCS) for the in vivo human brain dataset under various acceleration rates. In the prospective acceleration experiment, the proposed algorithm can still obtain results close to the fully-sampled images. The hybrid-feature hash encoding implicit neural representation combined with explicit sparse prior (INRESP) can efficiently accelerate CEST imaging. The proposed algorithm achieves reduced error and improved image quality compared to several state-of-the-art algorithms at relatively high acceleration factors. The superior performance and the training database-free characteristic make the proposed algorithm promising for accelerating CEST imaging in various applications.

Unsupervised deep metric learning algorithm for crop disease images based on knowledge distillation networks

Unsupervised deep metric learning algorithm for crop disease images based on knowledge distillation networks

Attention non-negative spectral clustering

Deep spectral clustering is an advanced unsupervised deep learning method with widespread applications. However, when applied to large-scale datasets, it may confront problems such as an unstable training process, limited scalability, and diminished interpretability. To mitigate these limitations, a novel unsupervised deep learning algorithm named Attention Non-negative Spectral Clustering (ANSC) is proposed. Specifically, a network architecture called the Spectral Attention Network (SAN) is designed to serve as the backbone network for ANSC. This design aims to improve the stability of the training process and the model’s scalability. This attention-based network architecture includes a feature extraction network, an eigenvector mapping network, and an orthogonalization module. Furthermore, to enhance the interpretability of ANSC, the objective function incorporates a cluster indicator matrix by introducing non-negative constraints. Additionally, the ℓ2,1-norm is employed as regularization to approximate the solution of the objective function. An analysis of a lower bound for the network size of ANSC is conducted based on the VC dimension. Extensive experiments validate that ANSC outperforms state-of-the-art clustering methods in terms of clustering accuracy, normalized mutual information, and adjusted rand index.

Anomaly detection in automated fibre placement: learning with data limitations

Introduction: Conventional defect detection systems in Automated Fibre Placement (AFP) typically rely on end-to-end supervised learning, necessitating a substantial number of labelled defective samples for effective training. However, the scarcity of such labelled data poses a challenge.Methods: To overcome this limitation, we present a comprehensive framework for defect detection and localization in Automated Fibre Placement. Our approach combines unsupervised deep learning and classical computer vision algorithms, eliminating the need for labelled data or manufacturing defect samples. It efficiently detects various surface issues while requiring fewer images of composite parts for training. Our framework employs an innovative sample extraction method leveraging AFP’s inherent symmetry to expand the dataset. By inputting a depth map of the fibre layup surface, we extract local samples aligned with each composite strip (tow).Results: These samples are processed through an autoencoder, trained on normal samples for precise reconstructions, highlighting anomalies through reconstruction errors. Aggregated values form an anomaly map for insightful visualization. The framework employs blob detection on this map to locate manufacturing defects.Discussion: The experimental findings reveal that despite training the autoencoder with a limited number of images, our proposed method exhibits satisfactory detection accuracy and accurately identifies defect locations. Our framework demonstrates comparable performance to existing methods, while also offering the advantage of detecting all types of anomalies without relying on an extensive labelled dataset of defects.

A novel data-driven relationship inference approach for automatic data tagging in building heating, ventilation and air conditioning systems

Building automation systems (BAS) data plays a critical role in monitoring a building's operational performance, implementing equipment/system's fault detection and diagnosis, as well as performing building maintenance. However, two major challenges hinder effective data analytics in the building industry. The first challenge is the low data interpretability and interoperability caused by customized naming conventions and various semantic models in BAS data labels. The second challenge is the inconsistent entity relationships among data labels used in various BAS applications. Although some studies have applied semantic analysis to annotate the contextual information of reading data, it relies too much on the quality of data pre-annotation. Considering the inherent ability of BAS data to display the operational status of building entities, we present a novel data-driven reference method to automatically tag the pivotal contextual information of measurements and equipment in heating, ventilation, and air-conditioning (HVAC) systems. The main contributions include: 1) propose an Incremental Classification (IC) method to achieve automatic group tagging of measurements. Hence, the measurements which belong to the same monitoring point or equipment can be classified, 2) integrate Cluster and Correlation (CC) algorithms to identify logical zone divisions, and hence the measurements can be labeled with zone tags, and 3) apply an unsupervised deep learning algorithm, Bidirectional Gated Recurrent Unit (BiGRU) to infer the functional relationship between the air handling unit (AHU) and associated variable air volume (VAV) boxes. We demonstrate the effectiveness of the proposed method by using real BAS data collected in a campus building. Our method results in 88.1 % accuracy in group tagging, 92.3 % accuracy in zone division inference, and 98.4 % accuracy in physical relationship inferring of AHUs and VAV boxes, respectively.

SurgT challenge: Benchmark of soft-tissue trackers for robotic surgery

This paper introduces the “SurgT: Surgical Tracking” challenge which was organized in conjunction with the 25th International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI 2022). There were two purposes for the creation of this challenge: (1) the establishment of the first standardized benchmark for the research community to assess soft-tissue trackers; and (2) to encourage the development of unsupervised deep learning methods, given the lack of annotated data in surgery. A dataset of 157 stereo endoscopic videos from 20 clinical cases, along with stereo camera calibration parameters, have been provided. Participants were assigned the task of developing algorithms to track the movement of soft tissues, represented by bounding boxes, in stereo endoscopic videos. At the end of the challenge, the developed methods were assessed on a previously hidden test subset. This assessment uses benchmarking metrics that were purposely developed for this challenge, to verify the efficacy of unsupervised deep learning algorithms in tracking soft-tissue. The metric used for ranking the methods was the Expected Average Overlap (EAO) score, which measures the average overlap between a tracker’s and the ground truth bounding boxes. Coming first in the challenge was the deep learning submission by ICVS-2Ai with a superior EAO score of 0.617. This method employs ARFlow to estimate unsupervised dense optical flow from cropped images, using photometric and regularization losses. Second, Jmees with an EAO of 0.583, uses deep learning for surgical tool segmentation on top of a non-deep learning baseline method: CSRT. CSRT by itself scores a similar EAO of 0.563. The results from this challenge show that currently, non-deep learning methods are still competitive. The dataset and benchmarking tool created for this challenge have been made publicly available at https://surgt.grand-challenge.org/. This challenge is expected to contribute to the development of autonomous robotic surgery and other digital surgical technologies.

An unsupervised deep learning algorithm for single-site reconstruction in quantum gas microscopes

In quantum gas microscopy experiments, reconstructing the site-resolved lattice occupation with high fidelity is essential for the accurate extraction of physical observables. For short interatomic separations and limited signal-to-noise ratio, this task becomes increasingly challenging. Common methods rapidly decline in performance as the lattice spacing is decreased below half the imaging resolution. Here, we present an algorithm based on deep convolutional neural networks to reconstruct the site-resolved lattice occupation with high fidelity. The algorithm can be directly trained in an unsupervised fashion with experimental fluorescence images and allows for a fast reconstruction of large images containing several thousand lattice sites. We benchmark its performance using a quantum gas microscope with cesium atoms that utilizes short-spaced optical lattices with lattice constant 383.5 nm and a typical Rayleigh resolution of 850 nm. We obtain promising reconstruction fidelities ≳ 96% across all fillings based on a statistical analysis. We anticipate this algorithm to enable novel experiments with shorter lattice spacing, boost the readout fidelity and speed of lower-resolution imaging systems, and furthermore find application in related experiments such as trapped ions.

Unsupervised Deep Learning for Structural Health Monitoring

In the last few decades, structural health monitoring has gained relevance in the context of civil engineering, and much effort has been made to automate the process of data acquisition and analysis through the use of data-driven methods. Currently, the main issues arising in automated monitoring processing regard the establishment of a robust approach that covers all intermediate steps from data acquisition to output production and interpretation. To overcome this limitation, we introduce a dedicated artificial-intelligence-based monitoring approach for the assessment of the health conditions of structures in near-real time. The proposed approach is based on the construction of an unsupervised deep learning algorithm, with the aim of establishing a reliable method of anomaly detection for data acquired from sensors positioned on buildings. After preprocessing, the data are fed into various types of artificial neural network autoencoders, which are trained to produce outputs as close as possible to the inputs. We tested the proposed approach on data generated from an OpenSees numerical model of a railway bridge and data acquired from physical sensors positioned on the Historical Tower of Ravenna (Italy). The results show that the approach actually flags the data produced when damage scenarios are activated in the OpenSees model as coming from a damaged structure. The proposed method is also able to reliably detect anomalous structural behaviors of the tower, preventing critical scenarios. Compared to other state-of-the-art methods for anomaly detection, the proposed approach shows very promising results.

Unsupervised TCN-AE-Based Outlier Detection for Time Series With Seasonality and Trend for Cellular Networks

Timely identification of outliers occurring in key performance indicators (KPIs) of mobile cellular networks is crucial for prompt action to unexpected events. The KPIs of cellular networks typically exhibit seasonality and trend effects and these make the detection of outliers challenging. In this paper, an online unsupervised deep machine learning (DML) algorithm for outlier detection for time series with seasonality and trend is proposed. The proposed algorithm utilizes a neural network based on a temporal convolution network (TCN) and an autoencoder (AE) to reconstruct a time series that captures the normal behavior of the input data and then the reconstruction errors between the input and reconstructed time series at the output of the TCN-AE network are used for outlier detection. To train the TCN-AE network to learn the normal behavior of the input time series, we propose a two-step training process to overcome the presence of outliers in the training data. In addition, a novel loss function specially designed to address the seasonality and trend effects of the time series is proposed. Furthermore, a pre-processing technique is used to combat the adverse trend effect which might cause false alarms in outlier detection. The performance of the proposed TCN-AE-based outlier detection algorithm is evaluated using synthetic time series, Yahoo Webscope dataset and real time series of KPIs of mobile cellular networks. The results show that the proposed TCN-AE-based outlier detection algorithm achieves better detection accuracy in terms of F-score than other DML-based algorithms such as long short-term memory (LSTM)-AE-based algorithm and the convolutional neural network (CNN) based algorithm.

Regimentation of geochemical indicator elements employing convolutional deep learning algorithm

Recently, deep learning algorithms have been popularly developed for identifying multi-element geochemical patterns related to various mineralization occurrences. Effective recognition of multi-element geochemical anomalies is essential for mineral exploration, and effective recognition is extremely dependent on integral clustering. Deep learning algorithms can achieve impressive results in comparison to the prior methods of clustering indicator elements correlated to mineralization for a region of interest due to their superb capability of extracting features from complex data. Although numerous supervised and unsupervised deep learning algorithms have been executed for the recognition of geochemical anomalies, employing them for clustering geochemical indicator elements is rarely observed. In this research, a convolutional deep learning (CDL) algorithm was architected to recognize and regiment geochemical indicator elements in Takht-e Soleyman District, Iran. Various opinions and experiments were considered to reach optimum parameters of this architecture. Fortunately, the achieved root mean square error (RMSE) values were in the appropriate range (<20%) which display the predicted values of the dependent variables (Pb as a pioneer of the first group and Ag as a pioneer of the second group) through their independent variables that are so close to their actual values. Also, the great R2adj calculated (more than 90%) for the last stage of regimentation confirms impressive accuracy and performance of the convolutional deep learning algorithm for clustering geochemical indicator elements of the study area.

Novelty detection on a laboratory benchmark slender structure using an unsupervised deep learning algorithm

Novelty detection on a laboratory benchmark slender structure using an unsupervised deep learning algorithm

Motion Compensated Unsupervised Deep Learning for 5D MRI.

We propose an unsupervised deep learning algorithm for the motion-compensated reconstruction of 5D cardiac MRI data from 3D radial acquisitions. Ungated free-breathing 5D MRI simplifies the scan planning, improves patient comfort, and offers several clinical benefits over breath-held 2D exams, including isotropic spatial resolution and the ability to reslice the data to arbitrary views. However, the current reconstruction algorithms for 5D MRI take very long computational time, and their outcome is greatly dependent on the uniformity of the binning of the acquired data into different physiological phases. The proposed algorithm is a more data-efficient alternative to current motion-resolved reconstructions. This motion-compensated approach models the data in each cardiac/respiratory bin as Fourier samples of the deformed version of a 3D image template. The deformation maps are modeled by a convolutional neural network driven by the physiological phase information. The deformation maps and the template are then jointly estimated from the measured data. The cardiac and respiratory phases are estimated from 1D navigators using an auto-encoder. The proposed algorithm is validated on 5D bSSFP datasets acquired from two subjects.

Burning Skin Detection System in Human Body

Early accurate burn depth diagnosis is crucial for selecting the appropriate clinical intervention strategies and assessing burn patient prognosis quality. However, with limited diagnostic accuracy, the current burn depth diagnosis approach still primarily relies on the empirical subjective assessment of clinicians. With the quick development of artificial intelligence technology, integration of deep learning algorithms with image analysis technology can more accurately identify and evaluate the information in medical images. The objective of the work is to detect and classify burn area in medical images using an unsupervised deep learning algorithm. The main contribution is to developing computations using one of the deep learning algorithm. To demonstrate the effectiveness of the proposed framework, experiments are performed on the benchmark to evaluate system stability. The results indicate that, the proposed system is simple and suits real life applications. The system accuracy was 75%, when compared with some of the state-of-the-art techniques.

Systematic Evaluation of Synthetic Panel Data Quality with an Application to Chronic Lymphocytic Leukemia

OBJECTIVES The dynamic and evolving clinical trial landscape, with its nascent incorporation of real-world data, has the potential to transform healthcare. With these exciting developments come challenges in protecting patient privacy, while retaining data information and encouraging transparency and reproducibility of methods. An emerging technology to ameliorate this challenge is synthetic data generation (SDG). The promises of SDG are to provide realistic, representative, and sharable data that retains all the potential learning of the original (parent) data. We demonstrate a flexible framework for generating and evaluating synthetic data from a challenging clinical use case of chronic lymphocytic leukemia (CLL) patients. We demonstrate the potential to generate a synthetic cohort using real-world data on a cohort of patients treated with chimeric antigen receptor T-cell (CAR T) therapy. Many analytes, and their longitudinal progression, have been shown to predict clinical outcomes in CLL. Therefore, when generating synthetic data for this cohort, it is essential to preserve these longitudinal analyte 'fingerprints’ associated with other clinical information to capture latent disease progression adequately. METHODS We used Generative Adversarial Networks (GANs), a type of unsupervised deep learning algorithm, to generate synthetic CLL patients and their latent disease progression over time. 389 Patients were identified within a large tertiary healthcare system (providing care to approximately 5mil patients) by ICD9/10 codes who could provide longitudinal values for the synthetic cohort. We simulated synthetic patient data using EMR (Electronic Medical Record) data, including laboratory test values, patient-reported health state utility values (HSUVs), and other baseline characteristics. RESULTS Clinical attributes showed a strong relation between analyte trajectories and outcomes. Synthetic data was indistinguishable from original data in both statistical tests and in performance in machine learning algorithms to predict disease progression and worsening outcomes. Wasserstein Conditional GAN outperformed vanilla GAN, conditional GAN, and Wasserstein GAN. Synthetic patient data generated by GAN accurately reflect the means, standard deviations, and correlations of each variable over time to the extent that synthetic data cannot be distinguished from actual data by a logistic regression. Moreover, our unsupervised model predicts changes in total HSUVs with the same accuracy as specifically trained supervised models, additionally capturing the correlation structure of the covariates. LIMITATIONS and CONCLUSIONS Many synthetic data-generative methods emphasize retention of relationship among data elements and may preclude certain data anomalies. The real-world data may retain properties associated with the experiment or data generation process and carry them over into the synthetic cohort. The ideal synthetic cohort would support any statistical discovery possible in, and be verifiable against, the parent dataset - while reducing probability of patient identification to zero. This application of statistical tests to evaluate deep learning algorithms provides a novel perspective on synthetic data generation and poses the bases for the establishment of best practices for synthetic data quality assessment.

Proposal of a new method for learning of diesel generator sounds and detecting abnormal sounds using an unsupervised deep learning algorithm

This study is to find a method to learn engine sound after the start-up of a diesel generator installed in nuclear power plant with an unsupervised deep learning algorithm (CNN autoencoder) and a new method to predict the failure of a diesel generator using it. In order to learn the sound of a diesel generator with a deep learning algorithm, sound data recorded before and after the start-up of two diesel generators was used. The sound data of 20 min and 2 h were cut into 7 s, and the split sound was converted into a spectrogram image. 1200 and 7200 spectrogram images were created from sound data of 20 min and 2 h, respectively. Using two different deep learning algorithms (CNN autoencoder and binary classification), it was investigated whether the diesel generator post-start sounds were learned as normal. It was possible to accurately determine the post-start sounds as normal and the pre-start sounds as abnormal. It was also confirmed that the deep learning algorithm could detect the virtual abnormal sounds created by mixing the unusual sounds with the post-start sounds. This study showed that the unsupervised anomaly detection algorithm has a good accuracy increased about 3% with comparing to the binary classification algorithm.

Attention‐based vector quantisation variational autoencoder for colour‐patterned fabrics defect detection

AbstractDefect detection is an essential link in the fabric production process. Due to the diversity of patterns and scarcity of defect samples for colour‐patterned fabrics, reconstruction‐based unsupervised deep learning algorithms have received extensive attention in the field of fabric defect detection. Among them, unsupervised reconstruction models based on variational autoencoders (VAEs) have been shown to be effective. However, there is a problem of posterior collapse in the process of modelling parametric distributions of continuous variables by VAEs. Therefore, VAE‐based defect detection methods for colour‐patterned fabrics usually produce ambiguous reconstruction results, thereby affecting the defect detection performance. In this article, an attention‐based vector quantisation variational autoencoder (AVQ‐VAE) is proposed for colour‐patterned fabric defect detection. The method adopts autoregressive modelling of discrete variables to avoid the posterior collapse problem of traditional VAEs, and utilises attention mechanism to enhance the feature representation ability of the model. AVQ‐VAE consists of encoder, embedding space, decoder and attention mechanism. The encoder is used to map the input image into multiple feature vectors. Vector quantisation in embedding space is used for discretisation and autoregressive modelling of feature vectors. A decoder is used to decode discrete variables into images of the same size as the original input. Furthermore, an attention mechanism is used to capture channel and spatial correlations, which help the model focus on important information by adaptively recalibrating feature maps. Experimental results on public datasets demonstrate that the proposed method is robust and effective for colour‐patterned fabric defect detection.

Attention-based Feature Fusion Generative Adversarial Network for yarn-dyed fabric defect detection

Defects on the surface of yarn-dyed fabrics are one of the important factors affecting the quality of fabrics. Defect detection is the core link of quality control. Due to the diversity of yarn-dyed fabric patterns and the scarcity of defect samples, reconstruction-based unsupervised deep learning algorithms have received extensive attention in the field of fabric defect detection. However, most existing deep learning algorithms cannot fully extract shallow, high-frequency and high-level information, which limits their ability to reconstruct yarn-dyed fabric images. In this article, we propose an Attention-based Feature Fusion Generative Adversarial Network framework for unsupervised defect detection of yarn-dyed fabrics. The framework utilizes a modified Feature Pyramid Network to fuse multi-level information and utilizes an attention mechanism to enhance the model's feature representation capabilities. The Attention-based Feature Fusion Generative Adversarial Network consists of an attention fusion generator and a patch-level discriminator. In the attention fusion generator, the Feature Pyramid Network with EfficientNetV2 as the backbone is used as the core building block, and different feature fusion methods are used to avoid the loss of information in the process of network deepening. The attention mechanism is used to enhance the channel and spatial-wise correlation of features, which helps the model to focus on more meaningful information by recalibrating the feature maps. In the discriminator, the patch-level discriminator is used to calculate the similarity between the reconstructed image and the original image from a local perspective, thereby improving the model's attention to texture details. Experimental results on public datasets demonstrate the effectiveness of the proposed method compared to other methods.

A New Unsupervised Deep Learning Algorithm for Fine-Grained Detection of Driver Distraction

Traffic accidents caused by distracted drivers account for a large proportion of traffic accidents each year, and monitoring the driving state of drivers to avoid traffic accidents caused by distracted driving has become a very important research direction. At present, the field of driver distraction detection mainly adopts supervised learning methods, which have problems such as poor generalization ability, large labeling cost, and weak artificial intelligence. This paper is oriented toward driver distraction fine-grained detection and innovatively proposes a new unsupervised deep learning algorithm, which is referred to as UDL, to achieve a more human-like level of intelligence. First, we build a new unsupervised deep learning algorithm; furthermore, we integrate the multilayer perceptron (MLP) architecture to build a new backbone and projection head to strengthen feature extraction capabilities; and finally, a new loss function based on contrast learning and a stop-gradient strategy is designed to guide the model to learn more robust features. The comparison results on large-scale driver distraction detection datasets show that our UDL method can accurately detect driver distraction without labels and exhibits excellent generalization performance with a linear evaluation accuracy of 97.38%; In addition, after fine-tuning with fewer labels, our UDL method can achieve superior performance close to state-of-the-art supervised learning methods, achieving 99.07% accuracy after fine-tuning using only 50% of the labeled data, which greatly reduces the cost and limitations of manual annotation.

An unsupervised machine learning approach using passive movement data to understand depression and schizophrenia

IntroductionSchizophrenia and Major Depressive Disorder (MDD) are highly burdensome mental disorders, with significant cost to both individuals and society. Despite these disorders representing distinct clinical categories, they are each heterogenous in their symptom profiles, with considerable transdiagnostic features. Although movement and sleep abnormalities exist in both disorders, little is known of the precise nature of these changes longitudinally. Passively-collected longitudinal data from wearable sensors is well suited to characterize naturalistic features which may cross traditional diagnostic categories (e.g., highlighting behavioral markers not captured by self-report information). MethodsThe present analyses utilized raw minute-level actigraphy data from three diagnostic groups: individuals with schizophrenia (N = 23), individuals with depression (N = 22), and controls (N = 32), respectively, to interrogate naturalistic behavioral differences between groups. Subjects' week-long actigraphy data was processed without diagnostic labels via unsupervised machine learning clustering methods, in order to investigate the natural bounds of psychopathology. Further, actigraphic data was analyzed across time to determine timepoints influential in model outcomes. ResultsWe find distinct actigraphic phenotypes, which differ between diagnostic groups, suggesting that unsupervised clustering of naturalistic data aligns with existing diagnostic constructs. Further, we found statistically significant inter-group differences, with depressed persons showing the highest behavioral variability. LimitationsHowever, diagnostic group differences only consider biobehavioral trends captured by raw actigraphy information. ConclusionsPassively-collected movement information combined with unsupervised deep learning algorithms shows promise in identifying naturalistic phenotypes in individuals with mental health disorders, specifically in discriminating between MDD and schizophrenia.

Learning Representations Using RNN Encoder-Decoder for Edge Security Control.

Whitelisting is a widely used method in the security field. However, due to the rapid development of the Internet, the traditional whitelisting method cannot promote the security of increasing Internet access. In recent years, with the success of machine learning in different areas, many researchers focus on the security of Internet access through machine learning methods. The most common form of machine learning is supervised learning. Supervised learning requires a large number of labeled samples, but it is difficult to obtain labeled samples in practical applications. This paper introduced an unsupervised deep learning algorithm based on seq2seq, which combined with the recurrent neural network and the autoencoder structure to realize an intelligent boundary security control mechanism. The main methods proposed in this paper are divided into two parts: data processing and modeling. In the phase of data processing, the access text table was coded with dicts, and all sequences were padded to the maximum. In the modeling phase, the network was optimized according to the principle of minimizing the reconstruction error. From the comparative experiments, the proposed method's AUC on the public data set reached 0.99, and its performance is better than several classical supervised learning algorithms, proving that the proposed method has an efficient defense against abnormal network access.