Unsupervised Method Research Articles

Gromov–Wasserstein unsupervised alignment reveals structural correspondences between the color similarity structures of humans and large language models

Large Language Models (LLMs), such as the General Pre-trained Transformer (GPT), have shown remarkable performance in various cognitive tasks. However, it remains unclear whether these models have the ability to accurately infer human perceptual representations. Previous research has addressed this question by quantifying correlations between similarity response patterns of humans and LLMs. Correlation provides a measure of similarity, but it relies pre-defined item labels and does not distinguish category- and item- level similarity, falling short of characterizing detailed structural correspondence between humans and LLMs. To assess their structural equivalence in more detail, we propose the use of an unsupervised alignment method based on Gromov–Wasserstein optimal transport (GWOT). GWOT allows for the comparison of similarity structures without relying on pre-defined label correspondences and can reveal fine-grained structural similarities and differences that may not be detected by simple correlation analysis. Using a large dataset of similarity judgments of 93 colors, we compared the color similarity structures of humans (color-neurotypical and color-atypical participants) and two GPT models (GPT-3.5 and GPT-4). Our results show that the similarity structure of color-neurotypical participants can be remarkably well aligned with that of GPT-4 and, to a lesser extent, to that of GPT-3.5. These results contribute to the methodological advancements of comparing LLMs with human perception, and highlight the potential of unsupervised alignment methods to reveal detailed structural correspondences.

Self-supervised tomographic image noise suppression via residual image prior network

Computed tomography (CT) denoising is a challenging task in medical imaging that has garnered considerable attention. Supervised networks require a lot of noisy-clean image pairs, which are always unavailable in clinical settings. Existing self-supervised algorithms for suppressing noise with paired noisy images have limitations, such as ignoring the residual between similar image pairs during training and insufficiently learning the spectrum information of images. In this study, we propose a Residual Image Prior Network (RIP-Net) to sufficiently model the residual between the paired similar noisy images. Our approach offers new insights into the field by addressing the limitations of existing methods. We first establish a mathematical theorem clarifying the non-equivalence between similar-image-based self-supervised learning and supervised learning. It helps us better understand the strengths and limitations of self-supervised learning. Secondly, we introduce a novel regularization term to model a low-frequency residual image prior. This can improve the accuracy and robustness of our model. Finally, we design a well-structured denoising network capable of exploring spectrum information while simultaneously sensing context messages. The network has dual paths for modeling high and low-frequency compositions in the raw noisy image. Additionally, context perception modules capture local and global interactions to produce high-quality images. The comprehensive experiments on preclinical photon-counting CT, clinical brain CT, and low-dose CT datasets, demonstrate that our RIP-Net is superior to other unsupervised denoising methods.

An unsupervised learning-based guidewire shape registration for vascular intervention surgery robot

In a vascular interventional surgery robot(VISR), a high transparency master-slave system can aid physicians in the more precise manipulation of guidewires for navigation and operation within blood vessels. However, deformations arising from the movement of the guidewire can affect the accuracy of the registration, thus reducing the transparency of the master-slave system. In this study, the degree of the guidewire’s deformation is analyzed based on the Kirchhoff model. An unsupervised learning-based guidewire shape registration method(UL-GSR) is proposed to estimate geometric transformations by learning displacement field functions. It can effectively achieve precise registration of flexible bodies. This method not only demonstrates high registration accuracy but also performs robustly under different complexity degrees of guidewire shapes. The experiments have demonstrated that the UL-GSR method significantly improves the accuracy of shape point set registration between the master and slave sides, thus enhancing the transparency and operational reliability of the VISR system.

Classification of Human Resources Readiness in Asia: K-Means Clustering Using R Studio

This research aims to classify dichotomously the differences and diversity in human resources of all countries in the Asian region to clarify the position of each country based on the state of its human resources. Several countries were not included in the analysis due to limited data availability. This research uses the K-Mean clustering method. Clustering is an approach to dividing a set of points into similar groups called clusters. K-Mean clustering is one of the most popular unsupervised learning methods in machine learning. Based on the results of the K-Mean clustering analysis, the readiness of human resources in Asian countries shows that there are clusters or groups, with cluster 1 having as many as 29 members and cluster 2 having as many as 9 members. K-Mean clustering for the optimal number of two clusters is dichotomous, so it uses clustering assumptions between country clusters with good and poor human resource readiness.

A Knowledge-Driven Approach for Automatic Semantic Aspect Term Extraction Using the Semantic Power of Linked Open Data

Aspect-Based Sentiment Analysis (ABSA) is a crucial process for assessing customer feedback and gauging satisfaction with products or services. It typically consists of three stages: Aspect Term Extraction (ATE), Aspect Categorization Extraction (ACE), and Sentiment Analysis (SA). Various techniques have been proposed for ATE, including unsupervised, supervised, and hybrid methods. However, many studies face challenges in detecting aspect terms due to reliance on training data, which may not cover all multiple aspect terms and relate semantic aspect terms effectively. This study presents a knowledge-driven approach to automatic semantic aspect term extraction from customer feedback using Linked Open Data (LOD) to enrich aspect extraction outcomes in the training dataset. Additionally, it utilizes the N-gram model to capture complex text patterns and relationships, facilitating accurate classification and analysis of multiple-word terms for each aspect. To assess the effectiveness of the proposed model, experiments were conducted on three benchmark datasets: SemEval 2014, 2015, and 2016. Comparative evaluations with contemporary unsupervised, supervised, and hybrid methods on these datasets yielded F-measures of 0.80, 0.76, and 0.77, respectively.

Identifying novel data-driven subgroups in congenital heart disease using multi-modal measures of brain structure

Individuals with congenital heart disease (CHD) have an increased risk of neurodevelopmental impairments. Given the hypothesized complexity linking genomics, atypical brain structure, cardiac diagnoses and their management, and neurodevelopmental outcomes, unsupervised methods may provide unique insight into neurodevelopmental variability in CHD. Using data from the Pediatric Cardiac Genomics Consortium Brain and Genes study, we identified data-driven subgroups of individuals with CHD from measures of brain structure. Using structural magnetic resonance imaging (MRI; N = 93; cortical thickness, cortical volume, and subcortical volume), we identified subgroups that differed primarily on cardiac anatomic lesion and language ability. In contrast, using diffusion MRI (N = 88; white matter connectivity strength), we identified subgroups that were characterized by differences in associations with rare genetic variants and visual-motor function. This work provides insight into the differential impacts of cardiac lesions and genomic variation on brain growth and architecture in patients with CHD, with potentially distinct effects on neurodevelopmental outcomes.

A restarted large-scale spectral clustering with self-guiding and block diagonal representation

Spectral clustering, a prominent unsupervised machine learning method, involves a critical task of constructing a similarity matrix. In existing approaches, this matrix is either computed once for all or updated alternatively. However, the former struggles to capture comprehensive relationships among data points, and the latter is often impractical for large-scale problems. This study introduces a new clustering framework with self-guidance and a block diagonal representation, retaining valuable information from previous cycles. To our knowledge, this is the first application of such a framework to spectral clustering, with a key distinction being the reclassification of samples in each cycle. To reduce computational overhead, we employ a block diagonal representation with Nyström approximation for constructing the similarity matrix. Theoretical results justify the rationality of approximate computations in spectral clustering. Comprehensive experiments on benchmark datasets demonstrate the superiority of our proposed algorithms over state-of-the-art methods for large-scale clustering. Notably, our framework has the potential to enhance clustering algorithms, performing well even with a randomly chosen initial guess.

Unsupervised Clustering of Membranoproliferative Glomerulonephritis and C3 Glomerulopathy Patients Discovers Distinct Patient Groups unlike the Current Classification.

Membranoproliferative glomerulonephritis is currently divided into immunoglobulin-mediated glomerulonephritis (IC-MPGN) and C3 glomerulopathy (C3G); however, the patients often overlap with histology, complement, clinical and prognostic factors. Our aim was to investigate if an unsupervised clustering method finds different patient groups in 44 IC-MPGN/C3G patients using only histological and clinical data available in everyday clinical work. Primary IC-MPGN/C3G adult patients were included whose diagnostic (baseline) native biopsy was obtained in 2006-2017. The biopsies were reassessed and the clinical data at baseline and during follow-up were obtained from the medical records. There were 39 baseline histological and clinical variables included in the unsupervised clustering. Follow-up information was combined with the clustering results. The clustering resulted in two clusters (n = 24 and n = 20 patients for clusters 1-2, respectively), where cluster 1 had a significantly higher baseline plasma creatinine (mean 213 vs. 104, respectively, p value <0.001) and a lower baseline eGFR than cluster 2 (mean 37 vs. 70, respectively, p value <0.001). Regarding histology, chronic changes such as lobulated glomeruli, mesangial matrix expansion, and glomeruli double contours were more prevalent in cluster 1 (p value <0.001). Biopsy morphology was more often crescentic and membranoproliferative in cluster 1 (p value <0.001). Although the differences were insignificant, cluster 1 patients were in dialysis in the last follow-up or had a progressive disease more often than cluster 2 patients (21% vs. 5%, 38% vs. 10%). Our results indicate that these patients share greater similarity than the current classification IC-MPGN versus C3G indicates.

A Method of Integrating Air Conditioning Usage Models to Building Simulations for Predicting Residential Cooling Energy Consumption

Great deviations in building energy consumption simulation are attributed to the simplified settings of occupants’ air conditioning (AC) usage schedules. This study was designed to develop a method to quantify the uncertainty and randomness of AC usage behavior and incorporate the model into simulations, in order to improve the prediction performance of AC energy consumption. Based on long-term onsite monitoring of household thermal environments and AC usage patterns, two stochastic models were built using unsupervised cluster and statistical methods. Based on the Monte Carlo method, the AC operation schedule was generated through AC opening duration, setpoints, and other relevant parameters, and was further incorporated into EnergyPlus. The results show that the ideally deterministic AC operation settings from the standard significantly overestimate the cooling energy consumption, where the value based on the fixed mode was 6.35 times higher. The distribution of daily AC energy consumption based on the stochastic modeling was highly consistent with the actual situation, thanks to the accurate prediction of the randomness and dynamics of residents’ AC usage patterns. The total cooling energy consumption based on two stochastic models was found to be much closer to the actual values. The work proposes a method of embedding stochastic AC usage models to EnergyPlus 22.1 benefits for an improvement in building energy consumption simulation and the energy efficiency evaluation regarding occupant behavior in the future.

Heuristic approaches for non-exhaustive pattern-based change detection in dynamic networks

AbstractDynamic networks are ubiquitous in many domains for modelling evolving graph-structured data and detecting changes allows us to understand the dynamic of the domain represented. A category of computational solutions is represented by the pattern-based change detectors (PBCDs), which are non-parametric unsupervised change detection methods based on observed changes in sets of frequent patterns over time. Patterns have the ability to depict the structural information of the sub-graphs, becoming a useful tool in the interpretation of the changes. Existing PBCDs often rely on exhaustive mining, which corresponds to the worst-case exponential time complexity, making this category of algorithms inefficient in practice. In fact, in such a case, the pattern mining process is even more time-consuming and inefficient due to the combinatorial explosion of the sub-graph pattern space caused by the inherent complexity of the graph structure. Non-exhaustive search strategies can represent a possible approach to this problem, also because not all the possible frequent patterns contribute to changes in the time-evolving data. In this paper, we investigate the viability of different heuristic approaches which prevent the complete exploration of the search space, by returning a concise set of sub-graph patterns (compared to the exhaustive case). The heuristics differ on the criterion used to select representative patterns. The results obtained on real-world and synthetic dynamic networks show that these solutions are effective, when mining patterns, and even more accurate when detecting changes.

Unsupervised denoising of photoacoustic images based on the Noise2Noise network.

In this study, we implemented an unsupervised deep learning method, the Noise2Noise network, for the improvement of linear-array-based photoacoustic (PA) imaging. Unlike supervised learning, which requires a noise-free ground truth, the Noise2Noise network can learn noise patterns from a pair of noisy images. This is particularly important for in vivo PA imaging, where the ground truth is not available. In this study, we developed a method to generate noise pairs from a single set of PA images and verified our approach through simulation and experimental studies. Our results reveal that the method can effectively remove noise, improve signal-to-noise ratio, and enhance vascular structures at deeper depths. The denoised images show clear and detailed vascular structure at different depths, providing valuable insights for preclinical research and potential clinical applications.

Enhancing corrosion detection in pulsed eddy current testing systems through autoencoder-based unsupervised learning

Pulsed Eddy Current Testing (PECT) stands out as an advanced method in Non-Destructive Testing due to its extensive spectrum characteristics in comparison to traditional ECT techniques, making it exceptionally suitable for identifying corrosion. Nonetheless, the analysis of PECT signals for corrosion detection poses a challenge due to the transient nature of these signals and the impact of sensor lift-off effects. As a result, conventional methods are facing hurdles in dealing with corrosion signals of poor quality. In this study, the challenge is addressed by employing unsupervised learning methods utilizing an autoencoder neural network. This autoencoder integrates Long Short-Term Memory and 1D convolutional layers, acquiring the underlying features of normal PECT signals from non-corrosive regions. Significantly, the model is trained exclusively on this normal data, thereby obviating the necessity for pre-existing corrosion information. Through learning the inherent structure of normal signals, the model can detect anomalies in unseen data, potentially indicating corrosion. The unsupervised framework presents several advantages, such as reducing reliance on prior corrosion knowledge, mitigating inherent noise, and addressing sensor lift-off effects. Experimental results were conducted to compare with traditional methods like the lift-off of intersection and lift-off compensation methods. This approach resulted in a significant improvement in SNR, ranging from 100 % to 200 %, thus facilitating more robust NDT applications employing smart PECT sensors empowered by unsupervised learning techniques.

Joint inference of discrete cell types and continuous type-specific variability in single-cell datasets with MMIDAS.

Reproducible definition and identification of cell types is essential to enable investigations into their biological function, and understanding their relevance in the context of development, disease and evolution. Current approaches model variability in data as continuous latent factors, followed by clustering as a separate step, or immediately apply clustering on the data. We show that such approaches can suffer from qualitative mistakes in identifying cell types robustly, particularly when the number of such cell types is in the hundreds or even thousands. Here, we propose an unsupervised method, MMIDAS, which combines a generalized mixture model with a multi-armed deep neural network, to jointly infer the discrete type and continuous type-specific variability. Using four recent datasets of brain cells spanning different technologies, species, and conditions, we demonstrate that MMIDAS can identify reproducible cell types and infer cell type-dependent continuous variability in both uni-modal and multi-modal datasets.

Graph-Collaborated Auto-Encoder Hashing for Multiview Binary Clustering.

Unsupervised hashing methods have attracted widespread attention with the explosive growth of large-scale data, which can greatly reduce storage and computation by learning compact binary codes. Existing unsupervised hashing methods attempt to exploit the valuable information from samples, which fails to take the local geometric structure of unlabeled samples into consideration. Moreover, hashing based on auto-encoders aims to minimize the reconstruction loss between the input data and binary codes, which ignores the potential consistency and complementarity of multiple sources data. To address the above issues, we propose a hashing algorithm based on auto-encoders for multiview binary clustering, which dynamically learns affinity graphs with low-rank constraints and adopts collaboratively learning between auto-encoders and affinity graphs to learn a unified binary code, called graph-collaborated auto-encoder (GCAE) hashing for multiview binary clustering. Specifically, we propose a multiview affinity graphs' learning model with low-rank constraint, which can mine the underlying geometric information from multiview data. Then, we design an encoder-decoder paradigm to collaborate the multiple affinity graphs, which can learn a unified binary code effectively. Notably, we impose the decorrelation and code balance constraints on binary codes to reduce the quantization errors. Finally, we use an alternating iterative optimization scheme to obtain the multiview clustering results. Extensive experimental results on five public datasets are provided to reveal the effectiveness of the algorithm and its superior performance over other state-of-the-art alternatives.

Topology of Pulsar Profiles (ToPP)

Some of the most important information on a radio pulsar is derived from its average pulse profile. Many early pulsar studies were necessarily based on only a few such profiles. In these studies, discrete profile components were linked to emission mechanism models for individual stars through human interpretation. For the population as a whole, profile morphology must reflect the geometry and overall evolution of the radio emitting regions. The problem, however, is that this population is becoming too large for individual intensive studies of each source. Moreover, connecting profiles from a large collection of pulsars rapidly becomes cumbersome. In this article, we present ToPP, the first-ever unsupervised method to sort pulsars by profile-shape similarity using graph topology. We applied ToPP to the publicly available European Pulsar Network profile database, providing the first organised visual overview of multi-frequency profiles representing 90 individual pulsars. We found discrete evolutionary tracks varying from simple single-component profiles at all frequencies towards diverse mixtures of more complex profiles with frequency evolution. The profile evolution is continuous, extending out to millisecond pulsars, and does not fall into sharp classes. We interpret the profiles as being a mixture of pulsar core-cone emission type, spin-down energetics, and the line-of-sight impact angle towards the magnetic axis. We show how ToPP can systematically classify sources into the Rankin empirical profile scheme. ToPP comprises one of the key unsupervised methods that will be essential to exploring upcoming pulsar census data, such as the data expected from the Square Kilometer Array.

Exploring transfer learning for improving ultrasonic guided wave-based damage localization

Designing maintenance strategies to reduce the failure risk of plated structures is paramount for increasing the safety level of aerospace, civil and mechanical systems. Although traditional time-scheduled maintenance policies are reliable, they come with costly operations and avoidable downtimes. Recently, more complex condition-based strategies have been studied in the literature. This class of maintenance actions rely on structural health monitoring (SHM) frameworks: a sensor network is installed on the structure diagnostic data are processed to monitor the health state of the structure. The high dimensionality of data and the limitations of model-based SHM algorithms have led researchers to investigate data-driven solutions for improving the reliability of condition-based strategies. So far, supervised machine learning strategies have mainly been considered. However, since the cost of generating labeled datasets usually turns out to be prohibitive, two alternative solutions have gained attention: unsupervised methods and transfer learning (TL). While the former approach has been proved to provide satisfactory damage detection performance, it requires external knowledge sources to also localize and quantify damage. Instead, transfer learning could be used for performing all the damage diagnosis tasks, without the need for coupling the data-driven method with complex algorithms to restore the information lost by using smaller datasets for training. TL allows adapting pre-trained ML tools to new situations, new tasks and new environments. Moreover, TL can be leveraged when few labeled data are available, or to adapt efficient tools that have already been trained on a slightly different task. In this work, TL and convolutional neural networks (CNNs) were leveraged for performing damage localization and quantification in metal and composite plated structures. That is, an in-house CNN-based framework for localizing and quantifying structural damage was considered, and the fine-tuning TL technique was employed to make the framework flexible enough to work in different domains.

Subspace learning for feature selection via rank revealing QR factorization: Fast feature selection

The identification of informative and distinguishing features from high-dimensional data has gained significant attention in the field of machine learning. Recently, there has been growing interest in employing matrix factorization-based techniques, such as non-negative matrix factorization, for feature selection. The primary objective of feature selection using matrix factorization is to extract a lower-dimensional subspace that captures the essence of the original space. This study introduces a novel unsupervised feature selection technique that leverages rank revealing QR (RRQR) factorization. Compared to singular value decomposition (SVD) and non-negative matrix factorization (NMF), RRQR is more computationally efficient. The uniqueness of this technique lies in the utilization of the permutation matrix of QR for feature selection. Additionally, we integrate QR factorization into the objective function of NMF to create a new unsupervised feature selection method. Furthermore, we propose a hybrid feature selection algorithm by combining RRQR and a Genetic algorithm. The algorithm eliminates redundant features using RRQR factorization and selects the most distinguishing subset of features using the Genetic algorithm. Experimental comparisons with state-of-the-art feature selection algorithms in supervised, unsupervised, and semi-supervised settings demonstrate the reliability and robustness of the proposed algorithm. The evaluation is conducted on eight microarray datasets using KNN, SVM, and C4.5 classifiers. The experimental results indicate that the proposed method achieves comparable performance to the state-of-the-art feature selection methods. Our empirical findings demonstrate that the proposed method exhibits a significantly lower computational cost compared to other techniques.

Machine learning‐assisted anomaly detection for power line components: A case study in Pakistan

AbstractA continuous supply of electricity is necessary to maintain an acceptable standard of life, and the power distribution system's overhead line components play a crucial role in this matter. In Pakistan, identifying defective parts often necessitates human involvement. An unmanned aerial vehicle was used to gather a collection of 10,343 photos to automate this procedure. Using supervised and unsupervised machine learning methods, a number of automated anomaly detection systems were created. Support vector machine, random forest, VGG16, and ResNet50 were used as supervised machine learning models, and a convolutional auto‐encoder was used as the unsupervised machine learning model. VGG16 achieved the best accuracy of 99.00% while random forest achieved the worst accuracy of 72.49%. The convolutional auto‐encoder was successful in distinguishing between normal and abnormal components. The aforementioned machine learning models can be put on unmanned aerial vehicles to immediately identify defective parts.

Characteristics of the immune environment in prostate cancer as an adjunct to immunotherapy.

The tumor microenvironment (TME) exerts an important role in carcinogenesis and progression. Several investigations have suggested that immune cell infiltration (ICI) is of high prognostic importance for tumor progression and patient survival in many tumors, particularly prostate cancer. The pattern of immune infiltration of PCa, on the other hand, has not been thoroughly understood. The Cancer Genome Atlas (TCGA) and the Gene Expression Omnibus (GEO) datasets on PCa were obtained, and several datasets were merged into one data set using the "ComBat" algorithm. The ICI profiles of PCa patients were then to be uncovered by two computer techniques. The unsupervised clustering method was utilized to identify three ICI patterns in tumor samples, and Principal Component Analysis (PCA) was conducted to estimate the ICI score. Three different clusters of three ICIs were identified in 1341 PCa samples, which also correlated with different clinical features/characteristics and biological pathways. Patients with PCa are classified into high and low subtypes based on the ICI scores extracted from immune-associated signature genes. High ICI score subtypes are associated with a worse prognosis, which may intrigue the activation of cancer-related and immune-related pathways such as pathways involving Toll-like receptors, T-cell receptors, JAK-STAT, and natural killer cells. The ICI score was linked to tumor mutation load and immune/cancer-relevant signaling pathways, which explain prostate cancer's poor prognosis. The findings of this study not only advanced our knowledge of the mechanism of immune response in the prostate tumor microenvironment but also provided a novel biomarker, that is, the ICI score, for disease prognosis and guiding precision immunotherapy.

An unsupervised damage detection method for an operating wind turbine blade

Wind turbine blades are normally exposed to significant dynamic loads over their lifetime, including extreme conditions. This can lead to several types of damages such as cracking and delamination. Monitoring the structural condition of the blades to detect damages before they become catastrophic can be important. In recent years, the concept of data-driven structural health monitoring (SHM) approaches for wind turbine blade damage detection has become popular. These methods employ features collected from time series obtained from monitored wind turbine blades to identify damages. However, selecting an optimum number of features, which can reduce computational cost and minimize the effects of environmental and operational conditions, is still a challenge. In this paper, a data-driven approach is developed combining a feature extraction strategy and un-supervised anomaly detection. Monitoring data collected from several accelerometers mounted on a blade of an operating Vestas V27 wind turbine in healthy and damaged scenarios are used in this regard. Several time-domain, temporal, and time-frequency domain features are employed for the purpose of anomaly detection using unsupervised learning methods. Isolation Forest and One-class Support Vector Machine are used to train for anomaly detection. A threshold of detection is established to distinguish between the healthy and damaged states. Results show that the proposed approach can successfully identify the damage cases, but environmental and operational conditions still have some effect when it comes to identifying the extent of the damage.