Unsupervised Data Research Articles

Exploring Diversity and Abundance of Stingless Bee using Clustering Approach

Stingless bees are paramount in food chain as they are important pollinators of field crops. Recent studies revealed that these bees are seriously threatened by climate change and rapid urbanization across the world. It is thus important to study the relationship between the stingless bee’s diversity and the characteristics of the locations they inhibit. At the same time, clustering algorithms is a powerful machine learning approach in exploring unsupervised data. Consequently, this study aims to explore the stingless bee diversity in Malaysia through hierarchical, k-means and DBSCAN clustering. The dataset of this study consists of individual stingless bees collected from 12 locations. It comprises 14 environmental features, 3 physical characteristics, 35 species count, 12 genera counts and 3 diversity-and-abundance weights. A four-stage methodology is employed in the study. The results show that DBSCAN effectively groups data into clusters that are well-defined, but the results are less informative. In contrast, hierarchical and k-means clustering are found producing results that provide clearer insights, with hierarchical clustering delivering notably richer results.

Unsupervised 3D seismic data reconstruction using a weighted-attentive deep-learning framework

The physical and cost limitations of seismic data acquisition often result in the spatial undersampling of data, which has a detrimental impact on subsequent data processing. Seismic data reconstruction plays a crucial role in recovering missing traces arising from the acquisition process. In recent years, deep learning (DL) has emerged as an intelligent solution for this purpose. We introduce an innovative approach called the weighted-attentive DL framework for unsupervised 3D seismic data reconstruction, which combines an attentive transformer network (ATNet) with a conventional projection onto convex sets (POCS) algorithm. Our framework follows the plug-and-play concept, requiring only the original subsampled data to achieve robust performance. It accomplishes this by iteratively updating ATNet parameters, wherein ATNet serves as the reconstruction operator during the DL process. This approach allows us to simultaneously recover missing traces and filter out random noise, with a key feature being its enhanced attention to the observed signal compared with convolutional neural networks. In addition, we incorporate the POCS algorithm into our framework to introduce a linear decay weight for the aliasing space containing noise and signal during the reconstruction process. We rigorously evaluate our method against four existing approaches: adaptive POCS, fast dictionary learning sequential generalized K-means, optimally damped rank-reduction, and DenseNet methods. Our comparative experiments, conducted on synthetic and field data examples, clearly demonstrate the substantial improvements achieved by our method in terms of reconstruction and denoising when compared with the four benchmarked methods.

Single-cell landscape of innate and acquired drug resistance in acute myeloid leukemia

Deep single-cell multi-omic profiling offers a promising approach to understand and overcome drug resistance in relapsed or refractory (rr) acute myeloid leukemia (AML). Here, we combine single-cell ex vivo drug profiling (pharmacoscopy) with single-cell and bulk DNA, RNA, and protein analyses, alongside clinical data from 21 rrAML patients. Unsupervised data integration reveals reduced ex vivo response to the Bcl-2 inhibitor venetoclax (VEN) in patients treated with both a hypomethylating agent (HMA) and VEN, compared to those pre-exposed to chemotherapy or HMA alone. Integrative analysis identifies both known and unreported mechanisms of innate and treatment-related VEN resistance and suggests alternative treatments, like targeting increased proliferation with the PLK inhibitor volasertib. Additionally, high CD36 expression in VEN-resistant blasts associates with sensitivity to CD36-targeted antibody treatment ex vivo. This study demonstrates how single-cell multi-omic profiling can uncover drug resistance mechanisms and treatment vulnerabilities, providing a valuable resource for future AML research.

Unsupervised data driven approaches to Raman imaging through a multimode optical fiber

Raman spectroscopy is a label-free, chemically specific optical technique that provides detailed information about the chemical composition and structure of the excited analyte. Because of this, there is growing research interest in miniaturizing Raman probes to reach deep regions of the body. Typically, such probes utilize multiple optical fibers to act as separate excitation/collection channels with optical filters attached to the distal facet to separate the collected signal from the background optical signal from the probe itself. Although these probes have achieved impressive diagnostic performance, their use is limited by the overall size of the probe, which is typically several hundred micrometers to millimeters. Here, we show how a wavefront shaping technique can be used to measure Raman images through a single, hair-thin multimode fiber. The wavefront shaping technique transforms the tip of the fiber to a micrometer spatial resolution Raman microscope. The resultant Raman images were analyzed with a variety of state-of-the-art statistical techniques, including principal component analysis (PCA), t-distributed stochastic neighbor embedding (t-SNE), uniform manifold approximation and projection (UMAP), and k-means clustering. Our data-driven approach enables us to create Raman images of microclusters of pharmaceuticals using fingerprint region Raman spectra through a standard silica multimode optical fiber.

Multi‐colour recipe prediction through invertible neural network and clustering algorithm

AbstractComputer‐assisted colour prediction has become increasingly significant for various consumer items in the industrial sector. Many professional colourists find it challenging to develop a suitable colour recipe. The most difficult aspect is predicting which dye mix will result in the required shade on a given fabric. This study introduces an advanced alternative to the traditional colour‐making method using an invertible neural network (INN) model. The INN model effectively addresses real‐world inverse problems due to its bi‐directional nature. In the forward phase, the model retrieves information about the colour recipe; in the backward phase, this information is combined with latent space data to predict the recipe. Furthermore, unsupervised data generated by the INN model is fed into clustering algorithms, such as K‐means and the Gaussian mixture model (GMM), to obtain multiple recipes. The forward procedure was reintroduced with a predicted recipe to assess the efficacy of the proposed model. An analysis was then conducted on the colour differences between the anticipated and actual recipes. The colour differences, rounded to perceptually significant precision, from 30,000 samples with 50 centre points, are as follows: 1.4, 2.2, 2.5, 3.7, 1.2, and 2.1. These results indicate that the INN model and the GMM clustering approach together provide a highly accurate and efficient solution to automating the colour‐matching process, offering a more precise and practical solution for the colour‐manufacturing industry.

ASCL: Accelerating semi‐supervised learning via contrastive learning

SummarySSL (semi‐supervised learning) is widely used in machine learning, which leverages labeled and unlabeled data to improve model performance. SSL aims to optimize class mutual information, but noisy pseudo‐labels introduce false class information due to the scarcity of labels. Therefore, these algorithms often need significant training time to refine pseudo‐labels for performance improvement iteratively. To tackle this challenge, we propose a novel plug‐and‐play method named Accelerating semi‐supervised learning via contrastive learning (ASCL). This method combines contrastive learning with uncertainty‐based selection for performance improvement and accelerates the convergence of SSL algorithms. Contrastive learning initially emphasizes the mutual information between samples as a means to decrease dependence on pseudo‐labels. Subsequently, it gradually turns to maximizing the mutual information between classes, aligning with the objective of semi‐supervised learning. Uncertainty‐based selection provides a robust mechanism for acquiring pseudo‐labels. The combination of the contrastive learning module and the uncertainty‐based selection module forms a virtuous cycle to improve the performance of the proposed model. Extensive experiments demonstrate that ASCL outperforms state‐of‐the‐art methods in terms of both convergence efficiency and performance. In the experimental scenario where only one label is assigned per class in the CIFAR‐10 dataset, the application of ASCL to Pseudo‐label, UDA (unsupervised data augmentation for consistency training), and Fixmatch benefits substantial improvements in classification accuracy. Specifically, the results demonstrate notable improvements in respect of 16.32%, 6.9%, and 24.43% when compared to the original outcomes. Moreover, the required training time is reduced by almost 50%.

Computer vision and deep transfer learning for automatic gauge reading detection

This manuscript proposes an automatic reading detection system for an analogue gauge using a combination of deep learning, machine learning, and image processing. The study suggests image-processing techniques in manual analogue gauge reading that include generating readings for the image to provide supervised data to address difficulties in unsupervised data in gauges and to achieve better accuracy using DenseNet 169 compared to other approaches. The model uses artificial intelligence to automate reading detection using deep transfer learning models like DenseNet 169, InceptionNet V3, and VGG19. The models were trained using 1011 labeled pictures, 9 classes, and readings from 0 to 8. The VGG19 model exhibits a high training precision of 97.00% but a comparatively lower testing precision of 75.00%, indicating the possibility of overfitting. On the other hand, InceptionNet V3 demonstrates consistent precision across both datasets, but DenseNet 169 surpasses other models in terms of precision and generalization capabilities.

Robust unsupervised 5D seismic data reconstruction on regular and irregular grids

Seismic data reconstruction has become a central focus in seismic data processing, addressing challenges posed by sparse sampling due to physical and budgetary constraints. The advent of 5D acquisition methodologies marks a significant advancement in the quality and completeness of seismic data sets. Most traditional 5D reconstruction methods commonly use the fast Fourier transform (FFT), requiring regular grids and preliminary 4D binning before 5D interpolation. Discrete Fourier transform and nonequidistant FFT can honor the original irregular coordinates. However, when using exact locations, these methods become computationally expensive. We introduce an unsupervised deep-learning methodology to learn a continuous function across the sampling points in seismic data, facilitating reconstruction on regular and irregular grids. The network comprises a multilayer perceptron with linear layers and element-wise periodic activation functions. It excels at mapping the input coordinates to the corresponding seismic data amplitudes without relying on external training sets. The network’s intrinsic low-frequency bias is crucial in prioritizing acquiring self-similar features over high-frequency and incoherent ones during training. This characteristic mitigates incoherent noise in seismic data, such as random and erratic components. To assess the robustness of the unsupervised reconstruction technique, we conduct comprehensive evaluations using synthetic data examples sampled regularly and irregularly, as well as field-data examples with and without binning. The findings demonstrate the efficacy of our deep-learning framework in achieving resilient and accurate seismic data reconstruction across diverse sampling scenarios.

Shedding light on the DICER1 mutational spectrum of uncertain significance in malignant neoplasms.

The Dicer protein is an indispensable player in such fundamental cell pathways as miRNA biogenesis and regulation of protein expression in a cell. Most recently, both germline and somatic mutations in DICER1 have been identified in diverse types of cancers, which suggests Dicer mutations can lead to cancer progression. In addition to well-known hotspot mutations in RNAase III domains, DICER1 is characterized by a wide spectrum of variants in all the functional domains; most are of uncertain significance and unstated clinical effects. Moreover, various new somatic DICER1 mutations continuously appear in cancer genome sequencing. The latest contemporary methods of variant effect prediction utilize machine learning algorithms on bulk data, yielding suboptimal correlation with biological data. Consequently, such analysis should be conducted based on the functional and structural characteristics of each protein, using a well-grounded targeted dataset rather than relying on large amounts of unsupervised data. Domains are the functional and evolutionary units of a protein; the analysis of the whole protein should be based on separate and independent examinations of each domain by their evolutionary reconstruction. Dicer represents a hallmark example of a multidomain protein, and we confirmed the phylogenetic multidomain approach being beneficial for the clinical effect prediction of Dicer variants. Because Dicer was suggested to have a putative role in hematological malignancies, we examined variants of DICER1 occurring outside the well-known hotspots of the RNase III domain in this type of cancer using phylogenetic reconstruction of individual domain history. Examined substitutions might disrupt the Dicer function, which was demonstrated by molecular dynamic simulation, where distinct structural alterations were observed for each mutation. Our approach can be utilized to study other multidomain proteins and to improve clinical effect evaluation.

Establishment of Agro-Eco Industrial Clusters: A Romanian Perspective

The Romanian economy feels the need to formalize links between the situational economic forecast and the forecast of technological events in industry and agriculture. The article studies a “dimension” of the organization for classifications of data, structures and economic / agro-eco-industrial events for clustering with the help of “dimensionality”. Meta-prognostic relationships between economics, forecasting and forecasting agents (mathematical elements) are described and the role of metricity, sub-metricity and ultra-metricity in the observation of an automatic ontology for classifications supporting agro-eco-industrial / food clustering is highlighted. The paper proposes the algorithm of the informal classes of supervised / unsupervised data about agro-eco-industrial / food companies in Romania in order to establish classifications that would motivate the formation of clusters in the field. It is concluded that the mathematical apparatus for classification must be developed because based on symbolic mathematical models can be developed computer / computer programs for calculations of ultra-metricity of transformations for linearization’s of similarities between firms, eliminating reminiscences / redundancies, aiming to reduce space / distances between data that represent, in fact, enterprises that can enter agro-eco-industrial / food clusters. Keywords: agro-eco-industrial clusters, ultra-metricity, clustering, economic forecasting, classification

An efficient and resilience linear prefix approach for mining maximal frequent itemset using clustering

The numerous volumes of data generated every day necessitate the deployment of new technologies capable of dealing with massive amounts of data efficiently. This is the case with Association Rules, a tool for unsupervised data mining that extracts information in the form of IF-THEN patterns. Although various approaches for extracting frequent itemset (prior step before mining association rules) in extremely large databases have been presented, the high computational cost and shortage of memory remain key issues to be addressed while processing enormous data. The objective of this research is to discover frequent itemset by using clustering for preprocessing and adopting the linear prefix tree algorithm for mining the maximal frequent itemset. The performance of the proposed CL-LP-MAX-tree was evaluated by comparing it with the existing FP-max algorithm. Experimentation was performed with the three different standard datasets to record evidence to prove that the proposed CL-LP-MAX-tree algorithms outperform the existing FP-max algorithm in terms of runtime and memory consumption.

Exploring other clustering methods and the role of Shannon Entropy in an unsupervised setting

In the ever-expanding landscape of digital technologies, the exponential growth of data in information science and health informatics presents both challenges and opportunities, demanding innovative approaches to data curation. This study focuses on evaluating various feasible clustering methods within the Data Washing Machine (DWM), a novel tool designed to streamline unsupervised data curation processes. The DWM integrates Shannon Entropy into its clustering process, allowing for adaptive refinement of clustering strategies based on entropy levels observed within data clusters. Rigorous testing of the DWM prototype on various annotated test samples revealed promising outcomes, particularly in scenarios with high-quality data. However, challenges arose when dealing with poor data quality, emphasizing the importance of data quality assessment and improvement for successful data curation. To enhance the DWM’s clustering capabilities, this study explored alternative unsupervised clustering methods, including spectral clustering, autoencoders, and density-based clustering like DBSCAN. The integration of these alternative methods aimed to augment the DWM’s ability to handle diverse data scenarios effectively. The findings demonstrated the practicability of constructing an unsupervised entity resolution engine with the DWM, highlighting the critical role of Shannon Entropy in enhancing unsupervised clustering methods for effective data curation. This study underscores the necessity of innovative clustering strategies and robust data quality assessments in navigating the complexities of modern data landscapes. This content is structured by the following sections: Introduction, Methodology, Results, Discussion, and Conclusion.

Application of Unsupervised Multi-Omic Factor Analysis to Uncover Patterns of Variation and Molecular Processes Linked to Cardiovascular Disease.

Disease mechanisms are usually complex and governed by the interaction of several distinct molecular processes. Complex, multidimensional datasets are a valuable resource to generate more insights into those processes, but the analysis of such datasets can be challenging due to the high dimensionality resulting, for example, from different disease conditions, timepoints, and omics capturing the process at different resolutions. Here, we showcase an approach to analyze and explore such a complex multiomics dataset in an unsupervised way by applying multi-omics factor analysis (MOFA) to a dataset generated from blood samples that capture the immune response in acute and chronic coronary syndromes. The dataset consists of several assays at differing resolutions, including sample-level cytokine data, plasma-proteomics and neutrophil prime-seq, and single-cell RNA-seq (scRNA-seq)data. Further complexity is added by having several different time points measured per patient and several patient subgroups. The analysis workflow outlines how to integrate and analyze the data in several steps: (1) Data pre-processing and harmonization, (2) Estimation of the MOFA model, (3) Downstream analysis. Step 1 outlines how to process the features of the different data types, filter out low-quality features, and normalize them to harmonize their distributions for further analysis. Step 2 shows how to apply the MOFA model and explore the major sources of variance within the dataset across all omics and features. Step 3 presents several strategies for the downstream analysis of the captured patterns, linking them to the disease conditions and potential molecular processes governing those conditions. Overall, we present a workflow for unsupervised data exploration of complex multi-omics datasets to enable the identification of major axes of variation composed of differing molecular features that can also be applied to other contexts and multi-omics datasets (including other assays as presented in the exemplary use case).

Pre-Reconstruction Processing with the Cycle-Consist Generative Adversarial Network Combined with Attention Gate to Improve Image Quality in Digital Breast Tomosynthesis.

The current study proposed and evaluated "residual squeeze and excitation attention gate" (rSEAG), a novel network that can improve image quality by reducing distortion attributed to artifacts. This method was established by modifying the Cycle Generative Adversarial Network (cycleGAN)-based generator network using projection data for pre-reconstruction processing in digital breast tomosynthesis. Residual squeeze and excitation were installed in the bridge of the generator network, and the attention gate was installed in the skip connection between the encoder and decoder. Based on the radiation dose index (exposure index and division index) incident on the detector, the cases approved by the ethics committee and used for the study were classified as reference (675 projection images) and object (675 projection images). For the cases, unsupervised data containing a mixture of cases with and without masses were used. The cases were trained using cycleGAN with rSEAG and the conventional networks (ResUNet and U-Net). For testing, predictive processing was performed on cases (60 projection images) that were not used for learning. Images were generated using filtered backprojection reconstruction (kernel: Ramachandran and Lakshminarayanan) from projection data for testing data and without pre-reconstruction processing data (evaluation: in-focus plane). The distortion was evaluated using perception-based image quality evaluation (PIQE) analysis, texture analysis (feature: "Homogeneity" and "Contrast"), and a statistical model with a Gumbel distribution. PIQE has a low rSEAG value. Texture analysis showed that rSEAG and a network without cycleGAN were similar in terms of the "Contrast" feature. In dense breasts, ResUNet had the lowest "Contrast" feature and U-Net had differences between cases. The maximal variations in the Gumbel plot, rSEAG reduced the high-frequency ripple artifacts. In this study, rSEAG could improve distortion and reduce ripple artifacts.

Enhancing training performance of convolutional neural network algorithm through an autoencoder-based unsupervised labeling framework for mineral exploration targeting

Convolutional neural networks (CNNs), a prominent deep learning approach, have garnered significant interest in the field of mineral potential mapping (MPM) due to their capability to capture and learn spatial features that traditional algorithms tend to overlook. The effectiveness of CNNs is closely tied to the quantity of training data available, thereby impacting the outcomes of MPM. Moreover, uncertainties arising from delineation of negative samples can compromise the reliability of MPM assessments. To deal with these challenges, we propose the utilization of an autoencoder-based anomaly detection technique for the purpose of annotating locations in an unsupervised manner. Subsequently, a CNN is trained using the unsupervised annotated data to generate a Fe prospectivity model within a specific region in Iran. To validate the effectiveness of the proposed method, we first perform an MPM with insufficient positive training samples that extract from the location of known occurrences. We then execute another MPM on a set of samples labeled based on the reconstruction error of an autoencoder network. When comparing the two prospectivity models, namely using augmented data or inadequate samples, it is evident that modeling with augmented data outperforms the MPM model trained with insufficient samples. This confirms the effectiveness of the adopted approach and shows that the unsupervised labeling technique proposed in this work can significantly improve the performance of the CNN in MPM.

Comparing human and machine speech recognition in noise with QuickSIN.

A test is proposed to characterize the performance of speech recognition systems. The QuickSIN test is used by audiologists to measure the ability of humans to recognize continuous speech in noise. This test yields the signal-to-noise ratio at which individuals can correctly recognize 50% of the keywords in low-context sentences. It is argued that a metric for automatic speech recognizers will ground the performance of automatic speech-in-noise recognizers to human abilities. Here, it is demonstrated that the performance of modern recognizers, built using millions of hours of unsupervised training data, is anywhere from normal to mildly impaired in noise compared to human participants.

Unsupervised meta-analysis on chemical elements and atomic energy prediction: A case study on the periodic table

This paper presents an artificial classification and atomic energy correlation analysis of the chemical components. The choice of data mining is due to its robustness, which can explore intrinsic or hidden relationships between chemical components and their properties. The Mendeleev table is conceivably the earliest example of the data analysis technique in materials science. However, the classical periodic table represents the arrangement of chemical elements based on particular periodicities, which has the issue of property progression for a few chemical components. In this investigation, we utilized one of the unsupervised data mining methods (principal component analysis) to explore knowledge from the chemical components database based on all the prepared properties. The main objective is to make an artificial classification of chemical components depending on their accessible physical and energetic properties. The results revealed the effectiveness of the data mining method in appreciating the relationships between the variables and properties that offer a new approach to seeing a Mendeleev table. The final step of this work highlights the significance of predictive polynomials that permit the scientific community to make atomic total energy predictions for each chemical component, from helium to lawrencium.

Physics and chemistry from parsimonious representations: image analysis via invariant variational autoencoders

Electron, optical, and scanning probe microscopy methods are generating ever increasing volume of image data containing information on atomic and mesoscale structures and functionalities. This necessitates the development of the machine learning methods for discovery of physical and chemical phenomena from the data, such as manifestations of symmetry breaking phenomena in electron and scanning tunneling microscopy images, or variability of the nanoparticles. Variational autoencoders (VAEs) are emerging as a powerful paradigm for the unsupervised data analysis, allowing to disentangle the factors of variability and discover optimal parsimonious representation. Here, we summarize recent developments in VAEs, covering the basic principles and intuition behind the VAEs. The invariant VAEs are introduced as an approach to accommodate scale and translation invariances present in imaging data and separate known factors of variations from the ones to be discovered. We further describe the opportunities enabled by the control over VAE architecture, including conditional, semi-supervised, and joint VAEs. Several case studies of VAE applications for toy models and experimental datasets in Scanning Transmission Electron Microscopy are discussed, emphasizing the deep connection between VAE and basic physical principles. Python codes and datasets discussed in this article are available at https://github.com/saimani5/VAE-tutorials and can be used by researchers as an application guide when applying these to their own datasets.

Health State Prediction Method Based on Multi-Featured Parameter Information Fusion

The prediction of the health status of critical components is an important influence in making accurate maintenance decisions for rotating equipment. Since vibration signals contain a large amount of fault information, they can more accurately describe the health status of critical components. Therefore, it is widely used in the field of rotating equipment health state prediction. However, there are two major problems in predicting the health status of key components based on vibration signals: (1) The working environment of rotating equipment is harsh, and if only one feature in the time or frequency domain is selected for fault analysis, it will be susceptible to harsh operating environments and cannot completely reflect the fault information. (2) The vibration signals are unlabeled time series data, which are difficult to accurately convert into the health status of key components. In order to solve the above problems, this paper proposes a combined prediction model combining a bidirectional long- and short-term memory network (BiLSTM), a self-organizing neural network (SOM) and particle swarm optimization (PSO). Firstly, the SOM is utilized to fuse the fault characteristics of multiple vibration signals of key components to obtain an indicator (HI) that can reflect the health status of rotating equipment and to also compensate for the vulnerability of single signal characteristics in the time or frequency domain to environmental influences. Secondly, the K-means clustering method is employed to cluster the health indicators and determine the health state, which solves the problem of determining the health of a component from unsupervised vibration signal data which is quite difficult. Finally, the particle swarm optimized BiLSTM model is used to predict the health state of key components and the bearing dataset from the IEEE PHM 2012 Data Challenge verifies the method’s effectiveness and validity.

MapReduce algorithms for robust center-based clustering in doubling metrics

Clustering is a pivotal primitive for unsupervised learning and data analysis. A popular variant is the (k,ℓ)-clustering problem, where, given a pointset P from a metric space, one must determine a subset S of k centers minimizing the sum of the ℓ-th powers of the distances of points in P from their closest centers. This formulation covers the well-studied k-median (ℓ=1) and k-means (ℓ=2) clustering problems. A more general variant, introduced to deal with noisy pointsets, features a further parameter z and allows up to z points of P (outliers) to be disregarded when computing the sum. We present a distributed coreset-based 3-round approximation algorithm for the (k,ℓ)-clustering problem with z outliers, using MapReduce as a computational model. An important feature of our algorithm is that it obliviously adapts to the intrinsic complexity of the dataset, captured by its doubling dimension D. Remarkably, for D=O(1), our algorithm requires sublinear local memory per reducer, and yields a solution whose approximation ratio is an additive term O(γ) away from the one achievable by the best known sequential (possibly bicriteria) algorithm, where γ can be made arbitrarily small. To the best of our knowledge, no previous distributed approaches were able to attain similar quality-performance tradeoffs for metrics with constant doubling dimension.