Unsupervised Learning Research Articles

Unsupervised approach to text line extraction in Belfort civil registers of births

Unsupervised approach to text line extraction in Belfort civil registers of births

Improved survival prediction for kidney transplant outcomes using artificial intelligence-based models: development of the UK Deceased Donor Kidney Transplant Outcome Prediction (UK-DTOP) Tool

The UK Deceased Donor Kidney Transplant Outcome Prediction (UK-DTOP) Tool, developed using advanced artificial intelligence (AI), significantly enhances the prediction of outcomes for deceased-donor kidney transplants in the UK. This study analyzed data from the UK Transplant Registry (UKTR), including 29,713 transplant cases between 2008 and 2022, to assess the predictive performance of three machine learning models: XGBoost, Random Survival Forest, and Optimal Decision Tree. Among these, XGBoost demonstrated exceptional performance with the highest concordance index of 0.74 and an area under the curve (AUC) consistently above 0.73, indicating robust discriminative ability and calibration. In comparison to the traditional Kidney Donor Risk Index (KDRI), which achieved a lower concordance index of 0.57, the UK-DTOP model marked a significant improvement, underscoring its superior predictive accuracy. The advanced capabilities of the XGBoost model were further highlighted through calibration assessments using the Integrated Brier Score (IBS), showing a score of 0.14, indicative of precise survival probability predictions. Additionally, unsupervised learning via k-means clustering was employed to identify five distinct clusters based on donor and transplant characteristics, uncovering nuanced insights into graft survival outcomes. These clusters were further analyzed using Bayesian Cox regression, which confirmed significant survival outcome variations across the clusters, thereby validating the model’s effectiveness in enhancing risk stratification. The UK-DTOP tool offers a comprehensive decision-support system that significantly refines pre-transplant decision-making. The study’s findings advocate for the adoption of AI-enhanced tools in healthcare systems to optimize organ matching and transplant success, potentially guiding future developments in global transplant practices.

Sustainability Assessment of Brown Trout Populations in Serbia (Central Balkans) Using the Modified ESHIPPO Model

In the inland waters of the Balkans, many brown trout populations have been severely depleted. Therefore, identifying potential threats to their continued survival and ranking populations based on their biological and evolutionary importance enables setting conservation priorities. To assess the sustainability of the brown trout populations in the territory of Serbia (central Balkans), a modification of the ESHIPPO model was performed. The main modification involves incorporating the investigated populations’ genetic structure into the model. Therefore, the new ESHIPPOsalmo model includes an analysis of biological parameters and the impact of multiple factors, including habitat alterations, invasive species, pollution, human population growth, and over-exploitation. In order to investigate individual levels of influence of the model’s analyzed parameters, a combination of supervised and unsupervised machine learning methods was used. The structure of the model is based on general and easily measurable indicators, which enables its application in any salmonid river in the world. By evaluating the parameters of the ESHIPPOsalmo model, we were able to establish that, of the analyzed populations from 46 localities, 37% have a moderate level of sustainability, 43% low, and 20% critically low.

Vegetative Index Intercalibration Between PlanetScope and Sentinel-2 Through a SkySat Classification in the Context of “Riserva San Massimo” Rice Farm in Northern Italy

Rice farming in Italy accounts for about 50% of the EU’s rice area and production. Precision agriculture has entered the scene to enhance sustainability, cut pollution, and ensure food security. Various studies have used remote sensing tools like satellites and drones for multispectral imaging. While Sentinel-2 is highly regarded for precision agriculture, it falls short for specific applications, like at the “Riserva San Massimo” (Gropello Cairoli, Lombardia, Northern Italy) rice farm, where irregularly shaped crops need higher resolution and frequent revisits to deal with cloud cover. A prior study that compared Sentinel-2 and the higher-resolution PlanetScope constellation for vegetative indices found a seasonal miscalibration in the Normalized Difference Vegetation Index (NDVI) and in the Normalized Difference Red Edge Index (NDRE). Dr. Agr. G.N. Rognoni, a seasoned agronomist working with this farm, stresses the importance of studying the radiometric intercalibration between the PlanetScope and Sentinel-2 vegetative indices to leverage the knowledge gained from Sentinel-2 for him to apply variable rate application (VRA). A high-resolution SkySat image, taken almost simultaneously with a pair of Sentinel-2 and PlanetScope images, offered a chance to examine if the irregular distribution of vegetation and barren land within rice fields might be a factor in the observed miscalibration. Using an unsupervised pixel-based image classification technique on SkySat imagery, it is feasible to split rice into two subclasses and intercalibrate them separately. The results indicated that combining histograms and agronomists’ expertise could confirm SkySat classification. Moreover, the uneven spatial distribution of rice does not affect the seasonal miscalibration object of past studies, which can be adjusted using the methods described here, even with images taken four days apart: the first method emphasizes accuracy using linear regression, histogram shifting, and histogram matching; whereas the second method is faster and utilizes only histogram matching.

Mozart: A Mobile ToF System for Sensing in the Dark Through Phase Manipulation

Sensing in low-light and dark environments has a wide range of applications. However, existing sensing technologies suffer several major challenges, such as excessive noise and low resolution. In this work, we propose Mozart - a new mobile sensing system that leverages off-the-shelf Time-of-Flight (ToF) depth cameras to generate high-resolution and rich-in-texture maps for applications in dark scenarios. The design of Mozart is based on our key observation that the phase components of ToF measurements can be manipulated to expose texture information. Through in-depth analysis of the physical reflection model, we show that the textures can be exposed and enhanced using highly compute-efficient phase manipulation functions. Moreover, by exploiting the physics texture models, we propose an autoencoderbased unsupervised learning approach that can automatically learn efficient representations from phase components to generate high-resolution maps. We implemented Mozart on several Android smartphone models and an edge testbed with standalone ToF camera platforms for various applications in the dark. The results show that Mozart can work in real time and delivers significant improvement over existing sensing technologies. The demo video at https:// www.youtube.com/watch?v=L_sxyTZxIdU shows that Mozart offers a low-cost, highperformance sensing technology for nextgeneration applications in the dark.

Unsupervised domain adaptation for drive-by condition monitoring of multiple railway tracks

Monitoring railway tracks through drive-by vibration data collected by in-service trains offers a cost-effective and adaptable solution for inspecting multiple railway lines. However, numerous existing drive-by monitoring methods rely on supervised learning models, necessitating extensive labelled data for each line. In this paper, a novel framework is proposed based on Unsupervised Domain Adaptation (UDA) concept which facilitates the transfer of a geometric defects diagnosis model learned from one line to a new line without the need for any labelled data from the new line. The proposed framework learns the dynamic-based features that are sensitive to damage and also invariant to different railway tracks. It comprises three components: data pre-processing, UDA implementation, and damage diagnosis. The framework uses the data from the source domain, including corresponding labels, as well as the unlabelled data from the target domain as input. The outputs of the framework consist of the predicted labels for the target domain. The performance of the proposed framework is evaluated using a comprehensive dataset of field measurements of a high-speed train passing 4 different lines within the French high-speed rail network. The proposed UDA framework is implemented using four common UDA algorithms including Information-Theoretical Learning (ITL), Geodesic Flow Kernel (GFK), Transfer Component Analysis (TCA), and Subspace Alignment (SA). The results show that the proposed framework has a 14% increase in the anomaly detection accuracy compared to traditional unsupervised learning methods in which UDA is not used. Furthermore, this study investigates the impact of incorporating a percentage of target data labels during training (semi-supervised domain adaptation), along with various sensor layouts and different tuning parameters, on the accuracy of the proposed approach. The results show that the proposed framework can significantly facilitate the monitoring of railway track conditions using the data collected by in-service trains which could be great interest of railway owners.

Improving out-of-sample forecasts of stock price indexes with forecast reconciliation and clustering

In this paper, we propose a novel approach to improving forecasts of stock market indexes by considering common stock prices as hierarchical time series, combining clustering with forecast reconciliation. We propose grouping the individual stock price series in various ways including via metadata and using unsupervised learning techniques. The proposed approach is applied to the Dow Jones Industrial Average Index and the Standard & Poor 500 Index and their component stocks, and the results obtained with different grouping approaches are compared. The results empirically demonstrate that the combined use of clustering and reconciliation improves the forecast accuracy of the stock market indexes and their constituents.

Unsupervised Clustering-based 3D Static Scene Construction Using LiDAR Channel and Azimuth Angle

Abstract. Cameras are typically used at road intersections to collect data to perform object detection but struggle in low-light and harsh weather. On the other hand, Light Detection and Ranging (LiDAR) is used as a key technology in 3D vision systems. It gives the 3D point cloud, which includes accurate depth information, but its resolution is cost-dependent, with higher resolutions being more expensive. Deep learning-based method requires large, labelled dataset which increases the cost, time and accuracy depending on the model trained on labelled dataset. To perform object detection in point cloud data is a challenging task due to the incomplete representations, data sparsity and unavailability of training data. To overcome this by using an unsupervised approach, it is important to identify the static scene and then detect the moving object. This work incorporated a novel approach to construct static scene using azimuth angle and laser channel information using an unsupervised clustering approach. This work incorporates two modules I) data collection using VLP-16 LiDAR, and II) static scene construction using DBSCAN (density-based spatial clustering and noise) clustering-based approach. Data is collected at a 4-legged intersection and pre-processed to extract aggregated distances corresponding to the unique pair of azimuth angle and laser channel. DBSCAN is used to perform clustering on the aggregated distances, based on the highest silhouette score and lowest intra distance between points in cluster, static points are identified, and static scene constructed. The qualitative evaluation of method demonstrates that the algorithm effectively and accurately filters out background points.

120 GOPS Photonic tensor core in thin-film lithium niobate for inference and in situ training

Photonics offers a transformative approach to artificial intelligence (AI) and neuromorphic computing by enabling low-latency, high-speed, and energy-efficient computations. However, conventional photonic tensor cores face significant challenges in constructing large-scale photonic neuromorphic networks. Here, we propose a fully integrated photonic tensor core, consisting of only two thin-film lithium niobate (TFLN) modulators, a III-V laser, and a charge-integration photoreceiver. Despite its simple architecture, it is capable of implementing an entire layer of a neural network with a computational speed of 120 GOPS, while also allowing flexible adjustment of the number of inputs (fan-in) and outputs (fan-out). Our tensor core supports rapid in-situ training with a weight update speed of 60 GHz. Furthermore, it successfully classifies (supervised learning) and clusters (unsupervised learning) 112 × 112-pixel images through in-situ training. To enable in-situ training for clustering AI tasks, we offer a solution for performing multiplications between two negative numbers.

Efficient training of energy-based models using Jarzynski equality**This article is an updated version of: Carbone D, Hua M, Coste S and Vanden-Eijnden E 2023 Efficient training of energy-based models using Jarzynski equality Advances in Neural Information Processing Systems vol 36, ed A Oh, T Naumann, A Globerson, K Saenko, M Hardt and S Levine (Curran Associates, Inc.) pp 52583–614.

Abstract Energy-based models (EBMs) are generative models inspired by statistical physics with a wide range of applications in unsupervised learning. Their performance is well measured by the cross-entropy (CE) of the model distribution relative to the data distribution. Using the CE as the objective for training is, however, challenging because the computation of its gradient with respect to the model parameters requires sampling of the model distribution. Here, we show how the results for nonequilibrium thermodynamics based on the Jarzynski equality together with tools from sequential Monte Carlo sampling can be used to perform this computation efficiently and avoid the uncontrolled approximations made using the standard contrastive divergence algorithm. Specifically, we introduce a modification of the unadjusted Langevin algorithm (ULA), in which each walker acquires a weight that enables the estimation of the gradient of the CE at any step during gradient descent, thereby bypassing sampling biases induced by slow mixing of the ULA. We illustrate these results with numerical experiments on Gaussian mixture distributions as well as the MNIST and CIFAR-10 datasets. We show that the proposed approach outperforms methods based on the contrastive divergence algorithm in all the considered situations.

Data-driven Machine Learning Models for Risk Stratification and Prediction of Emergence Delirium in Pediatric Patients Underwent Tonsillectomy/Adenotonsillectomy.

In the pediatric surgical population, Emergence Delirium (ED) poses a significant challenge. This study aims to develop and validate machine learning (ML) models to identify key features associated with ED and predict its occurrence in children undergoing tonsillectomy or adenotonsillectomy. The analysis involved data cleaning, exploratory data analysis (EDA), supervised predictive modeling, and unsupervised learning on a medical dataset (n = 423). After preliminary data cleaning, EDA encompassed plotting histograms, boxplots, pairplots, and correlation heatmaps to understand variable distributions and relationships. Four predictive models were trained including logistic regression (LR), random forest (RF), Support Vector Machine (SVM), and Gradient Boosting (XGBoost). The models were evaluated and compared using Receiver Operating Characteristic (ROC) Area Under the Curve (AUC), precision, recall, and feature importance. The RF model showed better performance and was used for the test (AUC-ROC 0.96, precision 1.00, and recall 0.92 on the validation set). K-means clustering was applied to find groups within the data. Elbow method and silhouette scores were used to determine the optimal number of clusters. The formed clusters were analyzed by aggregating features to understand the characteristics of each cluster. EDA revealed significant positive correlations between age, weight, American Society of Anesthesiologists (ASA) health score, and surgery duration with the risk of developing ED. Among the ML models, RF achieved the highest performance. Key predictive variables, based on the model's feature importance, included delirium screening scales, extubation time, and time to regain consciousness. Unsupervised K-means clustering identified 2-3 optimal clusters, which represented distinct patient subgroups: younger, healthier, low-risk individuals (cluster 0), and older patients with increasing chronic disease burden, higher delirium screening scores, and consequently higher post-operative delirium risk (clusters 1 and 2). ML techniques are valuable tools for extracting insights and making accurate predictions from healthcare data. High-performing algorithm-based models can be implemented for clinical decision support systems, facilitating early identification and intervention for ED in pediatric patients. By investigating various variables, it is possible to assess risk and implement preventive measures effectively. Furthermore, unsupervised clustering reveals distinct patient subgroups, enabling personalized perioperative management strategies and enhancing overall patient care.

Leveraging Tunability of Localized-Interfacial Memristors for Efficient Handling of Complex Neural Networks.

A scalable (<130 nm) resistive switching memristor that features both filamentary and interfacial switching aimed at neuromorphic computing is developed in this study. The typically perceived noise or volatility was effectively harnessed as a controlled mechanism for interfacial switching. The multilayer structure for the proposed memristor enhances switching stability by curbing ionic overmigration and mitigating leakage paths. Furthermore, the memristors showcased their reliability by demonstrating more than 15 M cycles in the filamentary mode and 1 M pulses in the interfacial mode. Additionally, retention tests at 85 °C for 104 s confirmed the stability across different states, affirming its reliability as a nonvolatile CMOS-compatible element. While many studies validate performance solely on the MNIST data set, this work also evaluates more complex data sets, demonstrating the robustness of the demonstrated memristor in supervised learning. Specifically, supervised learning simulations on MNIST and fashion MNIST data sets indicated a high learning rate with <4% deviations from numerical training, while offline inference trained on CIFAR-10 and CIFAR-100 data sets revealed <2.5% and <7% deviations caused by programing error accumulation, even with increased memristor counts for these highly complex data sets. Unsupervised learning via spike-timing-dependent plasticity further highlights the potential of the developed memristor in bridging artificial and biological paradigms, offering a significant advance toward efficient and biologically inspired computing architectures.

Self-supervised learning from images: No negative pairs, no cluster-balancing

Learning with self-derived targets provides a non-contrastive method for unsupervised image representation learning, where the variety in targets is crucial. Recent work has achieved good performance by learning with targets obtained via cluster-balancing. However, the equal-cluster-size constraint becomes too restrictive for handling data with imbalanced categories or coming in small batches. In this paper, we propose a new clustering-based approach for non-contrastive image representation learning with no need for a particular architecture design or extra memory bank and no explicit constraints on cluster size. A key formulation is to learn embedding consistency and variable decorrelation in the cluster space by tweaking the batch-wise cross-correlation matrix towards an identity one. With this identitization loss incorporated, predicted cluster assignments of two randomly augmented views of the same image serve as targets for each other. We carried out comprehensive experimental studies of linear classification with learned representations of benchmark image datasets. Our results show that the proposed approach significantly outperforms state-of-the-art approaches and is more robust to class imbalance than those with cluster balancing.

Wide Area VISTA Extra-galactic Survey (WAVES): Unsupervised star-galaxy separation on the WAVES-Wide photometric input catalogue using UMAP and hdbscan

Abstract Star-galaxy separation is a crucial step in creating target catalogues for extragalactic spectroscopic surveys. A classifier biased towards inclusivity risks including high numbers of stars, wasting fibre hours, while a more conservative classifier might overlook galaxies, compromising completeness and hence survey objectives. To avoid bias introduced by a training set in supervised methods, we employ an unsupervised machine learning approach. Using photometry from the Wide Area VISTA Extragalactic Survey (WAVES)-Wide catalogue comprising 9-band u - Ks data, we create a feature space with colours, fluxes, and apparent size information extracted by ProFound. We apply the non-linear dimensionality reduction method UMAP (Uniform Manifold Approximation and Projection) combined with the classifier hdbscan to classify stars and galaxies. Our method is verified against a baseline colour and morphological method using a truth catalogue from Gaia, SDSS, GAMA, and DESI. We correctly identify 99.75% of galaxies within the AB magnitude limit of Z = 21.2, with an F1 score of 0.9971 ± 0.0018 across the entire ground truth sample, compared to 0.9879 ± 0.0088 from the baseline method. Our method’s higher purity (0.9967 ± 0.0021) compared to the baseline (0.9795 ± 0.0172) increases efficiency, identifying 11% fewer galaxy or ambiguous sources, saving approximately 70,000 fibre hours on the 4MOST instrument. We achieve reliable classification statistics for challenging sources including quasars, compact galaxies, and low surface brightness galaxies, retrieving 92.7%, 84.6%, and 99.5% of them respectively. Angular clustering analysis validates our classifications, showing consistency with expected galaxy clustering, regardless of the baseline classification.

Ranking and Combining Latent Structured Predictive Scores without Labeled Data

Combining multiple predictors obtained from distributed data sources to an accurate meta-learner is promising to achieve enhanced performance in lots of prediction problems. As the accuracy of each predictor is usually unknown, integrating the predictors to achieve better performance is challenging. Conventional ensemble learning methods assess the accuracy of predictors based on extensive labeled data. In practical applications, however, the acquisition of such labeled data can prove to be an arduous task. Furthermore, the predictors under consideration may exhibit high degrees of correlation, particularly when similar data sources or machine learning algorithms were employed during their model training. In response to these challenges, this paper introduces a novel structured unsupervised ensemble learning model (SUEL) to exploit the dependency between a set of predictors with continuous predictive scores, rank the predictors without labeled data and combine them to an ensembled score with weights. Two novel correlation-based decomposition algorithms are further proposed to estimate the SUEL model, constrained quadratic optimization (SUEL.CQO) and matrix-factorization-based (SUEL.MF) approaches. The efficacy of the proposed methods is rigorously assessed through both simulation studies and real-world application of risk genes discovery. The results compellingly demonstrate that the proposed methods can efficiently integrate the dependent predictors to an ensemble model without the need of ground truth data.

Exploring spatial and seasonal water quality variations in Kelani River, Sri Lanka: a latent variable approach.

Water quality degradation poses a significant challenge globally, especially in developing nations like Sri Lanka. Extensive monitoring programs designed to address escalating river pollution collect multiple water quality parameters over extended periods and varied locations. However, the sheer volume of data can be overwhelming, making it difficult to process effectively and interpret accurately using conventional methods. In this study, latent variable (LV) and unsupervised machine learning techniques were used to investigate spatial and seasonal variations of surface water quality for 17 parameters across 17 locations along the Kelani River, Sri Lanka, using monthly water quality parameters from 2016 to 2020. Pearson's correlation matrix identified 10 parameters significantly affecting water quality variations and factor analysis (FA) generated five LVs, accounting for 77% of the total variance in the dataset. The identified LVs showed multiple methods of river pollution.Hierarchical clustering analysis and self-organizing mapping methods clustered stations in a closely analogous manner. Stations near industrial zones and the river mouth showed higher water quality variance, often exceeding national guidelines. Correlation testing revealed strong relationships between water quality and catchment hydrometeorological variations during monsoonal seasons. Spatial analyses showed increased LV variance in the Lower Kelani River Basin, indicating higher pollutant levels in different seasons. Industrial effluents (LV-2 and LV-4) and domestic and municipal sewage (LV-3 and LV-5) exhibit greater seasonal fluctuations. The results showed that the proposed LV approach has the potential to assist authorities in addressing water pollution amidst the complexity of multiple water quality parameters.

Unsupervised learning of effective quantum impurity models

Unsupervised learning of effective quantum impurity models

Leveraging Unsupervised Learning for Workload Balancing and Resource Utilization in Cloud Architectures

Leveraging Unsupervised Learning for Workload Balancing and Resource Utilization in Cloud Architectures

A Hierarchical Machine Learning Method for Detection and Visualization of Network Intrusions from Big Data

Machine learning is regarded as an effective approach in network intrusion detection, and has gained significant attention in recent studies. However, few intrusion detection methods have been successfully applied to detect anomalies in large-scale network traffic data, and low explainability of the complex algorithms has caused concerns about fairness and accountability. A further problem is that many intrusion detection systems need to work with distributed data sources in the cloud. In this paper, we propose an intrusion detection method based on distributed computing to learn the latent representations from large-scale network data with lower computation time while improving the intrusion detection accuracy. Our proposed classifier, based on a novel hierarchical algorithm combining adaptability and visualization ability from a self-structured unsupervised learning algorithm and achieving explainability from self-explainable supervised algorithms, is able to enhance the understanding of the model and data. The experimental results show that our proposed method is effective, efficient, and scalable in capturing the network traffic patterns and detecting detailed network intrusion information such as type of attack with high detection performance, and is an ideal method to be applied in cloud-computing environments.

Face Image Recognition by using some Dimension Reduction Algorithms and Logistic Regression

Face recognition is a common technique for artificial intelligence and image processing in which machine-learning algorithms automatically recognize a person's face through a huge image collection or even from a live video since there is difficulty in detecting data when using different shots for the same person in a facial recognition system, it takes time to determine person's membership. Furthermore, when working with high-dimensional data, where the 2D-pixel values of facial images, can be computationally challenging, the face recognition problem gets harder, so the current research compares the dimensionality reduction techniques like the principal component analysis algorithm PCA, an unsupervised learning algorithm that reduces dimensions by moving them from a high-dimensional area to a lower area without losing important information, and the two-dimensional principal component analysis algorithm 2DPCA to determine which technique is most effective for the process of detecting facial images, and this method will be compared with the logistic regression which is a statistical model for face recognition that has been used for classification tasks. Two different sets of data were used, the first group represented the ORL database, which was made up of 40 individuals, while the second group represented the real data, which was made up of 100 individuals from AL-Mamoon College, the results showed that the logistic regression model achieved the lowest MSE for ORL dataset and real data by achieving (0.0024414, 0.091936) respectively, followed by 2DPCA and PCA and all three techniques reached the highest accuracy rate of 100% for facial image recognition on real data. Paper type: Research paper