Unsupervised Learning Problem Research Articles

Ensemble clustering via synchronized relabelling

Ensemble clustering is an important problem in unsupervised learning that aims at aggregating multiple noisy partitions into a unique clustering solution. It can be formulated in terms of relabelling and voting, where relabelling refers to the task of finding optimal permutations that bring coherence among labels in input partitions. In this paper we propose a novel solution to the relabelling problem based on permutation synchronization. By effectively circumventing the need for a reference clustering, our method achieves superior performance than previous work under varying assumptions and scenarios, demonstrating its capability to handle diverse and complex datasets.

PNNCLR: Stochastic pseudo neighborhoods for contrastive learning based unsupervised representation learning problems

Nearest neighbor (NN) sampling provides more semantic variations than predefined transformations for self-supervised learning (SSL) based image recognition problems. However, its performance is restricted by the quality of the support set, which holds positive samples for the contrastive loss. In this work, we show that the quality of the support set plays a crucial role in any nearest neighbor based method for SSL. We then provide a refined baseline (pNNCLR) to the nearest neighbor based SSL approach (NNCLR). To this end, we introduce pseudo nearest neighbors (pNN) to control the quality of the support set, wherein, rather than sampling the nearest neighbors, we sample in the vicinity of hard nearest neighbors by varying the magnitude of the resultant vector and employing a stochastic sampling strategy to improve the performance. Additionally, to stabilize the effects of uncertainty in NN-based learning, we employ a smooth-weight-update approach for training the proposed network. Evaluation of the proposed method on multiple public image recognition and medical image recognition datasets shows that it performs up to 8 percent better than the baseline nearest neighbor method, and is comparable to other previously proposed SSL methods. The code is available at https://github.com/mb16biswas/pnnclr.

Multiple Hypothesis Dropout: Estimating the Parameters of Multi-Modal Output Distributions

In many real-world applications, from robotics to pedestrian trajectory prediction, there is a need to predict multiple real-valued outputs to represent several potential scenarios. Current deep learning techniques to address multiple-output problems are based on two main methodologies: (1) mixture density networks, which suffer from poor stability at high dimensions, or (2) multiple choice learning (MCL), an approach that uses M single-output functions, each only producing a point estimate hypothesis. This paper presents a Mixture of Multiple-Output functions (MoM) approach using a novel variant of dropout, Multiple Hypothesis Dropout. Unlike traditional MCL-based approaches, each multiple-output function not only estimates the mean but also the variance for its hypothesis. This is achieved through a novel stochastic winner-take-all loss which allows each multiple-output function to estimate variance through the spread of its subnetwork predictions. Experiments on supervised learning problems illustrate that our approach outperforms existing solutions for reconstructing multimodal output distributions. Additional studies on unsupervised learning problems show that estimating the parameters of latent posterior distributions within a discrete autoencoder significantly improves codebook efficiency, sample quality, precision and recall.

Research on improving accuracy and efficiency of animal data collection and classification using machine learning

With the continuous expansion of machine learning algorithms in various application domains, the application value of new algorithms, such as Support Vector Machines and Convolutional Neural Networks in data classification, has garnered increasing attention. This paper takes machine learning algorithms as the research entry point, explores the concept of machine learning, and delves into its application value in data classification. This paper, starting with an overview of machine learning algorithms, analyzes the supervised and unsupervised learning problems in machine learning, focusing on the applications of Convolutional Neural Networks, Support Vector Machine models, and logistic regression algorithms in data classification. This study emphasizes designing and implementing a machine learning-based image classification system. Through an in-depth exploration of the application of machine learning algorithms in data classification, a fully functional system is constructed, encompassing multiple modules, including machine vision and software development. This system accurately classifies and recognizes images, providing practical tools and technical support for image processing and analysis. In this study, the goal of achieving good image classification is realized through research and the application of machine learning algorithms. By designing and implementing a machine learning-based image classification system, the accuracy and efficiency of classification in handling massive data are improved. This system also demonstrates wide-ranging prospects in software development and machine vision, among other fields.

A Unified Framework for Tuning Hyperparameters in Clustering Problems

Selecting hyperparameters for unsupervised learning problems is challenging in general due to the lack of ground truth for validation. Despite the prevalence of this issue in statistics and machine learning, especially in clustering problems, there are not many methods for tuning these hyperparameters with theoretical guarantees. In this paper, we provide a framework with provable guarantees for selecting hyperparameters in a number of distinct models. We consider both the subgaussian mixture model and network models to serve as examples of i.i.d. and non-i.i.d. data. We demonstrate that the same framework can be used to choose the Lagrange multipliers of penalty terms in semi-definite programming (SDP) relaxations for community detection, and the bandwidth parameter for constructing kernel similarity matrices for spectral clustering. By incorporating a cross-validation procedure, we show the framework can also do consistent model selection for network models. Using a variety of simulated and real data examples, we show that our framework outperforms other widely used tuning procedures in a broad range of parameter settings.

Monitoring Changes in Clustering Solutions: A Review of Models and Applications

This article comprehensively reviews the applications and algorithms used for monitoring the evolution of clustering solutions in data streams. The clustering technique is an unsupervised learning problem that involves the identification of natural subgroups in a large dataset. In contrast to supervised learning models, clustering is a data mining technique that retrieves the hidden pattern in the input dataset. The clustering solution reflects the mechanism that leads to a high level of similarity between the items. A few applications include pattern recognition, knowledge discovery, and market segmentation. However, many modern-day applications generate streaming or temporal datasets over time, where the pattern is not stationary and may change over time. In the context of this article, change detection is the process of identifying differences in the cluster solutions obtained from streaming datasets at consecutive time points. In this paper, we briefly review the models/algorithms introduced in the literature to monitor clusters’ evolution in data streams. Monitoring the changes in clustering solutions in streaming datasets plays a vital role in policy-making and future prediction. Of course, it has a wide range of applications that cannot be covered in a single study, but some of the most common are highlighted in this article.

Multimodal Mutual Information Maximization: A Novel Approach for Unsupervised Deep Cross-Modal Hashing.

In this article, we adopt the maximizing mutual information (MI) approach to tackle the problem of unsupervised learning of binary hash codes for efficient cross-modal retrieval. We proposed a novel method, dubbed cross-modal info-max hashing (CMIMH). First, to learn informative representations that can preserve both intramodal and intermodal similarities, we leverage the recent advances in estimating variational lower bound of MI to maximizing the MI between the binary representations and input features and between binary representations of different modalities. By jointly maximizing these MIs under the assumption that the binary representations are modeled by multivariate Bernoulli distributions, we can learn binary representations, which can preserve both intramodal and intermodal similarities, effectively in a mini-batch manner with gradient descent. Furthermore, we find out that trying to minimize the modality gap by learning similar binary representations for the same instance from different modalities could result in less informative representations. Hence, balancing between reducing the modality gap and losing modality-private information is important for the cross-modal retrieval tasks. Quantitative evaluations on standard benchmark datasets demonstrate that the proposed method consistently outperforms other state-of-the-art cross-modal retrieval methods.

Кластерная модель анализа больших данных в животноводческом производстве

Intelligent methods of analysis, which include the problem of clustering, are widely used in the field of economics of the agro-industrial complex. The clustering problem belongs to the class of unsupervised learning problems. The essence of the problem is the grouping of research objects according to the use of similarity. If the regions of the Russian Federation are selected in terms of livestock production, they can also be automatically grouped according to the similarity recipe. The k-means method is currently a successful method for solving clustering problems. The main stage of solving the problem is the collection of data, which includes all the main characteristics of the object. For example, if you set up production in the region in terms of animal husbandry, then it would be more logical to x1 - meat production per capita and x2 – milk production per capita. The criterion for choosing the number of clusters is the root mean square error. In total, 79 regions of the Russian Federation participated in the analysis. It turned out that the potential to break all regions of the Russian Federation into 7 clusters of similarity. Regions with high milk and meat production (clusters 4 and 6), regions with high milk and meat production (clusters 2, 3, 5) and regions with low milk and meat production (clusters 0, 1) were identified.

UNSUPERVISED TRANSUDATIVE TL FEATURE LEARNING FOR IMAGE FEATURE EXTRACTION AND REPRESENTATION

In this study, we address the problem of unsupervised transductive transfer learning for image feature extraction and representation. While transfer learning has shown promising results in various domains, its application to image feature extraction in an unsupervised transductive setting remains relatively unexplored. The research gap lies in the scarcity of methods that can effectively learn meaningful image representations without access to labeled data in the target domain, hindering the broader applicability of transfer learning in computer vision. Our research seeks to bridge this gap by proposing a novel framework that leverages unsupervised feature learning to enhance the adaptability of models across different image domains, thus contributing to the advancement of transfer learning in the field of computer vision. Experimental results demonstrate the effectiveness of our method in addressing this critical research gap and its potential for real-world applications.

A moment-matching metric for latent variable generative models

It is difficult to assess the quality of a fitted model in unsupervised learning problems. Latent variable models, such as variational autoencoders and Gaussian mixture models, are often trained with likelihood-based approaches. In the scope of Goodhart’s law, when a metric becomes a target it ceases to be a good metric and therefore we should not use likelihood to assess the quality of the fit of these models. The solution we propose is a new metric for model comparison or regularization that relies on moments. The key idea is to study the difference between the data moments and the model moments using a matrix norm, such as the Frobenius norm. We first show how to use this new metric for model comparison and then for regularization. We show that our proposed metric is faster to compute and has a smaller variance than the commonly used procedure of drawing samples from the fitted distribution. We conclude this article with a demonstration of both applications and we discuss our findings and future work.

On the Philosophy of Unsupervised Learning

Unsupervised learning algorithms are widely used for many important statistical tasks with numerous applications in science and industry. Yet despite their prevalence, they have attracted remarkably little philosophical scrutiny to date. This stands in stark contrast to supervised and reinforcement learning algorithms, which have been widely studied and critically evaluated, often with an emphasis on ethical concerns. In this article, I analyze three canonical unsupervised learning problems: clustering, abstraction, and generative modeling. I argue that these methods raise unique epistemological and ontological questions, providing data-driven tools for discovering natural kinds and distinguishing essence from contingency. This analysis goes some way toward filling the lacuna in contemporary philosophical discourse on unsupervised learning, as well as bringing conceptual unity to a heterogeneous field more often described by what it is not (i.e., supervised or reinforcement learning) than by what it is. I submit that unsupervised learning is not just a legitimate subject of philosophical inquiry but perhaps the most fundamental branch of all AI. However, an uncritical overreliance on unsupervised methods poses major epistemic and ethical risks. I conclude by advocating for a pragmatic, error-statistical approach that embraces the opportunities and mitigates the challenges posed by this powerful class of algorithms.

Superpolynomial quantum-classical separation for density modeling

Density modelling is the task of learning an unknown probability density function from samples, and is one of the central problems of unsupervised machine learning. In this work, we show that there exists a density modelling problem for which fault-tolerant quantum computers can offer a super-polynomial advantage over classical learning algorithms, given standard cryptographic assumptions. Along the way, we provide a variety of additional results and insights, of potential interest for proving future distribution learning separations between quantum and classical learning algorithms. Specifically, we (a) provide an overview of the relationships between hardness results in supervised learning and distribution learning, and (b) show that any weak pseudo-random function can be used to construct a classically hard density modelling problem. The latter result opens up the possibility of proving quantum-classical separations for density modelling based on weaker assumptions than those necessary for pseudo-random functions.

Supervised learning of process discovery techniques using graph neural networks

Automatically discovering a process model from an event log is the prime problem in process mining. This task is so far approached as an unsupervised learning problem through graph synthesis algorithms. Algorithmic design decisions and heuristics allow for efficiently finding models in a reduced search space. However, design decisions and heuristics are derived from assumptions about how a given behavioral description—an event log—translates into a process model and were not learned from actual models which introduce biases in the solutions. In this paper, we explore the problem of supervised learning of a process discovery technique. We introduce a technique for training an ML-based model using graph convolutional neural networks, which translates a given input event log into a sound Petri net. We show that training this model on synthetically generated pairs of input logs and output models allows it to translate previously unseen synthetic and several real-life event logs into sound, arbitrarily structured models of comparable accuracy and simplicity as existing state of the art techniques in imperative mining. We analyze the limitations of the proposed technique and outline alleys for future work.

On Equivalence of Anomaly Detection Algorithms

In most domains, anomaly detection is typically cast as an unsupervised learning problem because of the infeasibility of labeling large datasets. In this setup, the evaluation and comparison of different anomaly detection algorithms is difficult. Although some work has been published in this field, they fail to account that different algorithms can detect different kinds of anomalies. More precisely, the literature on this topic has focused on defining criteria to determine which algorithm is better, while ignoring the fact that such criteria are meaningful only if the algorithms being compared are detecting the same kind of anomalies. Therefore, in this article, we propose an equivalence criterion for anomaly detection algorithms that measures to what degree two anomaly detection algorithms detect the same kind of anomalies. First, we lay out a set of desirable properties that such an equivalence criterion should have and why; second, we propose Gaussian Equivalence Criterion (GEC) as equivalence criterion and show mathematically that it has the desirable properties previously mentioned. Finally, we empirically validate these properties using a simulated and a real-world dataset. For the real-world dataset, we show how GEC can provide insight about the anomaly detection algorithms as well as the dataset.

Enhancement of K-means clustering in big data based on equilibrium optimizer algorithm

Abstract Data mining’s primary clustering method has several uses, including gene analysis. A set of unlabeled data is divided into clusters using data features in a clustering study, which is an unsupervised learning problem. Data in a cluster are more comparable to one another than to those in other groups. However, the number of clusters has a direct impact on how well the K-means algorithm performs. In order to find the best solutions for these real-world optimization issues, it is necessary to use techniques that properly explore the search spaces. In this research, an enhancement of K-means clustering is proposed by applying an equilibrium optimization approach. The suggested approach adjusts the number of clusters while simultaneously choosing the best attributes to find the optimal answer. The findings establish the usefulness of the suggested method in comparison to existing algorithms in terms of intra-cluster distances and Rand index based on five datasets. Through the results shown and a comparison of the proposed method with the rest of the traditional methods, it was found that the proposal is better in terms of the internal dimension of the elements within the same cluster, as well as the Rand index. In conclusion, the suggested technique can be successfully employed for data clustering and can offer significant support.

ClaSP: parameter-free time series segmentation

The study of natural and human-made processes often results in long sequences of temporally-ordered values, aka time series (TS). Such processes often consist of multiple states, e.g. operating modes of a machine, such that state changes in the observed processes result in changes in the distribution of shape of the measured values. Time series segmentation (TSS) tries to find such changes in TS post-hoc to deduce changes in the data-generating process. TSS is typically approached as an unsupervised learning problem aiming at the identification of segments distinguishable by some statistical property. Current algorithms for TSS require domain-dependent hyper-parameters to be set by the user, make assumptions about the TS value distribution or the types of detectable changes which limits their applicability. Common hyper-parameters are the measure of segment homogeneity and the number of change points, which are particularly hard to tune for each data set. We present ClaSP, a novel, highly accurate, hyper-parameter-free and domain-agnostic method for TSS. ClaSP hierarchically splits a TS into two parts. A change point is determined by training a binary TS classifier for each possible split point and selecting the one split that is best at identifying subsequences to be from either of the partitions. ClaSP learns its main two model-parameters from the data using two novel bespoke algorithms. In our experimental evaluation using a benchmark of 107 data sets, we show that ClaSP outperforms the state of the art in terms of accuracy and is fast and scalable. Furthermore, we highlight properties of ClaSP using several real-world case studies.

Research on milling chatter identification of thin-walled parts based on incremental learning and multi-signal fusion

Chatter is a kind of self-excited vibration that often occurs in the milling process of thin-walled parts, which has become the main factor restricting production efficiency and quality. Due to the occurrence of chatter, the signal becomes more complex and unstable. In order to realize milling chatter detection of thin-walled parts, the method of multi-sensor signal fusion is used. A chatter detection method based on variational mode decomposition (VMD) and nonlinear dimensionless index is proposed by analyzing the characteristics of signals in time–frequency domain. Firstly, a series of intrinsic mode function (IMF) components are obtained by decomposing force and acceleration signals with VMD. When chatter occurs, the energy is transferred to the chatter frequency band. Each IMF signal’s nonlinear energy entropy (EE) is extracted to construct the feature vector. A support vector machine chatter identification model based on multi-sensor signal fusion is established. To solve the problem of model incremental updating, supervised learning and unsupervised learning are combined to provide a method for chatter detection.

Intention recognition for multiple agents

Discovering common intentions of multiple agents is one of the important ways to detect the tendency of their collaborative behaviours. Existing work mainly focuses on intention recognition in a single-agent setting and uses a descriptive model, e.g. Bayesian networks, in the recognition process. In this article, we develop a new approach of identifying intentions for multiple agents through analysing their behaviours over time. We first define a prescriptive, behavioural model for a single agent that represents the agent’s behaviours where their intentions are hidden in the plan execution. We introduce landmarks into the behavioural model therefore enhancing informative features to identify common intentions for multiple agents. Subsequently, we refine the model by focusing only on action sequences in their plans and provide a light model for identifying and comparing their intentions. The new model provides a simple approach of grouping agents’ common intentions upon partial plans observed in agents’ interactions. After that, we transform the intention recognition into an un-supervised learning problem and adapt a clustering algorithm to group intentions of multiple agents through comparing their behavioural models. We conduct the clustering process through measuring similarity of probability distributions over potential landmarks in the behavioural models so as to discover agents’ common intentions. Finally, we examine the new intention recognition approaches in two problem domains. We demonstrate importance of recognising common intentions of multiple agents in achieving their goals and provide experimental results to show performance of the new approaches.

An Efficient Method to Accurately Cluster Large Number of High Dimensional Facial Images

Accurately clustering large, high dimensional datasets is a challenging problem in unsupervised learning. K-means is considered to be a fast, widely used and accurate centroid based data partitioning algorithm for spherical datasets. However, its non-determinism and heavy dependence on the selection of initial cluster centers along with vulnerability to noise make it a poor candidate for clustering large datasets with high dimensionality. To overcome these, we develop a novel, nature inspired, centroid based clustering algorithm, inspired from the principles of particle physics. Our method ensures that the convergence to local optima and non-deterministic outputs are avoided. We experiment the method on large datasets of human face images. Besides, our method addresses the problem of outliers and presence of not well-separated data in these datasets. We use a deep learning model for extracting facial features into a vector of 128 dimensions. We validate the quality and accuracy of our methods using different statistical parameters like <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">f-measure, accuracy, error rate, average in group proportion and normalized cluster size rand index</i> . These evaluations show that our method exhibits better accuracy and quality in clustering large face image datasets, in comparison with other existing mechanisms. The strength of our algorithms is more visible as the size of the dataset grows.

AI-Powered Blockchain Technology for Public Health: A Contemporary Review, Open Challenges, and Future Research Directions.

Blockchain technology has been growing at a substantial growth rate over the last decade. Introduced as the backbone of cryptocurrencies such as Bitcoin, it soon found its application in other fields because of its security and privacy features. Blockchain has been used in the healthcare industry for several purposes including secure data logging, transactions, and maintenance using smart contracts. Great work has been carried out to make blockchain smart, with the integration of Artificial Intelligence (AI) to combine the best features of the two technologies. This review incorporates the conceptual and functional aspects of the individual technologies and innovations in the domains of blockchain and artificial intelligence and lays down a strong foundational understanding of the domains individually and also rigorously discusses the various ways AI has been used along with blockchain to power the healthcare industry including areas of great importance such as electronic health record (EHR) management, distant-patient monitoring and telemedicine, genomics, drug research, and testing, specialized imaging and outbreak prediction. It compiles various algorithms from supervised and unsupervised machine learning problems along with deep learning algorithms such as convolutional/recurrent neural networks and numerous platforms currently being used in AI-powered blockchain systems and discusses their applications. The review also presents the challenges still faced by these systems which they inherit from the AI and blockchain algorithms used at the core of them and the scope of future work.