Unsupervised Learning Algorithm Research Articles

Distributional regression modeling via generalized additive models for location, scale, and shape: An overview through a data set from learning analytics.

The advent of technological developments is allowing to gather large amounts of data in several research fields. Learning analytics (LA)/educational data mining has access to big observational unstructured data captured from educational settings and relies mostly on unsupervised machine learning (ML) algorithms to make sense of such type of data. Generalized additive models for location, scale, and shape (GAMLSS) are a supervised statistical learning framework that allows modeling all the parameters of the distribution of the response variable with respect to the explanatory variables. This article overviews the power and flexibility of GAMLSS in relation to some ML techniques. Also, GAMLSS' capability to be tailored toward causality via causal regularization is briefly commented. This overview is illustrated via a data set from the field of LA. This article is categorized under:Application Areas > Education and LearningAlgorithmic Development > StatisticsTechnologies > Machine Learning.

An unsupervised image segmentation algorithm for coronary angiography

Computer visual systems can rapidly obtain a large amount of data and automatically process them with ease. These characteristics constitute advantages for the application of such systems in the automatic analysis of medical images, as well as in processing technology. The precision of image segmentation, which plays a critical role in computer visual systems, directly affects the quality of processing results. Coronary angiographs feature various background colors, complex patterns, and blurry edges. The image areas containing blood vessels cannot be precisely segmented through regular methods. Therefore, this study proposed an unsupervised learning algorithm that uses regional parameter expansion (RPE). This method was derived from the flood fill algorithm, which can effectively segment image areas containing blood vessels despite a complex background or uneven light and shadow. An optimal cover tree (OCT) algorithm was proposed for the establishment of coronary arteries and the estimation of vessel diameter. Through the region growing method, spanning trees were used to record the cover length of adjacent connections, thereby establishing vessel paths, and the length can be used to track changes in vessel diameter.

Classification and Feature Extraction Using Supervised and Unsupervised Machine Learning Approach for Broiler Woody Breast Myopathy Detection

Bioelectrical impedance analysis (BIA) was established to quantify diverse cellular characteristics. This technique has been widely used in various species, such as fish, poultry, and humans for compositional analysis. This technology was limited to offline quality assurance/detection of woody breast (WB); however, inline technology that can be retrofitted on the conveyor belt would be more helpful to processors. Freshly deboned (n = 80) chicken breast fillets were collected from a local processor and analyzed by hand-palpation for different WB severity levels. Data collected from both BIA setups were subjected to supervised and unsupervised learning algorithms. The modified BIA showed better detection ability for regular fillets than the probe BIA setup. In the plate BIA setup, fillets were 80.00% for normal, 66.67% for moderate (data for mild and moderate merged), and 85.00% for severe WB. However, hand-held BIA showed 77.78, 85.71, and 88.89% for normal, moderate, and severe WB, respectively. Plate BIA setup is more effective in detecting WB myopathies and could be installed without slowing the processing line. Breast fillet detection on the processing line can be significantly improved using a modified automated plate BIA.

Decoding conformal field theories: From supervised to unsupervised learning

We use machine learning to classify rational two-dimensional conformal field theories (CFTs). We first use the energy spectra of these minimal models to train a supervised learning algorithm. In contrast to conventional methods that are typically qualitative and also involve system size scaling, our method quantifies the similarity of the spectrum of a system at a fixed size to candidate CFTs. Such an approach allows us to correctly predict the nature and the value of critical points of several strongly correlated spin models using only their energy spectra. Our results are also relevant for the ground-state entanglement Hamiltonian of certain topological phases of matter, described by CFTs. Remarkably, we achieve high prediction accuracy by only using the lowest few R\'eyni entropies as the input. Finally, using autoencoders, an unsupervised learning algorithm, we find a hidden variable that has a direct correlation with the central charge and discuss prospects for using machine learning to investigate other conformal field theories, including higher-dimensional ones.

Bi-level optimization-based projective non-negative matrix factorization

Non-negative matrix factorization (NMF) is a powerful tool for image data analysis and has been utilized in a range of applications involving data classification, clustering, and image processing. Nevertheless, NMF and most of its variants are unsupervised learning algorithms and do not take account of the projection property of the basis image matrix. These flaws will have a negative impact on their classification performance. To resolve the mentioned drawbacks of NMF-based methods, this paper establishes a novel projective non-negative matrix factorization (PNMF) framework using the class-label information and deals with the model via a bi-level quadratic optimization (BLO) strategy. The convergence of the proposed BLO-PNMF algorithm is theoretically analyzed and proved in detail, and also verified by experiments. Our approach and the related state-of-the-art methods are evaluated and compared on three face databases. The experimental results indicate that the BLO-PNMF method can extract more localized features for image data representation and achieve superior performance in face recognition.

Classifying Facies in 3D Digital Rock Images Using Supervised and Unsupervised Approaches

Lithology is one of the critical parameters influencing drilling operations and reservoir production behavior. Well completion is another important area where facies type has a crucial influence on fracture propagation. Geological formations are highly heterogeneous systems that require extensive evaluation with sophisticated approaches. Classification of facies is a critical approach to characterizing different depositional systems. Image classification is implemented as a quick and easy method to detect different facies groups. Artificial intelligence (AI) algorithms are efficiently used to categorize geological formations in a large dataset. This study involves the classification of different facies with various supervised and unsupervised learning algorithms. The dataset for training and testing was retrieved from a digital rock database published in the data brief. The study showed that supervised algorithms provided more accurate results than unsupervised algorithms. In this study, the extreme gradient boosted tree regressor was found to be the best algorithm for facies classification for the synthetic digital rocks.

Identifying Patients With Heart Failure Who Are Susceptible to De Novo Acute Kidney Injury: Machine Learning Approach.

BackgroundStudies have shown that more than half of patients with heart failure (HF) with acute kidney injury (AKI) have newonset AKI, and renal function evaluation markers such as estimated glomerular filtration rate are usually not repeatedly tested during the hospitalization. As an independent risk factor, delayed AKI recognition has been shown to be associated with the adverse events of patients with HF, such as chronic kidney disease and death.ObjectiveThe aim of this study is to develop and assess of an unsupervised machine learning model that identifies patients with HF and normal renal function but who are susceptible to de novo AKI.MethodsWe analyzed an electronic health record data set that included 5075 patients admitted for HF with normal renal function, from which 2 phenogroups were categorized using an unsupervised machine learning algorithm called K-means clustering. We then determined whether the inferred phenogroup index had the potential to be an essential risk indicator by conducting survival analysis, AKI prediction, and the hazard ratio test.ResultsThe AKI incidence rate in the generated phenogroup 2 was significantly higher than that in phenogroup 1 (group 1: 106/2823, 3.75%; group 2: 259/2252, 11.50%; P<.001). The survival rate of phenogroup 2 was consistently lower than that of phenogroup 1 (P<.005). According to logistic regression, the univariate model using the phenogroup index achieved promising performance in AKI prediction (sensitivity 0.710). The generated phenogroup index was also significant in serving as a risk indicator for AKI (hazard ratio 3.20, 95% CI 2.55-4.01). Consistent results were yielded by applying the proposed model on an external validation data set extracted from Medical Information Mart for Intensive Care (MIMIC) III pertaining to 1006 patients with HF and normal renal function.ConclusionsAccording to a machine learning analysis on electronic health record data, patients with HF who had normal renal function were clustered into separate phenogroups associated with different risk levels of de novo AKI. Our investigation suggests that using machine learning can facilitate patient phengrouping and stratification in clinical settings where the identification of high-risk patients has been challenging.

Attention-based Feature Fusion Generative Adversarial Network for yarn-dyed fabric defect detection

Defects on the surface of yarn-dyed fabrics are one of the important factors affecting the quality of fabrics. Defect detection is the core link of quality control. Due to the diversity of yarn-dyed fabric patterns and the scarcity of defect samples, reconstruction-based unsupervised deep learning algorithms have received extensive attention in the field of fabric defect detection. However, most existing deep learning algorithms cannot fully extract shallow, high-frequency and high-level information, which limits their ability to reconstruct yarn-dyed fabric images. In this article, we propose an Attention-based Feature Fusion Generative Adversarial Network framework for unsupervised defect detection of yarn-dyed fabrics. The framework utilizes a modified Feature Pyramid Network to fuse multi-level information and utilizes an attention mechanism to enhance the model's feature representation capabilities. The Attention-based Feature Fusion Generative Adversarial Network consists of an attention fusion generator and a patch-level discriminator. In the attention fusion generator, the Feature Pyramid Network with EfficientNetV2 as the backbone is used as the core building block, and different feature fusion methods are used to avoid the loss of information in the process of network deepening. The attention mechanism is used to enhance the channel and spatial-wise correlation of features, which helps the model to focus on more meaningful information by recalibrating the feature maps. In the discriminator, the patch-level discriminator is used to calculate the similarity between the reconstructed image and the original image from a local perspective, thereby improving the model's attention to texture details. Experimental results on public datasets demonstrate the effectiveness of the proposed method compared to other methods.

Attention‐based vector quantisation variational autoencoder for colour‐patterned fabrics defect detection

AbstractDefect detection is an essential link in the fabric production process. Due to the diversity of patterns and scarcity of defect samples for colour‐patterned fabrics, reconstruction‐based unsupervised deep learning algorithms have received extensive attention in the field of fabric defect detection. Among them, unsupervised reconstruction models based on variational autoencoders (VAEs) have been shown to be effective. However, there is a problem of posterior collapse in the process of modelling parametric distributions of continuous variables by VAEs. Therefore, VAE‐based defect detection methods for colour‐patterned fabrics usually produce ambiguous reconstruction results, thereby affecting the defect detection performance. In this article, an attention‐based vector quantisation variational autoencoder (AVQ‐VAE) is proposed for colour‐patterned fabric defect detection. The method adopts autoregressive modelling of discrete variables to avoid the posterior collapse problem of traditional VAEs, and utilises attention mechanism to enhance the feature representation ability of the model. AVQ‐VAE consists of encoder, embedding space, decoder and attention mechanism. The encoder is used to map the input image into multiple feature vectors. Vector quantisation in embedding space is used for discretisation and autoregressive modelling of feature vectors. A decoder is used to decode discrete variables into images of the same size as the original input. Furthermore, an attention mechanism is used to capture channel and spatial correlations, which help the model focus on important information by adaptively recalibrating feature maps. Experimental results on public datasets demonstrate that the proposed method is robust and effective for colour‐patterned fabric defect detection.

Seismic Facies Analysis Using the Multiattribute SOM-K-Means Clustering.

An accurate seismic facies analysis (SFA) can provide insight into the subsurface sedimentary facies and has guiding significance for geological exploration. Many machine learning algorithms, including unsupervised, supervised, and deep learning algorithms, have been developed successfully for SFA over the past decades. However, SFA and facies classification are still challenging tasks due to the complex characteristics of geological and seismic data. A multiattribute SOM-K-means clustering algorithm, which implements a two-stage clustering by using multiple geological attributes, is proposed and applied for SFA. The proposed algorithm can effectively extract complementary features from the multiple attribute volumes and comprehensively use the different attributes to improve the recognition ability of seismic facies. Experimental results show that the proposed algorithm improves clustering accuracy and can be used as an effective and powerful tool for SFA.

Integrating geographical information systems, remote sensing, and machine learning techniques to monitor urban expansion: an application to Luanda, Angola

ABSTRACT According to many previous studies, application of remote sensing for the complex and heterogeneous urban environments in Sub-Saharan African countries is challenging due to the spectral confusion among features caused by diversity of construction materials. Resorting to classification based on spectral indices that are expected to better highlight features of interest and to be prone to unsupervised classification, this study aims (1) to evaluate the effectiveness of index-based classification for Land Use Land Cover (LULC) using an unsupervised machine learning algorithm Product Quantized K-means (PQk-means); and (2) to monitor the urban expansion of Luanda, the capital city of Angola in a Logistic Regression Model (LRM). Comparison with state-of-the-art algorithms shows that unsupervised classification by means of spectral indices is effective for the study area and can be used for further studies. The built-up area of Luanda has increased from 94.5 km2 in 2000 to 198.3 km2 in 2008 and to 468.4 km2 in 2018, mainly driven by the proximity to the already established residential areas and to the main roads as confirmed by the logistic regression analysis. The generated probability maps show high probability of urban growth in the areas where government had defined housing programs.

Can You Do That Again? Time Series Consolidation as a Robust Method of Tailoring Gesture Recognition to Individual Users.

Robust inter-session modeling of gestures is still an open learning challenge. A sleeve equipped with capacitive strap sensors was used to capture two gesture data sets from a convenience sample of eight subjects. Two pipelines were explored. In FILT a novel two-stage algorithm was introduced which uses an unsupervised learning algorithm to find samples representing gesture transitions and discards them prior to training and validating conventional models. In TSC a confusion matrix was used to automatically consolidate commonly confused class labels, resulting in a set of gestures tailored to an individual subject’s abilities. The inter-session testing accuracy using the Time Series Consolidation (TSC) method increased from a baseline inter-session average of 42.47 ± 3.83% to 93.02% ± 4.97% while retaining an average of 5.29 ± 0.46 out of the 11 possible gesture categories. These pipelines used classic machine learning algorithms which require relatively small amounts of data and computational power compared to deep learning solutions. These methods may also offer more flexibility in interface design for users suffering from handicaps limiting their manual dexterity or ability to reliably make gestures, and be possible to implement on edge devices with low computational power.

A New Unsupervised Deep Learning Algorithm for Fine-Grained Detection of Driver Distraction

Traffic accidents caused by distracted drivers account for a large proportion of traffic accidents each year, and monitoring the driving state of drivers to avoid traffic accidents caused by distracted driving has become a very important research direction. At present, the field of driver distraction detection mainly adopts supervised learning methods, which have problems such as poor generalization ability, large labeling cost, and weak artificial intelligence. This paper is oriented toward driver distraction fine-grained detection and innovatively proposes a new unsupervised deep learning algorithm, which is referred to as UDL, to achieve a more human-like level of intelligence. First, we build a new unsupervised deep learning algorithm; furthermore, we integrate the multilayer perceptron (MLP) architecture to build a new backbone and projection head to strengthen feature extraction capabilities; and finally, a new loss function based on contrast learning and a stop-gradient strategy is designed to guide the model to learn more robust features. The comparison results on large-scale driver distraction detection datasets show that our UDL method can accurately detect driver distraction without labels and exhibits excellent generalization performance with a linear evaluation accuracy of 97.38%; In addition, after fine-tuning with fewer labels, our UDL method can achieve superior performance close to state-of-the-art supervised learning methods, achieving 99.07% accuracy after fine-tuning using only 50% of the labeled data, which greatly reduces the cost and limitations of manual annotation.

Zero-phase angle asteroid taxonomy classification using unsupervised machine learning algorithms

Context. We are in an era of large catalogs and, thus, statistical analysis tools for large data sets, such as machine learning, play a fundamental role. One example of such a survey is the Sloan Moving Object Catalog (MOC), which lists the astrometric and photometric information of all moving objects captured by the Sloan field of view. One great advantage of this telescope is represented by its set of five filters, allowing for taxonomic analysis of asteroids by studying their colors. However, until now, the color variation produced by the change of phase angle of the object has not been taken into account. Aims. In this paper, we address this issue by using absolute magnitudes for classification. We aim to produce a new taxonomic classification of asteroids based on their magnitudes that is unaffected by variations caused by the change in phase angle. Methods. We selected 9481 asteroids with absolute magnitudes of Hg, Hi and Hz, computed from the Sloan Moving Objects Catalog using the HG*12 system. We calculated the absolute colors with them. To perform the taxonomic classification, we applied a unsupervised machine learning algorithm known as fuzzy C-means. This is a useful soft clustering tool for working with data sets where the different groups are not completely separated and there are regions of overlap between them. We have chosen to work with the four main taxonomic complexes, C, S, X, and V, as they comprise most of the known spectral characteristics. Results. We classified a total of 6329 asteroids with more than 60% probability of belonging to the assigned taxonomic class, with 162 of these objects having been characterized by an ambiguous classification in the past. By analyzing the sample obtained in the plane Semimajor axis versus inclination, we identified 15 new V-type asteroid candidates outside the Vesta family region.

GNGraph: Self-Organizing Maps for Autonomous Aerial Vehicle Planning

The present letter tackles the problem of planning a collision-free path in a known environment from a general point of view. We address the problem by using an unsupervised learning algorithm to generate a sparse graph representing the topological structure of the environment and use it for planning paths in 3D spaces. We propose GNGraph, an integrated solution combining the Growing Neural Gas algorithm to generate the sparse graph, a stop criterion to guarantee the graph's connectivity and a collision check to assess the edges and nodes validity. The proposed solution has been tested on simulated and real environment maps, and compared against a state-of-the-art graph planning algorithm among other global planning methods.

Thinking Aloud or Screaming Inside: Exploratory Study of Sentiment Around Work.

BackgroundMillions of workers experience work-related ill health every year. The loss of working days often accounts for poor well-being because of discomfort and stress caused by the workplace. The ongoing pandemic and postpandemic shift in socioeconomic and work culture can continue to contribute to adverse work-related sentiments. Critically investigating state-of-the-art technologies, this study identifies the research gaps in recognizing workers’ need for well-being support, and we aspire to understand how such evidence can be collected to transform the workforce and workplace.ObjectiveBuilding on recent advances in sentiment analysis, this study aims to closely examine the potential of social media as a tool to assess workers’ emotions toward the workplace.MethodsThis study collected a large Twitter data set comprising both pandemic and prepandemic tweets facilitated through a human-in-the-loop approach in combination with unsupervised learning and meta-heuristic optimization algorithms. The raw data preprocessed through natural language processing techniques were assessed using a generative statistical model and a lexicon-assisted rule-based model, mapping lexical features to emotion intensities. This study also assigned human annotations and performed work-related sentiment analysis.ResultsA mixed methods approach, including topic modeling using latent Dirichlet allocation, identified the top topics from the corpus to understand how Twitter users engage with discussions on work-related sentiments. The sorted aspects were portrayed through overlapped clusters and low intertopic distances. However, further analysis comprising the Valence Aware Dictionary for Sentiment Reasoner suggested a smaller number of negative polarities among diverse subjects. By contrast, the human-annotated data set created for this study contained more negative sentiments. In this study, sentimental juxtaposition revealed through the labeled data set was supported by the n-gram analysis as well.ConclusionsThe developed data set demonstrates that work-related sentiments are projected onto social media, which offers an opportunity to better support workers. The infrastructure of the workplace, the nature of the work, the culture within the industry and the particular organization, employers, colleagues, person-specific habits, and upbringing all play a part in the health and well-being of any working adult who contributes to the productivity of the organization. Therefore, understanding the origin and influence of the complex underlying factors both qualitatively and quantitatively can inform the next generation of workplaces to drive positive change by relying on empirically grounded evidence. Therefore, this study outlines a comprehensive approach to capture deeper insights into work-related health.

An interpretable multi-task system for clinically applicable COVID-19 diagnosis using CXR.

With the emergence of continuously mutating variants of coronavirus, it is urgent to develop a deep learning model for automatic COVID-19 diagnosis at early stages from chest X-ray images. Since laboratory testing is time-consuming and requires trained laboratory personal, diagnosis using chest X-ray (CXR) is a befitting option. In this study, we proposed an interpretable multi-task system for automatic lung detection and COVID-19 screening in chest X-rays to find an alternate method of testing which are reliable, fast and easily accessible, and able to generate interpretable predictions that are strongly correlated with radiological findings. The proposed system consists of image preprocessing and an unsupervised machine learning (UML) algorithm for lung region detection, as well as a truncated CNN model based on deep transfer learning (DTL) to classify chest X-rays into three classes of COVID-19, pneumonia, and normal. The Grad-CAM technique was applied to create class-specific heatmap images in order to establish trust in the medical AI system. Experiments were performed with 15,884 frontal CXR images to show that the proposed system achieves an accuracy of 91.94% in a test dataset with 2,680 images including a sensitivity of 94.48% on COVID-19 cases, a specificity of 88.46% on normal cases, and a precision of 88.01% on pneumonia cases. Our system also produced state-of-the-art outcomes with a sensitivity of 97.40% on public test data and 88.23% on a previously unseen clinical data (1,000 cases) for binary classification of COVID-19-positive and COVID-19-negative films. Our automatic computerized evaluation for grading lung infections exhibited sensitivity comparable to that of radiologist interpretation in clinical applicability. Therefore, the proposed solution can be used as one element of patient evaluation along with gold-standard clinical and laboratory testing.

Toward a Self-Supervised Architecture for Semen Quality Prediction Using Environmental and Lifestyle Factors

Male fertility has been seen to be declining, prompting for more effective and accessible means of its assessment. Artificial intelligence methods have been effective toward predicting semen quality through a questionnaire-based information source comprising a selection of factors from the medical literature which have been seen to influence semen quality. Prior work has seen the application of supervised learning toward the prediction of semen quality, but since supervised learning hinges on the provision of data class labels it can be said to depend on an external intelligence intervention, which can translate toward further costs and resources in practical settings. In contrast, unsupervised learning methods partition data into clusters and groups based on an objective function and do not rely on the provision of class labels and can allow for a fully automated flow of a prediction platform. In this paper, we apply three unsupervised learning models with different model architectures, namely Gaussian mixture model (GMM), K-means, and spectral clustering (SC), alongside low dimensional embedding methods which include sparse autoencoder (SAE), principal component analysis (PCA), and robust PCA. The best results were obtained with a combination of the SAE and the SC algorithm, which was likely due to its nonspecific and arbitrary cluster shape assumption. Further work would now involve the exploration of similar unsupervised learning algorithms with a similar framework to the SC to investigate the extent to which various clusters can be learned with maximal accuracy.

Supervised and Unsupervised Machine Learning Algorithms for Forecasting the Fracture Location in Dissimilar Friction-Stir-Welded Joints

Artificial-intelligence-based algorithms are used in manufacturing to automate difficult activities and discover workflow or process patterns that had never been noticed before. Recent studies deal with the forecasting of the fracture location in dissimilar friction-stir-welded AA5754–C11000 alloys. Four types of supervised machine-learning-based classification algorithms i.e., decision tree, logistic classification, random forest, and AdaBoost were implemented. Additionally, in the present work, for the first time, a neurobiological-based unsupervised machine learning algorithm, i.e., self-organizing map (SOM) neural network, is implemented for determining the fracture location in dissimilar friction-stir-welded AA5754–C11000 alloys. Tool shoulder diameter (mm), tool rotational speed (RPM), and tool traverse speed (mm/min) are input parameters, while the fracture location, i.e., whether the specimen’s fracture is in the thermo-mechanically affected zone (TMAZ) of copper, or if it fractures in the TMAZ of aluminium. The results show that out of all implemented algorithms, the SOM algorithm is able to predict the fracture location with the highest accuracy of 96.92%.

Seismic reservoir characterization of the Gassum Formation in the Stenlille aquifer gas storage, Denmark — Part 2: Unsupervised classification

Ideally, a good static reservoir model provides an accurate estimate of the extent, porosity, permeability, and lithology of the container as well as the properties of the seal and any faults or fractures that may allow the reservoir to leak. The major pitfall of deterministic, statistical, or supervised learning workflows is that they estimate only the properties sampled by the wells or provided by empirical relations and may miss mapping heterogeneities in the reservoir and seal that can give rise to flow baffles and reservoir leakage. This shortcoming is exacerbated when the number of wells is small, the types of logs recorded are limited, and the migrated seismic gathers are absent or of limited quality. In contrast, unsupervised learning looks for patterns in the seismic amplitude and attribute volumes themselves. In this paper, we apply unsupervised learning algorithms to evaluate the natural gas storage Stenlille aquifer in Denmark and compare the results with a supervised multiattribute regression reservoir characterization described in a companion paper. Specifically, we apply principal component analysis, self-organizing mapping, and generative topographic mapping workflows to extract patterns across eight attribute volumes: relative acoustic impedance, energy, sweetness, gray level co-occurrence matrices (GLCM) entropy, curvedness, and three spectral magnitude volumes. We find that the large-scale patterns are similar, but that the unsupervised learning algorithms provide greater detail. Because our deterministic model was built on poststack data using the limited well-log data available, we believe that the heterogeneity mapped by the unsupervised learning workflows provides a relatively unbiased means of estimating risk in our reservoir model. Quantifying the importance of these anomalies will need to be reconciled with a dynamic reservoir model.