Unsupervised Learning Research Articles

Representing and Quantifying Conformational Changes of Kinases and Phosphatases Using the TSR-Based Algorithm

Protein kinases and phosphatases are key signaling proteins and are important drug targets. An explosion in the number of publicly available 3D structures of proteins has been seen in recent years. Three-dimensional structures of kinase and phosphatase have not been systematically investigated. This is due to the difficulty of designing structure-based descriptors that are capable of quantifying conformational changes. We have developed a triangular spatial relationship (TSR)-based algorithm that enables a unique representation of a protein’s 3D structure using a vector of integers (keys). The main objective of this study is to provide structural insight into conformational changes. We also aim to link TSR-based structural descriptors to their functions. The 3D structures of 2527 kinases and 505 phosphatases are studied. This study results in several major findings as follows: (i) The clustering method yields functionally coherent clusters of kinase and phosphatase families and their superfamilies. (ii) Specific TSR keys are identified as structural signatures for different types of kinases and phosphatases. (iii) TSR keys can identify different conformations of the well-known DFG motif of kinases. (iv) A significant number of phosphatases have their own distinct DFG motifs. The TSR keys from kinases and phosphatases agree with each other. TSR keys are successfully used to represent and quantify conformational changes of CDK2 upon the binding of cyclin or phosphorylation. TSR keys are effective when used as features for unsupervised machine learning and for key searches. If discriminative TSR keys are identified, they can be mapped back to atomic details within the amino acids involved. In conclusion, this study presents an advanced computational methodology with significant advantages in not only representing and quantifying conformational changes of protein structures but also having the capability of directly linking protein structures to their functions.

A Proposed Method of Automating Data Processing for Analysing Data Produced from Eye Tracking and Galvanic Skin Response

The use of eye tracking technology, together with other physiological measurements such as psychogalvanic skin response (GSR) and electroencephalographic (EEG) recordings, provides researchers with information about users’ physiological behavioural responses during their learning process in different types of tasks. These devices produce a large volume of data. However, in order to analyse these records, researchers have to process and analyse them using complex statistical and/or machine learning techniques (supervised or unsupervised) that are usually not incorporated into the devices. The objectives of this study were (1) to propose a procedure for processing the extracted data; (2) to address the potential technical challenges and difficulties in processing logs in integrated multichannel technology; and (3) to offer solutions for automating data processing and analysis. A Notebook in Jupyter is proposed with the steps for importing and processing data, as well as for using supervised and unsupervised machine learning algorithms.

The application of Deep Learning in the field of Robotics

Abstract. As a new field with rapid development in the past two decades, deep learning has received more and more attention from researchers. Neural network adopts widely interconnected structure and effective learning mechanisms to simulate the process of information processing in the human brain, which is an important method in the development of artificial intelligence. Compared with shallow models, deep learning can achieve stronger unsupervised autonomous learning capabilities by increasing the number of layers of the network. In the past, the robots needed to be controlled by the humans; they controlled the robot to finish different tasks. In recent years, the researchers have tried to apply deep learning in the field of robots. Deep neural networks can simulate the complex cognitive laws of the human brain, so many researchers apply deep learning to robots to make robots have the thinking ability of humans. This article will review the cases of combining robot and in-depth learning in recent decades, Summarizing the achievements and problems faced in the three fields of robot vision, trajectory planning and motion control.

A Three-Stage Cross Domain Intelligent Fault Diagnosis Method for Multiple New Faults

Abstract Due to the randomness and complexity of mechanical faults, the new fault modes usually occur unexpectedly in the actual scenarios. In response to the challenges, a three-stage crossing domain intelligent fault diagnosis method is presented in article. Firstly, the partial domain alignment is achieved based on improved target weighted mechanism, and the outlier identifier is constructed to automatically separate the new fault classes. Then, the unsupervised learning model with silhouette coefficient are built to determine the number of new fault categories. Lastly, the simulation signals are further adopted to determine the specific fault categories. Sufficient experiments on the axial piston pump and public bearing datasets validate that the proposed method could predict the number of new fault categories and judge the specific fault categories. The results indicate that the proposed method outperforms the other methods and has promising practical application in fault diagnosis with multiple new faults.

Health Equity in AI Development and Policy: An AI-enabled Study of International, National and Intra-national AI Infrastructures

This study examines how concerns related to equity in AI for health are reflected at the international, national, and sub-national level. Utilizing unsupervised learning over corpora of published AI policy documents and graph structurization and analysis, the research identifies and visualizes the presence and variation of these concerns across different geopolitical contexts. The findings reveal interesting differences in how these issues are prioritized and addressed, highlighting the influence of local policies and cultural factors. The study underscores the importance of tailored approaches to AI governance in healthcare, advocating for increased global collaboration and knowledge sharing to ensure equitable and ethical AI deployment. By providing a comprehensive analysis of policy documents, this research contributes to a deeper understanding of the global landscape of AI in health, potentially offering insights for policymakers and stakeholders.

An Efficient Network Intrusion Detection and Classification System using Machine Learning

In today's digital landscape, network security is of paramount importance, with intrusion detection systems (IDS) playing a crucial role in protecting sensitive data from malicious attacks. Traditional IDS, often reliant on signature-based methods, struggle with high false positive rates, difficulty in adapting to novel threats, and significant computational demands. This paper explores the development of an efficient network intrusion detection and classification system utilizing machine learning techniques to address these challenges. By leveraging datasets such as NSL-KDD and UNSW-NB15, our study employs a combination of supervised learning algorithms, including Support Vector Machines (SVM), Random Forests, and Neural Networks, alongside comprehensive data preprocessing and feature engineering strategies. The evaluation of our models through metrics like accuracy, precision, recall, and ROC-AUC demonstrates a marked improvement in detection capabilities and computational efficiency. Our findings suggest that machine learning-based IDS can significantly enhance network security by reducing false positives and adapting to emerging threats more effectively than traditional systems. This research not only underscores the potential of advanced machine learning techniques in IDS but also provides a robust framework for future developments in the field. In the rapidly evolving landscape of cybersecurity, effective network intrusion detection and classification systems are critical for safeguarding sensitive data and maintaining operational integrity. This paper presents a novel approach utilizing machine learning techniques to enhance the efficiency and accuracy of intrusion detection systems (IDS). By employing a combination of supervised and unsupervised learning algorithms, our system can identify and classify both known and unknown threats in real-time. We leverage advanced feature selection methods to optimize the performance of our models, ensuring high detection rates with minimal false positives. Our experimental results, validated on benchmark datasets, demonstrate significant improvements in detection accuracy and processing speed compared to traditional IDS solutions. The proposed system not only strengthens network defenses but also provides a scalable and adaptive framework for future cybersecurity challenges..

Unsupervised Classification of the Host Response Identifies Dominant Pathobiological Signatures of Sepsis in Sub-Saharan Africa.

The global burden of sepsis is concentrated in sub-Saharan Africa, where inciting pathogens are diverse and HIV co-infection is a major driver of poor outcomes. Biological heterogeneity inherent to sepsis in this setting is poorly defined. To identify dominant pathobiological signatures of sepsis in sub-Saharan Africa and their relationship to clinical phenotypes, patient outcomes, and biological classifications of sepsis identified in high-income-countries (HICs). We analyzed two prospective cohorts of adults hospitalized with sepsis (severe infection with qSOFA score≥1) at disparate settings in Uganda (discovery cohort [Entebbe,urban], N=242; validation cohort [Tororo,rural], N=253). To identify pathobiological signatures in the discovery cohort, we applied unsupervised clustering to 173 soluble proteins reflecting key domains of the host response to severe infection. A random forest-derived classifier was used to predict signature assignment in the validation cohort. Two signatures (Uganda Sepsis Signature [USS]-1 and USS-2) were identified in the discovery cohort, distinguished by expression of proteins involved in myeloid cell and inflammasome activation, T cell co-stimulation and exhaustion, and endothelial barrier dysfunction. A five-protein classifier (AUROC 0.97) reproduced two signatures in the validation cohort with similar biological profiles. In both cohorts, USS-2 mapped to a more severe clinical phenotype associated with HIV and related immunosuppression, severe tuberculosis, and increased risk of 30-day mortality. Substantial biological overlap was observed between USS-2 and hyperinflammatory and reactive sepsis phenotypes identified in HICs. We identified prognostically-enriched pathobiological signatures among sepsis patients with diverse infections and high HIV prevalence in Uganda. Globally inclusive investigations are needed to define generalizable and context-specific mechanisms of sepsis pathobiology, with the goal of improving access to precision medicine treatment strategies.

Distilling Knowledge in Machine Translation of Agglutinative Languages with Backward and Morphological Decoders

Agglutinative languages often have morphologically complex words(MCWs) composed of multiple morphemes arranged in a hierarchical structure, posing significant challenges in translation tasks. We present a novel Knowledge Distillation approach tailored for improving the translation of such languages. Our method involves an encoder, a forward decoder, and two auxiliary decoders: a backward decoder and a morphological decoder. The forward decoder generates target morphemes autoregressively and is augmented by distilling knowledge from the auxiliary decoders. The backward decoder incorporates future context, while the morphological decoder integrates target-side morphological information. We have also designed a reliability estimation method to selectively distill only the reliable knowledge from these auxiliary decoders. Our approach relies on morphological word segmentation. We show that the word segmentation method based on unsupervised morphology learning outperforms the commonly used Byte Pair Encoding method on highly agglutinative languages in translation tasks. Our experiments conducted on English-Tamil, English-Manipuri, and English-Marathi datasets show that our proposed approach achieves significant improvements over strong Transformer-based NMT baselines.

Genetic diversity insights from population genomics and machine learning tools for Nordic Arctic charr (Salvelinus alpinus) populations

Arctic charr (Salvelinus alpinus) is a salmonid species of high ecological and commercial value in the Holarctic region. Nevertheless, more information is needed about its underlying genetic diversity and population structure in the Nordics, especially regarding farmed populations. High-throughput sequencing was applied in three Arctic charr populations of anadromous or landlocked origin from Finland, Norway and Sweden. More specifically, the animals from the Swedish and Norwegian populations originated from a major egg supplier and producer, respectively. Furthermore, in the case of the Finnish population, the sampled animals originated from the only active conservation program for Arctic charr in the country with a potential interest in farming. Using double-digest restriction site-associated DNA sequencing (ddRAD-seq) on more than 500 fish, over 2000 single nucleotide polymorphisms (SNPs), both in the form of individual SNPs and as read haplotypes, were used to study the genetic diversity and structure of those populations. Genetic diversity metrics were similar between the Norwegian and the Swedish populations. However, substantially lower (40–50 %) genetic diversity was found in the Finnish population. Moreover, considerable genetic differentiation was implied between the studied populations as the mean fixation index (FST) was above 0.1 in all pairwise comparisons. All populations were easily discernible through either principal component analysis (PCA) or discriminant analysis of principal components (DAPC). In addition, unsupervised machine learning models such as K-means, Gaussian and Bayesian Gaussian mixtures were assessed for their ability to detect genetic clusters. A preceding dimensionality reduction step by PCA resulted in all three models, suggesting that the most probable number of clusters was three. Overall, our study affirmed the utility of the developed ddRAD-seq genotyping method and unveiled the genetic structure of the studied populations, both of which could contribute to their more efficient management by captive breeding.

LightUrban: Similarity Based Fine‐grained Instancing for Lightweighting Complex Urban Point Clouds

AbstractLarge‐scale urban point clouds play a vital role in various applications, while rendering and transmitting such data remains challenging due to its large volume, complicated structures, and significant redundancy. In this paper, we present LightUrban, the first point cloud instancing framework for efficient rendering and transmission of fine‐grained complex urban scenes. We first introduce a segmentation method to organize the point clouds into individual buildings and vegetation instances from coarse to fine. Next, we propose an unsupervised similarity detection approach to accurately group instances with similar shapes. Furthermore, a fast pose and size estimation component is applied to calculate the transformations between the representative instance and the corresponding similar instances in each group. By replacing individual instances with their group's representative instances, the data volume and redundancy can be dramatically reduced. Experimental results on large‐scale urban scenes demonstrate the effectiveness of our algorithm. To sum up, our method not only structures the urban point clouds but also significantly reduces data volume and redundancy, filling the gap in lightweighting urban landscapes through instancing.

A Variant of the Growing Neural Gas Algorithm for the Design of an Electric Vehicle Charger Network

The Growing Neural Gas (GNG) algorithm constitutes an incremental neural network model based on the idea of a Self-Organizing Map (SOM), that is, unsupervised learning algorithms that reduce the dimensionality of datasets by locating similar samples close to each other. The design of an electric vehicle charging network is an essential aspect in the transition towards more sustainable and environmentally friendly mobility. The need to design and implement an efficient network that meets the needs of all users motivates us to propose the use of a model based on GNG-type neural networks for the design of the network in a specific geographical area. In this paper, a variant of this iterative neural network algorithm is used with the objective that, from an initial dataset of points in the plane, it calculates a new simplified dataset with the main characteristic that the final set of points maintains the geometric shape and topology of the original set. To demonstrate the capabilities of the algorithm, it is exemplified in a real case, in which the design of an electric vehicle charging network is proposed. This network is built by applying the algorithm, taking as the original set of points the ones formed by the nodes of the gas station network in the geographical area studied. Several tests of running the algorithm for different sizes of the final dataset are performed, showing the differences between the original network and the computationally generated one.

Unsupervised Image Segmentation on 2D Echocardiogram

Echocardiography is a widely used, non-invasive imaging technique for diagnosing and monitoring heart conditions. However, accurate segmentation of cardiac structures, particularly the left ventricle, remains a complex task due to the inherent variability and noise in echocardiographic images. Current supervised models have achieved state-of-the-art results but are highly dependent on large, annotated datasets, which are costly and time-consuming to obtain and depend on the quality of the annotated data. These limitations motivate the need for unsupervised methods that can generalize across different image conditions without relying on annotated data. In this study, we propose an unsupervised approach for segmenting 2D echocardiographic images. By combining customized objective functions with convolutional neural networks (CNNs), our method effectively segments cardiac structures, addressing the challenges posed by low-resolution and gray-scale images. Our approach leverages techniques traditionally used outside of medical imaging, optimizing feature extraction through CNNs in a data-driven manner and with a new and smaller network design. Another key contribution of this work is the introduction of a post-processing algorithm that refines the segmentation to isolate the left ventricle in both diastolic and systolic positions, enabling the calculation of the ejection fraction (EF). This calculation serves as a benchmark for evaluating the performance of our unsupervised method. Our results demonstrate the potential of unsupervised learning to improve echocardiogram analysis by overcoming the limitations of supervised approaches, particularly in settings where labeled data are scarce or unavailable.

Detection of hidden drawings using multi-wavelength dynamic speckle, tuneable algorithms, and unsupervised learning

Abstract We implemented an experiment to reveal hidden drawings on papyrus, utilizing an optical technique based on the speckle phenomenon. The goal is to optimize the detection of hidden objects. Our approach proposes using multiple wavelengths for illumination and tuneable algorithms to process the dynamic speckle images. By implementing the suggested method, we generated various results with varying quality, contingent upon the tuneable algorithm parameters. It's feasible to identify the optimal parameter combination to achieve the most effective visualization of the recovered image. To streamline the selection of tuneable algorithms and mitigate reliance on subjective visual judgment, we employed unsupervised machine learning techniques to determine the conditions necessary to achieve optimal results. This approach simplifies the selection procedure and offers an objective and non-invasive method. Furthermore, the proposed procedure holds promise for extending its application to uncover hidden paintings, subsurface archaeological artefacts, and other dynamic speckle experiments.

Data-Driven Approaches in Antimicrobial Resistance: Machine Learning Solutions

Background/Objectives: The emergence of antimicrobial resistance (AMR) due to the misuse and overuse of antibiotics has become a critical threat to global public health. There is a dire need to forecast AMR to understand the underlying mechanisms of resistance for the development of effective interventions. This paper explores the capability of machine learning (ML) methods, particularly unsupervised learning methods, to enhance the understanding and prediction of AMR. It aims to determine the patterns from AMR gene data that are clinically relevant and, in public health, capable of informing strategies. Methods: We analyzed AMR gene data in the PanRes dataset by applying unsupervised learning techniques, namely K-means clustering and Principal Component Analysis (PCA). These techniques were applied to identify clusters based on gene length and distribution according to resistance class, offering insights into the resistance genes’ structural and functional properties. Data preprocessing, such as filtering and normalization, was conducted prior to applying machine learning methods to ensure consistency and accuracy. Our methodology included the preprocessing of data and reduction of dimensionality to ensure that our models were both accurate and interpretable. Results: The unsupervised learning models highlighted distinct clusters of AMR genes, with significant patterns in gene length, including their associated resistance classes. Further dimensionality reduction by PCA allows for clearer visualizations of relationships among gene groupings. These patterns provide novel insights into the potential mechanisms of resistance, particularly the role of gene length in different resistance pathways. Conclusions: This study demonstrates the potential of ML, specifically unsupervised approaches, to enhance the understanding of AMR. The identified patterns in resistance genes could support clinical decision-making and inform public health interventions. However, challenges remain, particularly in integrating genomic data and ensuring model interpretability. Further research is needed to advance ML applications in AMR prediction and management.

Contrastive learning-enabled digital twin framework for fault diagnosis of rolling bearing

Abstract Rolling bearings are essential components in various industrial machines, and their failures can lead to significant downtime and maintenance costs. Traditional data-driven fault diagnosis methods often require extensive fault datasets for training, which may not always be available in critical industrial scenarios, limiting their practicality. Digital twins, virtual representations of physical entities reflecting their operational conditions, offer a promising solution for the fault diagnosis of rolling bearings with limited fault data. In this paper, we propose a novel digital twin-driven framework to address the challenge of limited training data in rolling bearing fault diagnosis. Firstly, a virtual bearing simulation model is used to generate the simulated data. Subsequently, a transformer-based network is introduced to learn the discrepancy features from the raw data. Then, a Maximum Mean Discrepancy loss and a supervised contrastive learning loss for raw and augmentation data are established to achieve global domain alignment and instance-based domain alignment. Finally, an unsupervised contrastive learning loss for the augmentation data of the target domain is established to further improve the diagnostic performance. In five cross-domain fault diagnosis tasks representing real industrial scenarios set in this study, the proposed method achieved an average diagnostic accuracy of 84.40%, exceeding two existing advanced domain adaptation methods by more than 10%. The experimental results demonstrate that the proposed method achieves high diagnostic performance in real industrial scenarios where labeled data is lacking. This shows its significant benefits for monitoring the condition of critical bearings.

Identifying Landslide Hotspots Using Unsupervised Clustering: A Case Study

Landslides pose significant threats to life, property, and infrastructure. This study explores applying unsupervised learning techniques to identify and understand landslide-prone areas. We analyzed topographic data by employing K-Means, Hierarchical Clustering, Spectral Clustering, Mean Shift Clustering, and DBSCAN to uncover hidden patterns in landslide occurrence. Evaluation metrics, including the Silhouette Score, Davies-Bouldin Index, and Calinski-Harabasz Index, were used to assess the performance of these algorithms. Hierarchical Clustering achieved the highest Silhouette Score of 0.635, indicating excellent cluster separation. However, Mean Shift Clustering outperformed the other methods with a superior Davies-Bouldin Index of 0.603 and the highest Calinski-Harabasz Index of 4121.75, demonstrating the best overall clustering performance. DBSCAN also performed well, with a Silhouette Score of 0.610 and 12 noise points identified. These findings contribute to a deeper understanding of landslide spatial distribution and can inform the development of effective early warning systems and mitigation strategies.

A multi-paradigm approach to characterize dominance behaviors in CD1 and C57BL6 male mice.

Social status and dominance are critical factors influencing well-being and survival across multiple species. However, dominance behaviors vary widely across species, from elaborate feather displays in birds to aggression in chimps. To effectively study dominance, it is essential to clearly define and reliably measure dominance behaviors. In laboratory settings, C57BL/6 mice are commonly used to study dominance due to their stable and linear social hierarchies. However, other mouse strains are also used for laboratory research. Despite substantial evidence for strain effects on behavioral repertoires, the impact of strain on dominance in mice remains largely unstudied. To address this gap, we compared dominance behaviors between CD1 and C57BL/6 male mice across four assays: observation of agonistic behaviors, urine marking, tube test, and a reward competition. We found that CD1 mice demonstrate increased fighting, increased territorial marking through urination, and increased pushing and resisting in the tube test. We used unsupervised machine learning and pose estimation data from the reward competitions to uncover behavioral differences across strains and across rank differences between competing pairs. Of the four assays, urine marking and agonistic behaviors showed the strongest correlation with dominance in both strains. Most notably, we found that CD1 dominance rankings based on the tube test negatively correlated with rankings from all three other assays, suggesting that the tube test may measure a different behavior in CD1 mice. Our results highlight that behaviors can be strain-specific in mice and studies that measure social rank should consider assays carefully to promote reproducibility.Significance Statement Recent studies have highlighted that social dominance can significantly impact behavior and the brain. As such, accurately measuring dominance behavior in laboratory settings is crucial in neuroscience research. In this study, we investigated the consistency of four dominance assays for male mice across two common mouse strains. We find that not all assays result in the same dominance rankings and dominance behaviors differ across strains. Our study sheds light on the importance of considering strains for assay selection, rigor, and reproducibility.

Automatic detection of water pipeline leakage via unsupervised learning applied on ground penetrating radar data

ABSTRACT Aging urban water supply pipelines are facing significant leakage issues. Ground penetrating radar (GPR) is an effective non-destructive method for leakage localization, but its image analysis mainly relies on interpreter experience, leading to uncertainty and inefficiency. To address this issue, an unsupervised learning method based on GPR data attributes is proposed for automatic leakage detection. First, velocity and energy attributes are extracted from B-Scan profiles to differentiate wet areas caused by leakage from normal areas. Then, the density-based spatial clustering of applications with noise (DBSCAN) clustering method is applied to automatically classify these two types of areas. The proposed method was evaluated on an experimental platform with constant water pressure and pre-drilled leakage points. Experiments show that the velocity and energy attributes decrease by 25.2 and 26.6% after leakage. Using velocity attributes yields better results, and suggestions for DBSCAN hyperparameters are provided. Feature importance analysis indicates that two-way-travel-time with a score of 46.1 and energy attribution index with a score of 32.7 significantly influence leakage identification. This method can also estimate affected leakage areas with an error not exceeding the spacing of three measurement lines, and is available across various underground conditions without needing well-labeled training data.

Machine Learning-Based Optimization Models for Defining Storage Rules in Maritime Container Yards

This paper proposes an integrated approach to define the best consignment strategy for storing containers in an export yard of a maritime terminal. The storage strategy identifies the rules for grouping homogeneous containers, which are defined simultaneously with the assignment of each group of containers to the available blocks (bay-locations) in the yard. Unlike recent literature, this study focuses specifically on weight classes and their respective limits when establishing the consignment strategy. Another novel aspect of this work is the integration of a data-driven algorithm and operations research. The integrated approach is based on unsupervised learning and optimization models and allows us to solve large instances within a few seconds. Results obtained by spectral clustering are treated as input datasets for the optimization models. Two different formulations are described and compared: the main difference lies in how containers are assigned to bay-locations, shifting from a time-consuming individual container assignment to the assignment of groups of containers, which offers significant advantages in computational efficiency. Experimental tests are organized into three campaigns to evaluate the following: (i) The computational time and solution quality (i.e., space utilization) of the proposed models; (ii) The performance of these models against a benchmark model; (iii) The practical effectiveness of the proposed solution approach.

Surface Profile Recovery from Electromagnetic Fields with Physics-Informed Neural Networks

Physics-informed neural networks (PINN) have shown their potential in solving both direct and inverse problems of partial differential equations. In this paper, we introduce a PINN-based deep learning approach to reconstruct one-dimensional rough surfaces from field data illuminated by an electromagnetic incident wave. In the proposed algorithm, the rough surface is approximated by a neural network, with which the spatial derivatives of surface function can be obtained via automatic differentiation, and then the scattered field can be calculated using the method of moments. The neural network is trained by minimizing the loss between the calculated and the observed field data. Furthermore, the proposed method is an unsupervised approach, independent of any surface data, where only the field data are used. Both transverse electric (TE) field (Dirichlet boundary condition) and transverse magnetic (TM) field (Neumann boundary condition) are considered. Two types of field data are used here: full-scattered field data and phaseless total field data. The performance of the method is verified by testing with Gaussian-correlated random rough surfaces. Numerical results demonstrate that the PINN-based method can recover rough surfaces with great accuracy and is robust with respect to a wide range of problem regimes.