Unsupervised Method Research Articles

DGSIST: Clustering spatial transcriptome data based on deep graph structure Infomax

Although spatial transcriptomics data provide valuable insights into gene expression profiles and the spatial structure of tissues, most studies rely solely on gene expression information, underutilizing the spatial data. To fully leverage the potential of spatial transcriptomics and graph neural networks, the DGSI (Deep Graph Structure Infomax) model is proposed. This innovative graph data processing model uses graph convolutional neural networks and employs an unsupervised learning approach. It maximizes the mutual information between graph-level and node-level representations, emphasizing flexible sampling and aggregation of nodes and their neighbors. This effectively captures and incorporates local information from nodes into the overall graph structure. Additionally, this paper developed the DGSIST framework, an unsupervised cell clustering method that integrates the DGSI model, SVD dimensionality reduction algorithm, and k-means++ clustering algorithm. This aims to identify cell types accurately. DGSIST fully uses spatial transcriptomics data and outperforms existing methods in accuracy. Demonstrations of DGSIST’s capability across various tissue types and technological platforms have shown its effectiveness in accurately identifying spatial domains in multiple tissue sections. Compared to other spatial clustering methods, DGSIST excels in cell clustering and effectively eliminates batch effects without needing batch correction. DGSIST excels in spatial clustering analysis, spatial variation identification, and differential gene expression detection and directly applies to graph analysis tasks, such as node classification, link prediction, or graph clustering. Anticipation lies in the contribution of the DGSIST framework to a deeper understanding of the spatial organizational structures of diseases such as cancer.

An unsupervised transfer learning gear fault diagnosis method based on parameter-optimized VMD and residual attention networks

An unsupervised transfer learning gear fault diagnosis method based on parameter-optimized VMD and residual attention networks

Acquiring better load estimates by combining anomaly and change point detection in power grid time-series measurements

In this paper we present novel methodology for automatic anomaly and switch event filtering to improve load estimation in power grid systems. By leveraging unsupervised methods with supervised optimization, our approach prioritizes interpretability while ensuring robust and generalizable performance on unseen data. Through experimentation, a combination of binary segmentation for change point detection and statistical process control for anomaly detection emerges as the most effective strategy, specifically when ensembled in a novel sequential manner. Results indicate the clear wasted potential when filtering is not applied. The automatic load estimation is also fairly accurate, with approximately 90% of estimates falling within a 10% error margin, with only a single significant failure in both the minimum and maximum load estimates across 60 measurements in the test set. Our methodology’s interpretability makes it particularly suitable for critical infrastructure planning, thereby enhancing decision-making processes.

Machine learning applications of network security enhancement: review

Machine learning (ML) is being used to improve intrusion detection mechanisms and identification in cyber security. Network data volume scaling (with the help of Machine learning) — Automated analysis and pattern recognition for large amounts of network-data, thereby detection of anomalies / potentially malicious activities that escape current rule-based techniques. By training ML models on historical data, these models can learn benign network behavior as well the anomalies in them that may result from malicious activities. The purpose of this essay/report is to begin taking a look under the hood at how ML can be used for security threat detection and analysis in-networking on an ongoing basis. This paper covers the automation of ML algorithms to enhance network security. Given the current state of electronic threats and their evolution, traditional security methods are typically insufficient. ML can analyze large volumes of data, learn patterns from it and this makes it suitable to complement network defense mechanisms. It then discusses different ML applications in network cybersecurity: intrusion detection, anomaly detection, spam and malware analysis; which the paper characterizes. It analyzes the potential, benefits and constrains of major ML methods in network security like supervised learning; unsupervised learning and reinforcement-learning. Finally, this paper represents recent progress in the use and impact of ML techniques along with case studies.The paper discusses the existing difficulties in the field such as the necessity for datasets and the vulnerability of machine learning models to adversarial attacks.T he paper also highlights avenues for exploration by focusing on developing scalable security solutions based on machine learning that are resilient and flexible.The goal of this examination is to offer both researchers and industry professionals valuable perspectives into the opportunities and obstacles linked to utilizing machine learning, in the domain of network security. ML methods have potential to improve network security by addressing the challenges posed by the increasing cyber threats that traditional security measures struggle to combat effectively over time. One key strength of ML lies in its capacity to analyze datasets and identify intricate patterns efficiently. The research paper delves into applications of ML in enhancing network security. The list covers security tools like Intrusion Detection Systems (IDS) examining malware and phishing attempts as well as anomalies in network activity and user behavior analysis (UEBA). The study explores both supervised and unsupervised learning methods. How they are used for quick threat detection and response in real time scenarios.You will find case studies and recent developments that showcase the implementation and effectiveness of these strategies.In addition the article delves into the obstacles linked to using machine learning techniques in network security including the necessity, for datasets, The paper's goal is to give an enlightening summary to scholars and practitioners about how machine learning can be applied to network security in order to provide solutions that are robust, adaptive, and scalable. To this end, it touches on several relevant aspects. One is the threat posed by adversarial attacks on the sorts of models that are likely to be used in this context. Another is the imperative, deriving from both adversarial threat and model drift, that models needed in this context be available in a form usable for continuous update. Keywords: Network Security, machine learning (ML), Intrusion Detection Systems (IDS), entity behavior analytics (UEBA).

Longitudinal Data Clustering Methods: A Systematic ReviewLongitudinal Data Clustering Methods: A Systematic Review

Introduction: In the last few decades, in many research fields, different methods were introduced to discover groups with the same trends in longitudinal data. The clustering process is an unsupervised learning method, which classifies longitudinal data based on different criteria by performing algorithms. The current study was performed with the aim of reviewing various methods of longitudinal data clustering, including two general categories of non-parametric methods and model-based methods. Methods: In this research, to obtain related scientific articles, PubMed, Science Direct Scopus, ISI, and Google Scholar were searched between 2000 and 2021. Results: According to our systematic review, the non-parametric k-means Clustering Method utilizing Euclidean distance emerges as a leading approach for clustering longitudinal data. Conclusion: This research, with an overview of the studies done in the field of clustering, can help researchers as a toolbox to choose various methods of longitudinal data clustering in idea generation and choosing the appropriate method in the classification and analysis of longitudinal data.

Fractional Derivative to Symmetrically Extend the Memory of Fuzzy C-Means

The fuzzy C-means (FCM) clustering algorithm is a widely used unsupervised learning method known for its ability to identify natural groupings within datasets. While effective in many cases, FCM faces challenges such as sensitivity to initial cluster assignments, slow convergence, and difficulty in handling non-linear and overlapping clusters. Aimed at these limitations, this paper introduces a novel fractional fuzzy C-means (Frac-FCM) algorithm, which incorporates fractional derivatives into the FCM framework. By capturing non-local dependencies and long memory effects, fractional derivatives offer a more flexible and precise representation of data relationships, making the method more suitable for complex datasets. Additionally, a genetic algorithm (GA) is employed to optimize a new least-squares objective function that emphasizes the geometric properties of clusters, particularly focusing on the Fukuyama–Sugeno and Xie–Beni indices, thereby enhancing the balance between cluster compactness and separation. Furthermore, the Frac-FCM algorithm is evaluated on several benchmark datasets, including Iris, Seed, and Statlog, and compared against traditional methods like K-means, SOM, GMM, and FCM. The results indicate that Frac-FCM consistently outperforms these methods in terms of the Silhouette and Dunn indices. For instance, Frac-FCM achieves higher Silhouette scores of most cases, indicating more distinct and well-separated clusters. Dunn’s index further shows that Frac-FCM generates clusters that are better separated, surpassing the performance of traditional methods. These findings highlight the robustness and superior clustering performance of Frac-FCM. The Friedman test was employed to enhance and validate the effectiveness of Frac-FCM.

Unsupervised Multi-Scale Hybrid Feature Extraction Network for Semantic Segmentation of High-Resolution Remote Sensing Images

Generating pixel-level annotations for semantic segmentation tasks of high-resolution remote sensing images is both time-consuming and labor-intensive, which has led to increased interest in unsupervised methods. Therefore, in this paper, we propose an unsupervised multi-scale hybrid feature extraction network based on the CNN-Transformer architecture, referred to as MSHFE-Net. The MSHFE-Net consists of three main modules: a Multi-Scale Pixel-Guided CNN Encoder, a Multi-Scale Aggregation Transformer Encoder, and a Parallel Attention Fusion Module. The Multi-Scale Pixel-Guided CNN Encoder is designed for multi-scale, fine-grained feature extraction in unsupervised tasks, efficiently recovering local spatial information in images. Meanwhile, the Multi-Scale Aggregation Transformer Encoder introduces a multi-scale aggregation module, which further enhances the unsupervised acquisition of multi-scale contextual information, obtaining global features with stronger feature representation. The Parallel Attention Fusion Module employs an attention mechanism to fuse global and local features in both channel and spatial dimensions in parallel, enriching the semantic relations extracted during unsupervised training and improving the performance of unsupervised semantic segmentation. K-means clustering is then performed on the fused features to achieve high-precision unsupervised semantic segmentation. Experiments with MSHFE-Net on the Potsdam and Vaihingen datasets demonstrate its effectiveness in significantly improving the accuracy of unsupervised semantic segmentation.

An Analytical Approach for IGBT Life Prediction Using Successive Variational Mode Decomposition and Bidirectional Long Short-Term Memory Networks

The precise estimation of the operational lifespan of insulated gate bipolar transistors (IGBT) holds paramount significance for ensuring the efficient and uncompromised safety of industrial equipment. However, numerous methodologies and models currently employed for this purpose often fall short of delivering highly accurate predictions. The analytical approach that combines the Pattern Optimization Algorithm (POA) with Successive Variational Mode Decomposition (SVMD) and Bidirectional Long Short-term Memory (BiLSTM) network is introduced. Firstly, SVMD is employed as an unsupervised feature learning method to partition the data into intrinsic modal functions (IMFs), which are used to eliminate noise and preserve the essential signal. Secondly, the BiLSTM network is integrated for supervised learning purposes, enabling the prediction of the decomposed sequence. Additionally, the hyperparameters of BiLSTM and the penalty coefficients of SVMD are optimized utilizing the POA technique. Subsequently, the various modal functions are predicted utilizing the trained prediction model, and the individual mode predictions are subsequently aggregated to yield the model’s definitive final life prediction. Through case studies involving IGBT aging datasets, the optimal prediction model was formulated and its lifespan prediction capability was validated. The superiority of the proposed method is demonstrated by comparing it with benchmark models and other state-of-the-art methods.

Interpretable multi-omics integration with UMAP embeddings and density-based clustering.

Integrating high-dimensional cellular multi-omics data is crucial for understanding various layers of biological control. Single 'omic methods provide important insights, but often fall short in handling the complex relationships between genes, proteins, metabolites and beyond. Here, we present a novel, non-linear, and unsupervised method called GAUDI (Group Aggregation via UMAP Data Integration) that leverages independent UMAP embeddings for the concurrent analysis of multiple data types. GAUDI uncovers non-linear relationships among different omics data better than several state-of-the-art methods. This approach not only clusters samples by their multi-omic profiles but also identifies latent factors across each omics dataset, thereby enabling interpretation of the underlying features contributing to each cluster. Consequently, GAUDI facilitates more intuitive, interpretable visualizations to identify novel insights and potential biomarkers from a wide range of experimental designs.

ORCO: Ollivier-Ricci Curvature-Omics - an unsupervised method for analyzing robustness in biological systems.

Although recent advanced sequencing technologies have improved the resolution of genomic and proteomic data to better characterize molecular phenotypes, efficient computational tools to analyze and interpret the large-scale omic data are still needed. To address this, we have developed a network-based bioinformatic tool called Ollivier-Ricci curvature-omics (ORCO). ORCO incorporates gene interaction information with omic data into a biological network, and computes Ollivier-Ricci curvature (ORC) values for individual interactions. ORC, an edge-based measure, indicates network robustness and captures global gene signaling changes in functional cooperation using a consistent information passing measure, thereby helping identify therapeutic targets and regulatory modules in biological systems. This tool can be applicable to any data that can be represented as a network. ORCO is an open-source Python package and publicly available on GitHub at https://github.com/aksimhal/ORC-Omics .

Unsupervised Learning for the Automatic Counting of Grains in Nanocrystals and Image Segmentation at the Atomic Resolution.

Identifying the grain distribution and grain boundaries of nanoparticles is important for predicting their properties. Experimental methods for identifying the crystallographic distribution, such as precession electron diffraction, are limited by their probe size. In this study, we developed an unsupervised learning method by applying a Gabor filter to HAADF-STEM images at the atomic level for image segmentation and automatic counting of grains in polycrystalline nanoparticles. The methodology comprises a Gabor filter for feature extraction, non-negative matrix factorization for dimension reduction, and K-means clustering. We set the threshold distance and angle between the clusters required for the number of clusters to converge so as to automatically determine the optimal number of grains. This approach can shed new light on the nature of polycrystalline nanoparticles and their structure-property relationships.

Supervised kernel principal component analysis-polynomial chaos-Kriging for high-dimensional surrogate modelling and optimization

Surrogate-based optimization (SBO) approach is becoming more and more popular in the expensive aerodynamic design of aircraft. However, with increasing number of design variables required for parameterizing a complex shape, SBO is suffering from the serious difficulty of the curse of dimensionality. To ameliorate this issue, a supervised nonlinear dimensionality-reduction surrogate modelling method was proposed. Such a method combines the supervised kernel principal component analysis (SKPCA) and polynomial chaos-Kriging (PCK) techniques into the jointly surrogate modelling process and adaptively establishes the accurate mapping from the high-dimensional inputs to the output of the system. This SKPCA-PCK method, which fully considers the effect of inputs on outputs and adaptively trains these hyper-parameters in the surrogate modelling process, escapes from the low prediction accuracy and instability of the surrogate model in conjunction with current linear or unsupervised dimensionality-reduction methods. Further, an efficient SKPCA-PCK-based global optimization method for high-dimensional aerodynamic design was developed. The performance of the proposed method is examined by investigating two numerical examples, the transonic RAE2822 airfoil and the wing of the NASA Common Research Model. Results demonstrate that the proposed SKPCA-PCK method significantly improves the modelling efficiency and accuracy compared to the unsupervised linear PCA-Kriging method. More importantly, the proposed SKPCA-PCK-based optimization method provides better performance and an appreciably higher optimization efficiency for expensive single-point and robust aerodynamic design involving high-dimensional design variables compared to the Kriging-based optimization method. These results provide further evidence that the proposed method provides a promising approach for mitigating the curse of dimensionality in SBO.

Enhancing healthcare data integrity: fraud detection using unsupervised learning techniques

Data in healthcare forms the backbone of any treatment and decision-making for patients. However, the data from a healthcare institution can sometimes be prone to abnormalities, thereby putting treatment and patient safety in jeopardy. This paper points to the dire need for reliable anomaly detection systems in the healthcare industry. It employs various unsupervised learning methods, including Isolation Forest, Local Outlier Factor (LOF), K-Nearest Neighbours (KNN), and autoencoder models for detecting abnormalities in healthcare data with better accuracy. Anomaly detection capabilities also allow health providers to reduce risks and provide some assurance of the integrity of the data, as these capabilities are more likely to indicate unusual patient profiles or incorrect test results. Isolation Forest, LOF, and KNN are the preliminary methods for performing anomaly detection in this work, with Isolation Forest yielding the best results. Then, autoencoder models that learn subtle variations and complex patterns in data are employed. This paper seeks to enhance anomaly detection in terms of accuracy and reliability to ensure better healthcare data quality and patient safety.

Classifying adaxial and abaxial sides of diseased citrus leaves with selected hyperspectral bands and YOLOv8

Huanglongbing (HLB; citrus greening) and citrus canker are invasive diseases afflicting Florida citrus groves, causing financial losses through yield loss, smaller fruit, blemishes, premature fruit drop and/or eventual tree death. Often, symptoms of these diseases resemble those of other disease or disorders. Early detection of HLB and canker via in-grove leaf inspection can permit more effective mitigation tactics and management of groves. Autonomous, vision-based disease scouting in a grove offers a financial benefit to the Florida citrus industry. This study investigates the potential of hyperspectral reflectance imagery (HSI) for detecting and classifying these conditions in the presence of other, less consequential leaf defects. Leaves with visible symptoms of HLB, canker, zinc deficiency, scab, melanose, greasy spot, and a control set were collected and both sides were imaged with a line-scan hyperspectral camera. Spectral bands from this imagery were selected using two methods: an unsupervised method based on principal component analysis (PCA), a supervised method based on linear discriminant analysis (LDA), which were compared with randomly selected bands as a control. The YOLOv8 network architecture was trained to classify each side of these leaves with each band combination. LDA-selected bands from the front of the leaves yielded an overall classification accuracy of 87.09 %, with higher recall of melanose and precision of control than any other model tested. Leaves with HLB and zinc deficiency were classified most accurately, with both band selection methods yielding F1 scores of at least 0.955 and 0.934, respectively. These findings favor the use of supervised band selection for HSI-based in-grove disease detection.

An unsupervised approach for the detection of zero‐day distributed denial of service attacks in Internet of Things networks

AbstractThe authors introduce an unsupervised Intrusion Detection System designed to detect zero‐day distributed denial of service (DDoS) attacks in Internet of Things (IoT) networks. This system can identify anomalies without needing prior knowledge or training on attack information. Zero‐day attacks exploit previously unknown vulnerabilities, making them hard to detect with traditional deep learning and machine learning systems that require pre‐labelled data. Labelling data is also a time‐consuming task for security experts. Therefore, unsupervised methods are necessary to detect these new threats. The authors focus on DDoS attacks, which have recently caused significant financial and service disruptions for many organisations. As IoT networks grow, these attacks become more sophisticated and harmful. The proposed approach detects zero‐day DDoS attacks by using random projection to reduce data dimensionality and an ensemble model combining K‐means, Gaussian mixture model, and one‐class SVM with a hard voting technique for classification. The method was evaluated using the CIC‐DDoS2019 dataset and achieved an accuracy of 94.55%, outperforming other state‐of‐the‐art unsupervised learning methods.

Anomaly Sign Detection for Automatic Ticket Gates by the Histogram Limitation Method

It is crucial to appropriately maintain automatic ticket gates (ATGs) to keep transportation operating smoothly in urban areas. Although the average failure rate of new ATGs is extremely low, continuous operation for many years might lead to unstable performance due to deterioration, and the need for periodic maintenance to avoid fatal faults might halt operations for extended periods. To detect anomalies at an early stage, “anomaly signs” can be utilized to flag ATGs for maintenance by service engineers before anomalies occur. In addition, to minimize the cost of ATG monitoring, the necessary computing resources should be minimized, which means using only light-weight statistical methods rather than deep learning or machine learning. In this paper, we focus on the automatic separation modules inside ATGs that separate multiple tickets by complicated mechatronic controls because this module is the major cause of maintenance calls from station attendants. We propose a simple anomaly sign detection, called the histogram limitation method (HLM). We evaluated the anomaly sign scores over time with maintenance timing and compared them with the conventional fast unsupervised anomaly detection method, Histogram-Based Outlier Score (HBOS) widely used in various domains. The experimental results using real field ATG monitoring data show that HLM successfully detected anomaly signs before a maintenance call was necessary, which is better performance compared with HBOS. Despite being a simple modification based on HBOS, HLM also provides anomaly sign scores that agree adequately with assessments by maintenance service engineers.

Unsupervised detection of coordinated fake-follower campaigns on social media

Automated social media accounts, known as bots, are increasingly recognized as key tools for manipulative online activities. These activities can stem from coordination among several accounts and these automated campaigns can manipulate social network structure by following other accounts, amplifying their content, and posting messages to spam online discourse. In this study, we present a novel unsupervised detection method designed to target a specific category of malicious accounts designed to manipulate user metrics such as online popularity. Our framework identifies anomalous following patterns among all the followers of a social media account. Through the analysis of a large number of accounts on the Twitter platform (rebranded as X after the acquisition of Elon Musk), we demonstrated that irregular following patterns are prevalent and are indicative of automated fake accounts. Notably, we found that these detected groups of anomalous followers exhibited consistent behavior across multiple accounts. This observation, combined with the computational efficiency of our proposed approach, makes it a valuable tool for investigating large-scale coordinated manipulation campaigns on social media platforms.

Unsupervised Change Detection Methods Applied to Landslide Mapping: Case Study in São Sebastião, Brazil

ABSTRACTLandslides represent a growing global geological hazard, further intensified by climate‐induced changes. Remote sensing data, through its capacity for repetitive collection and change detection techniques, that compare and quantify the spatio‐temporal alterations over time, plays a critical role in landslide detection. Considering the February 2023 São Sebastião event and Sentinel‐2 imagery, we assessed diverse unsupervised change detection techniques, encompassing both traditional and recent machine learning‐based approaches. Notably, the Floating References (FR) and Homogeneous Blocks Single‐class Classification (HBSC) methods outperform classic approaches and deliver the most accurate results with F1‐Score and kappa coefficient exceeding 0.96 and 0.92, respectively. These outcomes demonstrate the efficacy of machine learning in automating landslide delineation and underscore the necessity of meticulous data and parameter selection in achieving high‐accuracy automatic landslide mapping. Lastly, this study fills a significant gap in the existing literature by evaluating unsupervised change detection methods for landslide mapping within the Brazilian context.

Latent personality profiles of analog astronauts: An unsupervised clustering method analysis

The selection of astronauts for space missions is an important process that involves the assessment of multiple factors, including personality traits. Previous studies have corroborated that these traits exert a significant influence on the performance and adaptation of the crew during the execution of space missions. In this study, International Personality Item Pool - NEO – 120 (IPIP-120) was used to assess the personality of participants in a sample of 43 analog astronauts who took part in 9 different analog missions between 2022 and 2023. The objective was to identify specific personality profiles among the analog astronauts using the unsupervised k-means clustering method. The centroids of each cluster were analyzed to describe distinctive personality profiles. The results revealed the existence of two distinct personality clusters among the analog astronauts. The first cluster showed a greater propensity toward adventure, intellect, goal striving, extroversion, and cooperation. In contrast, the second cluster exhibited lower scores on these traits. The identification of these two differentiated personality profiles reinforces the importance of personality assessment as a valuable tool in the astronaut selection process. Furthermore, the application of the k-means method demonstrates its utility as an effective technique for identifying underlying patterns in complex datasets. These findings may inform future astronaut selection strategies, contributing to the formation of more effective and cohesive teams for space missions.

Unsupervised Hyperspectral and Multispectral Image Blind Fusion Based on Deep Tucker Decomposition Network With Spatial-Spectral Manifold Learning.

Hyperspectral image (HSI) and multispectral image (MSI) fusion aims to generate high spectral and spatial resolution hyperspectral image (HR-HSI) by fusing high-resolution multispectral image (HR-MSI) and low-resolution hyperspectral image (LR-HSI). However, existing fusion methods encounter challenges such as unknown degradation parameters, and incomplete exploitation of the correlation between high-dimensional structures and deep image features. To overcome these issues, in this article, an unsupervised blind fusion method for LR-HSI and HR-MSI based on Tucker decomposition and spatial-spectral manifold learning (DTDNML) is proposed. We design a novel deep Tucker decomposition network that maps LR-HSI and HR-MSI into a consistent feature space, achieving reconstruction through decoders with shared parameters. To better exploit and fuse spatial-spectral features in the data, we design a core tensor fusion network (CTFN) that incorporates a spatial-spectral attention mechanism for aligning and fusing features at different scales. Furthermore, to enhance the capacity to capture global information, a Laplacian-based spatial-spectral manifold constraint is introduced in shared-decoders. Sufficient experiments have validated that this method enhances the accuracy and efficiency of hyperspectral and multispectral fusion on different remote sensing datasets. The source code is available at https://github.com/Shawn-H-Wang/DTDNML.