Unsupervised Learning Algorithm Research Articles

Integrating Yolo V5 Analysis and KNN to Improve Lung Cancer Detection

Lung cancer is a leading cause of cancer-related mortality globally, emphasizing the urgent need for early detection and accurate diagnosis. This project aims to leverage advanced deep learning techniques, specifically YOLO-v5 (You Only Look Once) for object detection, and the k-Nearest Neighbors (kNN) algorithm for unsupervised learning, to enhance the detection and analysis of lung cancer from CT scan images. YOLO-v5, known for its exceptional speed and accuracy in detecting objects within images, will be used to identify and localize lung nodules, which are potential indicators of lung cancer. Simultaneously, we will employ the kNN algorithm in a novel application of unsupervised learning to cluster CT scan images based on the similarity of detected lung tumors, enabling the identification of patterns and characteristics that may correlate with specific types of lung cancer. This project involves collecting and preprocessing a diverse dataset of CT images annotated with radiologist insights to train the YOLO-v5 model. Subsequently, the kNN algorithm will be applied to perform clustering on the detected tumors. By achieving high accuracy in nodule detection and effectively clustering similar tumors, the system aims to become an invaluable tool for radiologists, providing rapid diagnostic assistance and facilitating a deeper understanding of lung cancer characteristics.

Deep embedded clustering of physical behaviour in patients with an implanted cardioverter-defibrillator identifies high and low risk profiles

Abstract Introduction Patients at high risk of sudden cardiac death are treated with an implantable cardioverter-defibrillator (ICD). Patterns in physical behaviour may be prognostic for clinical deterioration. Deep embedded clustering (DEC) employs deep neural networks to learn compact and meaningful representations from high-dimensional data, on which an unsupervised clustering then operates. Purpose The objectives of this study were to use DEC to identify and describe behavioural profiles in a prospective ICD cohort and assess their risk of clinical outcomes. Methods Data from the multicenter, prospective SafeHeart study conducted at our 2 centers, was used. Patients received an implantable cardioverter defibrillator (ICD) between May 2021 and September 2022, who then wore wearable devices during 6 months that capture physical behaviour. Latent representations were extracted from the time-series behavioural data using a variational autoencoder, and used as input to an unsupervised machine learning algorithm (Figure 1). Kaplan-Meier survival curves were used to visualise the annual risk of appropriate ICD-therapy, and the composite of all ICD-therapy and death. Results A total of 272 patients (mean age of 63.1 ±10.2 years, 81% male) were eligible, with a total of 37478 days of behavioural data (138 ±47 days per patient). DEC identified five distinct behavioural profiles: Cluster A (n=70) had high activity levels, mainly at light-to-moderate intensity. Cluster B (n=51) showed above-average sleep efficiency. Cluster C (n=63) exhibited a high-intensity activity profile. Cluster D (n=42) had frequent waking episodes and poor sleep. Cluster E (n=46) had very low physical activity levels and an impaired sleep architecture. Significantly differences were observed between clusters A-E in ICD-therapy and mortality (log-rank p-value 0.04) (Figure 2). Clusters B and D exhibited a threefold decrease in the annual risk of appropriate ICD therapy compared to cluster E. Conclusion DEC reveals distinct behavioural profiles in a prospective ICD cohort, with their inherent risk on outcomes. Wearable accelerometer devices enable behavioural monitoring that may provide opportunities for improved personalised risk prediction in a notoriously heterogeneous high-risk patient population.Deep embedded clustering of behaviourEvent rates stratified for clusters

Forecasting Criminal Activity Using Machine Learning Approaches

Predicting criminal activity has long been a challenge for law enforcement agencies worldwide. Traditional methods often rely on historical data and human intuition, which may be limited in their accuracy and scope. In recent years, machine learning techniques have emerged as promising tools for forecasting criminal activity by leveraging large-scale datasets and advanced algorithms. This paper presents a novel machine learning approach to forecasting criminal activity, focusing on the development and evaluation of predictive models using various data sources, including crime reports, demographic information, and environmental factors. We explore the application of supervised and unsupervised learning algorithms, such as decision trees, random forests, support vector machines, and neural networks, to identify patterns and trends in crime data. Furthermore, we discuss the challenges and ethical considerations associated with deploying predictive models in real-world law enforcement settings, emphasizing the importance of transparency, fairness, and accountability. Through empirical analysis and case studies, we demonstrate the potential of machine learning techniques to enhance crime prediction and prevention efforts, providing valuable insights for policymakers, law enforcement agencies, and researchers in the field of criminal justice.

Self-Adaptive Incremental PCA-Based DBSCAN of Acoustic Features for Anomalous Sound Detection

In modern industry, maintaining continuous machine operations is important for improving production efficiency and reducing costs. Therefore, the smart technology of acoustic monitoring to detect anomalous machine conditions earlier before breakdowns works as part of predictive maintenance and is applied not only in industry fault detection but also in safety monitoring and surveillance systems. This paper proposes a self-adaptive unsupervised machine learning algorithm with dimension-reduction technology to detect anomalous sounds after extracting acoustic machine features. Technically, the automatic EPS calculation algorithm-based genetic algorithm optimizes the automatic clustering algorithm’s configuration for incremental principal component analysis and density-based spatial clustering algorithms with noise. IPCA is enhanced by the sequential Karhunen–Loeve (SKL) algorithm, and the condensation algorithm works as the second layer of the algorithm to reduce the number of effective components. This architecture could select an optimized set of parameters based on different test environments and keeps performance quality with fewer computational requirements. In the experiments, 228 sets of normal sounds and 100 sets of anomaly sounds are used. The sound files are collected from the same machine type (stepper motors) at a real plant site. We compare the proposed algorithm with K-means++, one-class SVM, agglomerative clustering, DCGAN and DCNN-Autoencoder, and this new algorithm performs best, with an AUC of 0.84 and the shortest execution time. The algorithm is generic and can be applied to detect anomalies in machines to provide early warning to people to avoid serious accidents or disasters.

Using machine learning to discover traumatic brain injury patient phenotypes: national concussion surveillance system Pilot

ABSTRACT Objective The objective is to determine whether unsupervised machine learning identifies traumatic brain injury (TBI) phenotypes with unique clinical profiles. Methods Pilot self-reported survey data of over 10,000 adults were collected from the Centers for Disease Control and Prevention (CDC)’s National Concussion Surveillance System (NCSS). Respondents who self-reported a head injury in the past 12 months (n = 1,364) were retained and queried for injury, outcome, and clinical characteristics. An unsupervised machine learning algorithm, partitioning around medoids (PAM), that employed Gower’s dissimilarity matrix, was used to conduct a cluster analysis. Results PAM grouped respondents into five TBI clusters (phenotypes A-E). Phenotype C represented more clinically severe TBIs with a higher prevalence of symptoms and association with worse outcomes. When compared to individuals in Phenotype A, a group with few TBI-related symptoms, individuals in Phenotype C were more likely to undergo medical evaluation (odds ratio [OR] = 9.8, 95% confidence interval[CI] = 5.8–16.6), have symptoms that were not currently resolved or resolved in 8+ days (OR = 10.6, 95%CI = 6.2–18.1), and more likely to report at least moderate impact on social (OR = 54.7, 95%CI = 22.4–133.4) and work (OR = 25.4, 95%CI = 11.2–57.2) functioning. Conclusion Machine learning can be used to classify patients into unique TBI phenotypes. Further research might examine the utility of such classifications in supporting clinical diagnosis and patient recovery for this complex health condition.

Enhanced Bearing Fault Diagnosis Through Trees Ensemble Method and Feature Importance Analysis

Abstract Purpose This research introduces a groundbreaking method for bearing defect detection. It leverages ensemble machine learning (ML) models and conducts comprehensive feature importance analysis. The key innovation is the training and benchmarking of three tree ensemble models—Decision Tree (DT), Random Forest (RF), and Extreme Gradient Boosting (XGBoost)—on an extensive experimental dataset (QU-DMBF) collected from bearing tests with seeded defects of varying sizes on the inner and outer raceways under different operating conditions. Method The dataset was meticulously prepared with categorical variable encoding and Min–Max data normalization to ensure consistent class distribution and model accuracy. Implementing the ML models involved a grid search method for hyperparameter tuning, focusing on reporting the models’ accuracy. The study also explores applying ensemble methods and using supervised and unsupervised learning algorithms for bearing fault detection. It underscores the value of feature importance analysis in understanding the contributions of specific inputs to the model’s performance. The research compares the ML models to traditional methods and discusses their potential for advanced fault diagnosis in bearing systems. Results and Conclusions The XGBoost model, trained on data from actual bearing tests, outperformed the others, achieving 92% accuracy in detecting bearing health and fault location. However, a deeper analysis of feature importance reveals that the models weigh certain experimental conditions differently—such as sensor location and motor speed. This research’s primary novelties and contributions are comparative evaluation, experimental validation, accuracy benchmarking, and interpretable feature importance analysis. This comprehensive methodology advances the bearing health monitoring field and has significant practical implications for condition-based maintenance, potentially leading to substantial cost savings and improved operational efficiency.

Anomaly Detection in Blockchain Networks Using Unsupervised Learning: A Survey

In decentralized systems, the quest for heightened security and integrity within blockchain networks becomes an issue. This survey investigates anomaly detection techniques in blockchain ecosystems through the lens of unsupervised learning, delving into the intricacies and going through the complex tapestry of abnormal behaviors by examining avant-garde algorithms to discern deviations from normal patterns. By seamlessly blending technological acumen with a discerning gaze, this survey offers a perspective on the symbiotic relationship between unsupervised learning and anomaly detection by reviewing this problem with a categorization of algorithms that are applied to a variety of problems in this field. We propose that the use of unsupervised algorithms in blockchain anomaly detection should be viewed not only as an implementation procedure but also as an integration procedure, where the merits of these algorithms can effectively be combined in ways determined by the problem at hand. In that sense, the main contribution of this paper is a thorough study of the interplay between various unsupervised learning algorithms and how this can be used in facing malicious activities and behaviors within public and private blockchain networks. The result is the definition of three categories, the characteristics of which are recognized in terms of the way the respective integration takes place. When implementing unsupervised learning, the structure of the data plays a pivotal role. Therefore, this paper also provides an in-depth presentation of the data structures commonly used in unsupervised learning-based blockchain anomaly detection. The above analysis is encircled by a presentation of the typical anomalies that have occurred so far along with a description of the general machine learning frameworks developed to deal with them. Finally, the paper spotlights challenges and directions that can serve as a comprehensive compendium for future research efforts.

Identification of sleep phenotypes in COPD using machine learning-based cluster analysis

BackgroundDisturbed sleep in patients with COPD impact quality of life and predict adverse outcomes. Research questionTo identify distinct phenotypic clusters of patients with COPD using objective sleep parameters and evaluate the associations between clusters and all-cause mortality to inform risk stratification. Study design and methodsA longitudinal observational cohort study using nationwide Veterans Health Administration data of patients with COPD investigated for sleep disorders. Sleep parameters were extracted from polysomnography physician interpretation using a validated natural language processing algorithm. We performed cluster analysis using an unsupervised machine learning algorithm (K-means) and examined the association between clusters and mortality using Cox regression analysis, adjusted for potential confounders, and visualized with Kaplan-Meier estimates. ResultsAmong 9992 patients with COPD and a clinically indicated baseline polysomnogram, we identified five distinct clusters based on age, comorbidity burden and sleep parameters. Overall mortality increased from 9.4 % to 42 % and short-term mortality (<5.3 years) ranged from 3.4 % to 24.3 % in Cluster 1 to 5. In Cluster 1 younger age, in 5 high comorbidity burden and in the other three clusters, total sleep time and sleep efficiency had significant associations with mortality. InterpretationWe identified five distinct clinical clusters and highlighted the significant association between total sleep time and sleep efficiency on mortality. The identified clusters highlight the importance of objective sleep parameters in determining mortality risk and phenotypic characterization in this population.

Prediction of Engineering Characteristics of Rock Masses Using Actual TBM Performance Data with Supervised and Unsupervised Learning Algorithms (a Case Study in Strong to Very Strong Igneous and Pyroclastic Rocks)

Prediction of Engineering Characteristics of Rock Masses Using Actual TBM Performance Data with Supervised and Unsupervised Learning Algorithms (a Case Study in Strong to Very Strong Igneous and Pyroclastic Rocks)

A low-latency graph computer to identify metastable particles at the Large Hadron Collider for real-time analysis of potential dark matter signatures

Image recognition is a pervasive task in many information-processing environments. We present a solution to a difficult pattern recognition problem that lies at the heart of experimental particle physics. Future experiments with very high-intensity beams will produce a spray of thousands of particles in each beam-target or beam-beam collision. Recognizing the trajectories of these particles as they traverse layers of electronic sensors is a massive image recognition task that has never been accomplished in real time. We present a real-time processing solution that is implemented in a commercial field-programmable gate array using high-level synthesis. It is an unsupervised learning algorithm that uses techniques of graph computing. A prime application is the low-latency analysis of dark-matter signatures involving metastable charged particles that manifest as disappearing tracks.

Detection of Botnet in IOT Using Machine Learning

The proliferation of Internet of Things (IoT) devices has introduced unprecedented connectivity and convenience but also heightened the vulnerability to botnet attacks. There are an increasing number of Internet of Things (IoT) devices connected to the network these days, and due to the advancement in technology, the security threads and cyberattacks, such as botnets, are emerging and evolving rapidly with high-risk attacks. These attacks disrupt IoT transition by disrupting networks and services for IoT devices. Many recent studies have proposed ML and DL techniques for detecting and classifying botnet attacks in the IoT environment. This project presents a straightforward approach to detect botnet activity within IoT networks through the utilization of machine learning techniques. By analyzing network traffic patterns and employing unsupervised learning algorithms, we demonstrate an effective method to identify and mitigate botnet threats in IoT environments. By this project we intend to offer a valuable contribution in enhancing the security of IoT ecosystem. Key Word: Internet of Things(IoT),cybersecurity, botnet attacks, machine learning(ML),UNSW-NB15 dataset, exploratory data analysis, XgBoost

Identification of spatial clusters of potentially toxic elements in different soil types using unsupervised machine learning and compositional data analysis

Soil health is an important concept and the generation of soil chemical composition data, including potentially toxic elements (PTEs) is one of its conceptual components. This study aims to reveal the spatial distribution patterns of soil PTEs contents, identify their potential sources, and unveil their geochemical associations in Aragatsotn region (Armenia). For that purpose, the contents of Cr, V, Ti, As, Zn, Cu, Co, Fe, Mn, Ba, and Pb were determined using the X-ray fluorescence spectrometer (Innov X-5000). The results showed that mean contents of Cr and As exceeded their upper continental crust by 1.50 and 3.12 times and their maximum acceptable values by1.53, 1.50 times, respectively. The detailed analysis of the spatial distribution pattern of PTEs demonstrated the presence of sites where all these elements displayed comparatively higher values. The combined application of compositional data analysis and geospatial mapping allowed to reveal multivariate outliers which were located in structural-metallogenic zones with an active exploitation of ore deposits. The application of unsupervised machine learning algorithm unveiled three groups within the main dataset and the clr-biplot identified the source-specific elements. Particularly, Group I included Cu and displayed the highest mean value among the identified groups. The soil samples included in Group I were found in areas where Calcisols were developed and comparatively high Cu contents were attributed to agricultural activities (vineyards) and vehicle emissions. The Group II is represented by the geochemical association of Fe, Co, Cr, Mn, Zn, and As. The formation of this group is conditioned by volcanic rocks of the local geological origin. However, no spatial pattern was identified in Group II distribution aligned with the soil types. The Group III included Ti, V, Pb, and Ba. This group may have a mixed origin as it is spatially distributed in areas where regional highways pass through and where Group II elements also exhibit their higher values.

Hyperseed: Unsupervised Learning With Vector Symbolic Architectures.

Motivated by recent innovations in biologically inspired neuromorphic hardware, this article presents a novel unsupervised machine learning algorithm named Hyperseed that draws on the principles of vector symbolic architectures (VSAs) for fast learning of a topology preserving feature map of unlabeled data. It relies on two major operations of VSA, binding and bundling. The algorithmic part of Hyperseed is expressed within the Fourier holographic reduced representations (FHRR) model, which is specifically suited for implementation on spiking neuromorphic hardware. The two primary contributions of the Hyperseed algorithm are few-shot learning and a learning rule based on single vector operation. These properties are empirically evaluated on synthetic datasets and on illustrative benchmark use cases, IRIS classification, and a language identification task using the n -gram statistics. The results of these experiments confirm the capabilities of Hyperseed and its applications in neuromorphic hardware.

A Two-stage Method for Damage Detection in Z24 Bridge Based on K-nearest Neighbor and Artificial Neural Network

In this paper, we propose an effective approach for identifying damages in the Z24 bridge, a large-scale bridge in Switzerland. The dataset of the Z24 bridge is evaluated as a benchmark, reflecting the behavior of a real structure that has been used for numerous studies. However, most of the previous studies have only addressed the issues of updating the model or estimating the location and severity of damages. The core idea behind our proposed approach is to leverage the strengths of two effective Machine Learning (ML) algorithms: K-Nearest Neighbor (KNN) and Artificial Neural Network (ANN), to assess both the location and severity of damages in the Z24 bridge. First, we employ KNN, an unsupervised learning algorithm, for pinpointing the damage location. This strategy proves highly efficient, significantly reducing computation time by circumventing the need for a loss function during KNN training. By adopting this approach, KNN effectively mitigates the risk of encountering local minima in the ANN optimization process. Subsequently, we deploy ANN to determine the damage severity. When compared to previous studies on the Z24 bridge, our proposed method (KNN-ANN) exhibits promising results. Furthermore, our results illustrate that KNN-ANN consistently outperforms traditional ANN methodologies.

Training an Ising machine with equilibrium propagation

Ising machines, which are hardware implementations of the Ising model of coupled spins, have been influential in the development of unsupervised learning algorithms at the origins of Artificial Intelligence (AI). However, their application to AI has been limited due to the complexities in matching supervised training methods with Ising machine physics, even though these methods are essential for achieving high accuracy. In this study, we demonstrate an efficient approach to train Ising machines in a supervised way through the Equilibrium Propagation algorithm, achieving comparable results to software-based implementations. We employ the quantum annealing procedure of the D-Wave Ising machine to train a fully-connected neural network on the MNIST dataset. Furthermore, we demonstrate that the machine’s connectivity supports convolution operations, enabling the training of a compact convolutional network with minimal spins per neuron. Our findings establish Ising machines as a promising trainable hardware platform for AI, with the potential to enhance machine learning applications.

Utilizing data-driven algorithms for blind modal parameter identification of structures from output-only video measurements

In structural dynamics and vibration analysis, modal identification plays a pivotal role in understanding and characterizing the dynamic behavior of complex systems. Traditional modal analysis techniques heavily rely on accurate sensor placement and knowledge of the excitation forces, which may not always be feasible or practical in real-world scenarios. Recently, non-contact video-based modal analysis methods for structures with arbitrary complexity using computer vision techniques have gained much importance. But these methods need to know ahead of time where the structure's natural frequency ranges are, and they use steerable pyramids, which are complicated multi-scale image decomposition filters. To address these issues, a blind modal identification technique is suggested to extract the modal parameters (modal frequency and mode shapes) from the recorded structural vibration video signal. The proposed algorithm for unsupervised machine learning is the Non-Negative Matrix Factorization (NNMF) method, which is combined with the Generalized Complexity Pursuit (GCP) method for blind source separation. The NNMF algorithm is directly applied to the raw pixel-time series made from the video data to get the temporal components, which are then separated using GCP to find the individual modal frequencies and mode shapes. The above algorithm is first validated on a numerical model and then implemented on laboratory scale models (cantilever, single-degree, and multi-degree frame models). Finally, the proposed methodology is implemented on real-world recorded structural vibration videos to determine its noise sensitivity, such as the Tacoma Narrows bridge and the London Millennium footbridge. The modal parameters extracted are compared with those from the available literature for validation. The estimation errors obtained from all the validations are well below 1%, which makes the technique quite suitable and reliable for structural vibration monitoring by identifying and reproducing complex vibration modes.

Benchmarking Unsupervised Clustering Algorithms for Atomic Force Microscopy Data on Polyhydroxyalkanoate Films.

Surface of polyhydroxyalkanoate (PHA) films of varying monomer compositions are analyzed using atomic force microscopy (AFM) and unsupervised machine learning (ML) algorithms to investigate and classify films based on global attributes such as the scan size, film thickness, and monomer type. The experiment provides benchmarked results for 12 of the most widely used clustering algorithms via a hybrid investigation approach while highlighting the impact of using the Fourier transform (FT) on high-dimensional vectorized data for classification on various pools of data. Our findings indicate that the use of a one-dimensional (1D) FT of vectorized data produces the most accurate outcome. The experiment also provides insights into case-by-case investigations of algorithm performances and the impact of various data pools. Lastly, we show an early version of our tool aimed at investigating surfaces using ML approaches and discuss the results of our current experiment to configure future improvements.

Machine Learning to Enhance the Detection of Terrorist Financing and Suspicious Transactions in Migrant Remittances

Migrant remittances have become significant in poverty alleviation and microeconomic development in low-income countries. However, the ease of conducting global migrant remittance transfers has also introduced the risk of misuse by terrorist organizations to quickly move and conceal operational funds, facilitating terrorism financing. This study aims to develop an unsupervised machine learning algorithm capable of detecting suspicious financial transactions associated with terrorist financing in migrant remittances. To achieve this goal, a structural equation model (SEM) and an outlier detection algorithm were developed to analyze and identify suspicious transactions among the financial activities of migrants residing in Belgium. The results show that the SEM model classifies a significantly high number of transactions as suspicious, making it prone to detecting false positives. Finally, the study developed an ensemble outlier detection algorithm that comprises an isolation forest (IF) and a local outlier factor (LOF) to detect suspicious transactions in the same dataset. The model performed exceptionally well, being able to detect over 90% of suspicious transactions.

Transfer learning-based hybrid deep learning method for gas-bearing distribution prediction with insufficient training samples and uncertainty analysis

The availability of sample data, required for seismicity-based reservoir prediction using deep learning (DL), is often limited by well availability. To alleviate this problem, this study proposes a hybrid DL method combining unsupervised learning, transfer learning (TL), and intelligent optimization algorithms to predict gas-bearing distributions. First, three unsupervised learning methods were used to optimize multicomponent seismic attribute data during preprocessing and improve the sample data quality. Subsequently, a convolutional neural network (CNN) model was pre-trained using synthetic data, and the adaptive mutation particle swarm optimization (AMPSO) algorithm was used to optimize the hyperparameters of the CNN model during training to obtain the AMPSO-CNN model. Finally, the partial convolutional layer parameters of the pre-trained network were transferred, and the remaining parameters were adjusted using real data samples to obtain the TL-AMPSO-CNN model. The trained model was used for gas-bearing distribution prediction, and its stability was assessed using uncertainty analysis. The results confirmed that the hybrid DL model performed efficiently in gas-bearing prediction compared to other individual models. Thus, for datasets with insufficient training samples, the proposed scheme can be applied as a feasible alternative to predict gas-bearing distribution.

Class-driven nonnegative matrix factorization with manifold regularization for data clustering

Nonnegative matrix factorization (NMF) is an effective technique to extract the underlying low-dimensional structure of data by utilizing its parts-based representation, which has been widely used in feature extraction and machine learning. However, NMF is an unsupervised learning algorithm without utilizing the discriminative prior information. In this paper, we put forward a new class-driven NMF with manifold regularization (MCDNMF) algorithm, which incorporates both the local manifold regularization and the label information of data into the NMF model. Specifically, MCDNMF not only encodes the local geometrical structure of data space by using the manifold regularization, but also takes the available label information by introducing the class-driven constraint. This class-driven constraint forces the new representations of data points to be more similar within the same class while different between other classes. Therefore, the discriminative abilities of clustering are greatly boosted. Experimental results on several datasets validate the effectiveness of proposed MCDNMF in comparison with the other state-of-the-art methods.