Unsupervised Learning Research Articles

An unsupervised transfer learning gear fault diagnosis method based on parameter-optimized VMD and residual attention networks

An unsupervised transfer learning gear fault diagnosis method based on parameter-optimized VMD and residual attention networks

Machine Learning in Cybersecurity: A Comprehensive Analysis of Intrusion Detection Systems

Cyber threats are getting more complex and larger, and traditional security measures can't keep up with the sophisticated attacks. Machine Learning (ML) has emerged as a powerful tool in enhancing cybersecurity, particularly in the development of Intrusion Detection Systems (IDS). a comprehensive analysis of the role of machine learning in intrusion detection, focusing on its ability to detect and respond to both known and unknown threats in real-time. various machine learning techniques, including supervised, unsupervised, and deep learning models, and evaluates their effectiveness in identifying anomalies and preventing security breaches. Key performance metrics such as accuracy, false positive rates, and scalability are analyzed, with a focus on the advantages and limitations of each approach. Additionally, the paper explores the challenges associated with implementing ML-based IDS, including data quality, adversarial attacks, and computational requirements. Through case studies and real-world examples, this paper highlights how machine learning is revolutionizing intrusion detection and presents insights into future advancements that will further strengthen cybersecurity defenses.

Ejection of Real-Time Malicious Intrusion and Attacks in IoT Empowered Infrastructure

The project recognizes the pervasive threat of computer viruses, malware, and hostile attacks on computer networks, highlighting the critical role of intrusion detection as a proactive defense technology. The project introduces a novel approach based on deep learning to identify and mitigate cybersecurity vulnerabilities and breaches in IoT-driven cyber-physical systems, aiming for enhanced security measures. The project's objective is to elevate intrusion detection beyond the limitations of traditional systems by addressing issues like accuracy, detection effectiveness, and reducing false positives. This emphasizes the advancement and innovation in cybersecurity. To achieve the project's goals, the method employs a generative adversarial network, a cutting-edge deep learning technique. Additionally, it distinguishes itself by contrasting unsupervised and deep learning-based discriminative approaches, showcasing a comprehensive and effective approach to cybersecurity. In our project, we successfully implemented an ensemble method to boost predictive accuracy by integrating multiple individual models. Particularly noteworthy is the inclusion of a hybrid architecture, combining Convolutional Neural Networks (CNN) and Long Short-Term Memory (LSTM), denoted as CNN+LSTM. This hybrid model achieved an impressive accuracy of 99% when applied to the KDD-Cup dataset, underscoring the efficacy of our ensemble technique for intrusion detection in IoT- based cybersecurity infrastructures.

DGSIST: Clustering spatial transcriptome data based on deep graph structure Infomax

Although spatial transcriptomics data provide valuable insights into gene expression profiles and the spatial structure of tissues, most studies rely solely on gene expression information, underutilizing the spatial data. To fully leverage the potential of spatial transcriptomics and graph neural networks, the DGSI (Deep Graph Structure Infomax) model is proposed. This innovative graph data processing model uses graph convolutional neural networks and employs an unsupervised learning approach. It maximizes the mutual information between graph-level and node-level representations, emphasizing flexible sampling and aggregation of nodes and their neighbors. This effectively captures and incorporates local information from nodes into the overall graph structure. Additionally, this paper developed the DGSIST framework, an unsupervised cell clustering method that integrates the DGSI model, SVD dimensionality reduction algorithm, and k-means++ clustering algorithm. This aims to identify cell types accurately. DGSIST fully uses spatial transcriptomics data and outperforms existing methods in accuracy. Demonstrations of DGSIST’s capability across various tissue types and technological platforms have shown its effectiveness in accurately identifying spatial domains in multiple tissue sections. Compared to other spatial clustering methods, DGSIST excels in cell clustering and effectively eliminates batch effects without needing batch correction. DGSIST excels in spatial clustering analysis, spatial variation identification, and differential gene expression detection and directly applies to graph analysis tasks, such as node classification, link prediction, or graph clustering. Anticipation lies in the contribution of the DGSIST framework to a deeper understanding of the spatial organizational structures of diseases such as cancer.

Unsupervised learning of spatially-resolved ARPES spectra for epitaxially grown graphene via non-negative matrix factorization

This study proposed an unsupervised machine-learning approach for analyzing spatially-resolved ARPES. A combination of non-negative matrix factorization (NMF) and k-means clustering was applied to spatially-resolved ARPES spectra of the graphene epitaxially grown on a SiC substrate. The Dirac cones of graphene were decomposed and reproduced fairly well using NMF. The base and activation matrices obtained from the NMF results reflected the detailed spectral features derived from the number of graphene layers and growth directions. The spatial distribution of graphene thickness on the substrate was clearly visualized by the clustering using the activation matrices acquired via NMF. Integration with k-means clustering enables clear visualization of spatial variations. Our method efficiently handles large datasets, extracting spectral features without manual inspection. It offers broad applicability beyond graphene studies to analyze ARPES spectra in various materials.

Noise attenuation in distributed acoustic sensing data using a guided unsupervised deep learning network

Distributed acoustic sensing (DAS) is a promising technology introducing a new paradigm in the acquisition of high-resolution seismic data. However, DAS data often show weak signals compared with the background noise, especially in challenging installation environments. In this study, we develop a new approach to denoise DAS data that leverages an unsupervised deep learning (DL) model, eliminating the need for labeled training data. The input DAS data undergo band-pass filtering to eliminate high-frequency content. Subsequently, a continuous wavelet transform (CWT) is performed, and the finest scale is used to guide the DL model in reconstructing the DAS signal. First, we extract 2D patches from the band-pass filtered data and the CWT scale of the data. Then, these patches are converted using an unrolling mechanism into 1D vectors to form the input of the DL model. A self-attention layer is included in each layer to extract the spatial relation between the band-pass filtered data and the CWT scale. Through an iterative process, the DL model tunes its parameters to suppress DAS noise, with the band-pass filtered data serving as the target for the network. The denoising performance of our framework is validated using field examples from the San Andreas Fault Observatory at Depth and Frontier Observatory for Research in Geothermal Energy data sets, where the data are recorded by a fiber-optic cable. Comparative analyses against three benchmark methods reveal the robust denoising performance of our framework.

A method for extracting Martian terrain features based on unsupervised contrastive learning

Intelligent recognition of Martian surface terrain is of great significance to autonomous exploration of Mars rovers. Feature extraction methods for Martian terrain images are currently mainly divided into two categories: traditional shallow visual feature extraction and deep feature extraction based on supervised learning. Loss of image information and the need for a large amount of labeled data are key issues to be addressed. A method for Martian terrain feature recognition based on unsupervised contrastive learning is proposed. By establishing an image dictionary dataset, two sets of neural networks, "query" and "encoding", are used to compare a single image with other images in the "dictionary" dataset, and the network is trained using the similarity functional as the loss function, thereby realizing feature recognition of Martian terrain images. The proposed method also has the ability to recognize new types of terrain images outside the training dataset, and has outstanding superiority in subsequent recognition and classification. Computer simulation verification shows that the recognition accuracy of the proposed method is and the 85.4%recognition accuracy of new types of terrain images is 84.5%.

Quarta: quantum supervised and unsupervised learning for binary classification in domain-incremental learning

Quantum machine learning recently gained prominence due to the promise of quantum computers in solving machine learning problems that are intractable on a classical computer. Nevertheless, several studies on problems which remain challenging for classical computing algorithms are emerging. One of these is classifying continuously incoming data instances in incremental fashion, which is studied in this paper through a hybrid computational solution that combines classical and quantum techniques. Hybrid approaches represents one of the current ways for the use of quantum computation in practical applications. In this paper, we show how typical issues of domain-incremental learning can be equally addressed with the properties of quantum mechanics, until to offer often better results. We propose the framework QUARTA to combine algorithms of quantum supervised learning, that is, variational quantum circuits, and techniques used in quantum unsupervised learning, that is, distance estimation. We aim at keeping the classification capabilities, which have learned on previously processed data instances, preserved as much as possible, and then acquiring new knowledge on new data instances. Experiments are performed on real-world datasets with quantum simulators.

Machine learning applications of network security enhancement: review

Machine learning (ML) is being used to improve intrusion detection mechanisms and identification in cyber security. Network data volume scaling (with the help of Machine learning) — Automated analysis and pattern recognition for large amounts of network-data, thereby detection of anomalies / potentially malicious activities that escape current rule-based techniques. By training ML models on historical data, these models can learn benign network behavior as well the anomalies in them that may result from malicious activities. The purpose of this essay/report is to begin taking a look under the hood at how ML can be used for security threat detection and analysis in-networking on an ongoing basis. This paper covers the automation of ML algorithms to enhance network security. Given the current state of electronic threats and their evolution, traditional security methods are typically insufficient. ML can analyze large volumes of data, learn patterns from it and this makes it suitable to complement network defense mechanisms. It then discusses different ML applications in network cybersecurity: intrusion detection, anomaly detection, spam and malware analysis; which the paper characterizes. It analyzes the potential, benefits and constrains of major ML methods in network security like supervised learning; unsupervised learning and reinforcement-learning. Finally, this paper represents recent progress in the use and impact of ML techniques along with case studies.The paper discusses the existing difficulties in the field such as the necessity for datasets and the vulnerability of machine learning models to adversarial attacks.T he paper also highlights avenues for exploration by focusing on developing scalable security solutions based on machine learning that are resilient and flexible.The goal of this examination is to offer both researchers and industry professionals valuable perspectives into the opportunities and obstacles linked to utilizing machine learning, in the domain of network security. ML methods have potential to improve network security by addressing the challenges posed by the increasing cyber threats that traditional security measures struggle to combat effectively over time. One key strength of ML lies in its capacity to analyze datasets and identify intricate patterns efficiently. The research paper delves into applications of ML in enhancing network security. The list covers security tools like Intrusion Detection Systems (IDS) examining malware and phishing attempts as well as anomalies in network activity and user behavior analysis (UEBA). The study explores both supervised and unsupervised learning methods. How they are used for quick threat detection and response in real time scenarios.You will find case studies and recent developments that showcase the implementation and effectiveness of these strategies.In addition the article delves into the obstacles linked to using machine learning techniques in network security including the necessity, for datasets, The paper's goal is to give an enlightening summary to scholars and practitioners about how machine learning can be applied to network security in order to provide solutions that are robust, adaptive, and scalable. To this end, it touches on several relevant aspects. One is the threat posed by adversarial attacks on the sorts of models that are likely to be used in this context. Another is the imperative, deriving from both adversarial threat and model drift, that models needed in this context be available in a form usable for continuous update. Keywords: Network Security, machine learning (ML), Intrusion Detection Systems (IDS), entity behavior analytics (UEBA).

Quantum anomaly detection in the latent space of proton collision events at the LHC

The ongoing quest to discover new phenomena at the LHC necessitates the continuous development of algorithms and technologies. Established approaches like machine learning, along with emerging technologies such as quantum computing show promise in the enhancement of experimental capabilities. In this work, we propose a strategy for anomaly detection tasks at the LHC based on unsupervised quantum machine learning, and demonstrate its effectiveness in identifying new phenomena. The designed quantum models-an unsupervised kernel machine and two clustering algorithms-are trained to detect new-physics events using a latent representation of LHC data, generated by an autoencoder designed to accommodate current quantum hardware limitations on problem size. For kernel-based anomaly detection, we implement an instance of the model on a quantum computer, and we identify a regime where it significantly outperforms its classical counterparts. We show that the observed performance enhancement is related to the quantum resources utilised by the model.

Identifying the Vertical Stratification of Sediment Samples by Visible and Near-Infrared Spectroscopy

Vertical stratification in marine sediment profiles indicates physical and chemical sedimentary processes and, thus, is the first step in sedimentary research and in studying their relationship with global climate change. Traditional technologies for studying vertical stratification have low efficiency; thus, new technologies are highly needed. Recently, visible and near-infrared spectroscopy (VNIR) has been explored to rapidly determine sediment parameters, such as clay content, particle size, total carbon (TC), total nitrogen (TN), and so on. Here, we explored vertical stratification in a sediment column in the South China Sea using VNIR. The sediment column was 160 cm and divided into 160 samples by 1 cm intervals. All samples were classified into three layers by depth, that is, 0–50 cm (the upper layer), 50–100 cm (the middle layer), and 100–160 cm (the bottom layer). Concentrations of TC and TN in each sample were measured by Elementa Vario EL III. Visible and near-infrared reflectance spectra of each sample were collected by Agilent Cary 5000. A global model and several classification models for vertical stratification in sediments were established by a Support Vector Machine (SVM) after the characteristic spectra were identified using Competitive Adaptive Reweighted Sampling. In the classification models, K-means clustering and Density Peak Clustering (DPC) were employed as the unsupervised clustering algorithms. The results showed that the stratification was successful by VNIR, especially when using the combination of unsupervised clustering and machine learning algorithms. The correct classification rate (CCR) was much higher in the classification models than in the global model. And the classification models had a higher CCR using K-means combined with SVM (94.8%) and using DPC combined with SVM (96.0%). The higher CCR might be derived from the chemical classification. Indeed, similar results were also found in the chemical stratification. This study provided a theoretical basis for the rapid and synchronous measurement of chemical and physical parameters in sediment profiles by VNIR.

Blood transcriptome profiling reveals distinct gene networks induced by mRNA vaccination against COVID-19.

Messenger RNA (mRNA) vaccines represent a new class of vaccines that has been shown to be highly effective during the COVID-19 pandemic and that holds great potential for other preventative and therapeutic applications. While it is known that the transcriptional activity of various genes is altered following mRNA vaccination, identifying and studying gene networks could reveal important scientific insights that might inform future vaccine designs. In this study, we conducted an in-depth weighted gene correlation network analysis of the blood transcriptome before and 24h after the second and third vaccination with licensed mRNA vaccines against COVID-19 in humans, following a prime vaccination with either mRNA or ChAdOx1 vaccines. Utilizing this unsupervised gene network analysis approach, we identified distinct modular networks of co-varying genes characterized by either an expressional up- or downregulation in response to vaccination. Downregulated networks were associated with cell metabolic processes and regulation of transcription factors, while upregulated networks were associated with myeloid differentiation, antigen presentation, and antiviral, interferon-driven pathways. Within this interferon-associated network, we identified highly connected hub genes such as STAT2 and RIGI and associated upstream transcription factors, potentially playing important regulatory roles in the vaccine-induced immune response. The expression profile of this network significantly correlated with S1-specific IgG levels at the follow-up visit in vaccinated individuals. Those findings could be corroborated in a second, independent cohort of mRNA vaccine recipients. Collectively, results from this modular gene network analysis enhance the understanding of mRNA vaccines from a systems immunology perspective. Influencing specific gene networks could lead to optimized vaccines that elicit augmented vaccine responses.

Scalable Clustering: Large Scale Unsupervised Learning of Gaussian Mixture Models with Outliers

Clustering is a widely used technique with a long and rich history in a variety of areas. However, most existing algorithms do not scale well to large datasets, or are missing theoretical guarantees of convergence. This paper introduces a provably robust clustering algorithm based on loss minimization that performs well on Gaussian mixture models with outliers. It provides theoretical guarantees that the algorithm obtains high accuracy with high probability under certain assumptions. Moreover, it can also be used as an initialization strategy for k-means clustering. Experiments on real-world large-scale datasets demonstrate the effectiveness of the algorithm when clustering a large number of clusters, and a k-means algorithm initialized by the algorithm outperforms many of the classic clustering methods in both speed and accuracy, while scaling well to large datasets such as ImageNet.

Reinforcement Learning-Based Optimal Hull Form Design with Variations in Fore and Aft Parts

Abstract With recent advancements in artificial intelligence technology, various studies are being conducted in the shipbuilding industry. Traditionally, hull form variation methods have relied on the intuition and expertise of designers, leading to inconsistent results and unintended changes in the ship's main dimensions depending on the designer's competence. Moreover, the iterative process of design variation and analysis to derive the optimal hull form is both costly and time-consuming. To address these issues, this study proposes an optimal hull design technique utilizing reinforcement learning, a type of unsupervised learning in machine learning. Reinforcement learning allows the model to learn from past policies by recording and accumulating the rewards associated with various actions taken by an agent in a specific environment. In this study, after calculating the main parameters of the ship, the agent defines a state representing hull information and performs local transformations of the bow and stern. The reward of reinforcement learning is defined as the change in total resistance due to the hull deformation, constrained by limiting the tolerance of the ship's prismatic coefficient (CP) and longitudinal center of buoyancy (LCB). In this study, the problem is solved by comparing the proximal policy optimization (PPO) algorithm and the deep deterministic policy gradient (DDPG) algorithm to find the best deep reinforcement learning (DRL) model for the hull optimization problem. The results were compared with the genetic algorithm and speed-constrained multi-objective particle swarm optimization (SMPSO), and the optimal hull resistance values were less different, but the time of the reinforcement learning model was five times shorter.

An alloy-agnostic machine learning framework for process mapping in laser powder bed fusion

PurposeThis study aims to introduce an image-based method to determine the processing window for a given alloy system using laser powder bed fusion equipment based on achieving the desired melting mode across multiple materials for powder-free specimens. The method uses a convolutional neural network trained to classify different track morphologies across different alloy systems to select appropriate printing settings. This method is intended for the development of new alloy systems, where the powder feedstock may be unavailable, or prohibitively expensive to manufacture.Design/methodology/approachA convolutional neural network is designed from scratch to identify the 4 key melting modes that are observed in laser powder bed fusion additive manufacturing across different alloy systems. To increase the prediction accuracy and generalisation accuracy across different materials, the network is trained using a novel hybrid data set that combines fully unsupervised learning with semi-supervised learning.FindingsThis study demonstrates that our convolutional network with a novel hybrid training approach can be generalised across different materials, and k-fold validation shows that the model retains good accuracy with changing training conditions. The model can predict the processing maps for the different alloys with an accuracy of up to 96% in some cases. It is also shown that powder-free single-track experiments are a useful indicator for predicting the final print quality of a component.Originality/valueThe “invariant information clustering” (IIC) approach is applied to process optimisation for additive manufacturing, and a novel hybrid data set construction approach that accounts for uncertainty in the ground truth data, enables the trained convolutional model to perform across a range of different materials and most importantly, generalise to materials outside of the training data set. Compared to the traditional cross-sectioning approach, this method considers the whole length of the single track when determining the melting mode.

Development of E-Tourism to Achieve Excellence and Sustainable Development in Tourism: Ha’il Region Case Study

E-tourism is one of the most important levers for social and economic development. The World Tourism Organization (UNWTO) has declared 2017 the year of sustainable tourism for development. When relying on sophisticated communication technologies. Certainly, these innovations in the field of this analysis aim to determine the main factors of e-tourism achieving excellence in tourism in the Ha’il region. Tourism has disrupted this sector with new and different dimensions of tourism development. This has led various regions and countries to accelerate the search for the tools necessary to achieve excellence and sustainable development in tourism. This research aims to study the development factors of e-tourism to achieve excellence and sustainable development in the tourism of the Ha’il region. Therefore, the gap in this research is represented by the lack of a comprehensive analysis of the interaction between e-tourism development factors and factors that promote tourism excellence in the Ha’il region to achieve sustainable development. For this reason, we applied two analyses: in the first one, we used a statistical analysis based on the survey sent to 500 employees from tourism companies. This analysis aimed to determine the main factors that influence e-tourism to achieve excellence in tourism in the Ha’il region. The results of the statistical analysis gave us the following elements: There are only four main factors that allow authorities to achieve excellence in tourism in the Ha’il region, namely Supporting Technology (ST), Competition Pressure (CP), Human Resource Skills (HRS), and Socio-Cultural (SC) factors. Then, we proceed with a second analysis to determine the main factors of excellence in tourism that drive the achievement of sustainable development in tourism in the Ha’il region. For this reason, we applied an artificial intelligence method based on the unsupervised machine learning technique. The results obtained from the machine learning technique show that five factors determine the excellence in tourism and reveal the sustainable development of Ha’il’s tourism. These factors are ST, CP, Financial Performance (FP), HRS, and Government Commitment (GC). Consequently, we consider that all the factors found in both analyses are important to realise excellence in tourism and sustainable development in tourism. However, to achieve sustainable development in tourism in Ha’il, Saudi authorities must develop the three common factors found in the two analyses, namely ST, CP, and HRS. Therefore, this study is expected to help introduce ways of excellence in e-tourism by marketing it using artificial intelligence, enabling the development of the tourism sector in an enormous area. It will also contribute significantly to the formulation of plans, policies, and strategies that will largely ensure the development of Ha’il’s tourism sector.

Unsupervised deep denoising for four-dimensional scanning transmission electron microscopy

By simultaneously achieving high spatial and angular sampling resolution, four dimensional scanning transmission electron microscopy (4D STEM) is enabling analysis techniques that provide great insight into the atomic structure of materials. Applying these techniques to scientifically and technologically significant beam-sensitive materials remains challenging because the low doses needed to minimise beam damage lead to noisy data. We demonstrate an unsupervised deep learning model that leverages the continuity and coupling between the probe position and the electron scattering distribution to denoise 4D STEM data. By restricting the network complexity it can learn the geometric flow present but not the noise. Through experimental and simulated case studies, we demonstrate that denoising as a preprocessing step enables 4D STEM analysis techniques to succeed at lower doses, broadening the range of materials that can be studied using these powerful structure characterization techniques.

Longitudinal Data Clustering Methods: A Systematic ReviewLongitudinal Data Clustering Methods: A Systematic Review

Introduction: In the last few decades, in many research fields, different methods were introduced to discover groups with the same trends in longitudinal data. The clustering process is an unsupervised learning method, which classifies longitudinal data based on different criteria by performing algorithms. The current study was performed with the aim of reviewing various methods of longitudinal data clustering, including two general categories of non-parametric methods and model-based methods. Methods: In this research, to obtain related scientific articles, PubMed, Science Direct Scopus, ISI, and Google Scholar were searched between 2000 and 2021. Results: According to our systematic review, the non-parametric k-means Clustering Method utilizing Euclidean distance emerges as a leading approach for clustering longitudinal data. Conclusion: This research, with an overview of the studies done in the field of clustering, can help researchers as a toolbox to choose various methods of longitudinal data clustering in idea generation and choosing the appropriate method in the classification and analysis of longitudinal data.

Enhancing Spectrometer Performance with Unsupervised Machine Learning.

Solid-state NMR spectroscopy (SSNMR) is a powerful technique to probe structural and dynamic properties of biomolecules at an atomic level. Modern SSNMR methods employ multidimensional pulse sequences requiring data collection over a period of days to weeks. Variations in signal intensity or frequency due to environmental fluctuation introduce artifacts into the spectra. Therefore, it is critical to actively monitor instrumentation subject to fluctuations. Here, we demonstrate a method rooted in the unsupervised machine learning algorithm principal component analysis (PCA) to evaluate the impact of environmental parameters that affect sensitivity, resolution and peak positions (chemical shifts) in multidimensional SSNMR protein spectra. PCA loading spectra illustrate the unique features associated with each drifting parameter, while the PCA scores quantify the magnitude of parameter drift. This is demonstrated both for double (HC) and triple resonance (HCN) experiments. Furthermore, we apply this methodology to identify magnetic field B0 drift, and leverage PCA to "denoise" multidimensional SSNMR spectra of the membrane protein, EmrE, using several spectra collected over several days. Finally, we utilize PCA to identify changes in B1 (CP and decoupling) and B0 fields in a manner that we envision could be automated in the future. Overall, these approaches enable improved objectivity in monitoring NMR spectrometers, and are also applicable to other forms of spectroscopy.

A concise review towards a novel target specific multi-source unsupervised transfer learning technique for GDP estimation using CO2 emission data

Though economic growths of most of the nations have seen exponential rise due to industrialization, it has also caused proportional increase in their carbon emissions. This paper exploits this proportionate relationship of carbon emission with GDP to predict the per-capita GDP of those nations whose GDP values are missing in the world bank database. The reason behind the same was, those countries were either war-torn or politically isolated/unstable. To achieve the objective of predicting the missing GDP values of those countries from their carbon emissions, this paper exploits the non-linear relationship among the carbon emissions from solid fuels, liquid fuels, and gaseous fuels. It is so because even the differential utilization of these fuels impact economy differently. Use of traditional solid fuel for cooking points toward energy poverty, and access to clean cooking gas indicates higher living standard. However, the available data from the war-torn or isolated countries are very little, and hence insufficient for building a robust predictive machine learning model. So, this paper employs multi-source unsupervised transfer learning to precisely estimate the missing per-capita GDP of those nations. It suitably enlarges the training domains for the prediction models to be more robust. We empirically evaluate the proposed methodology for different regression techniques to estimate the missing GDP values of eleven different nations belonging to diverse strata of economies viz. developed economies, developing, and/or least developing economies. Proposed methodology profoundly improves the prediction preciseness of these regression techniques in estimating the missing per-capita GDP of the considered nations.