Unsupervised Learning Strategy Research Articles

A defect classification algorithm for gas tungsten arc welding process based on unsupervised learning and few-shot learning strategy

Welding defect prediction is the foundation for ensuring welding quality in gas tungsten arc welding (GTAW). In the prediction process, method based on molten pool vision is the most effective. Since the classification of molten pool defects relies on a substantial volume of labeled data, it is challenging for the models to be applied industrially. This paper presents an algorithm, FS-Classifier, that can achieve high prediction accuracy based on a limited amount of labeled data. The FS-Classifier comprises two stages: Firstly, an unsupervised training approach named RaP is designed to pre-train the feature extractor using extensive unlabeled daily datasets. The RaP consists of a rotation angle prediction task and a position prediction task, which ensure that the network focuses on salient features and precise elements, respectively. Secondly, the support vectors constructed from limited labeled data are used for the feature classifier. The input data is classified to certain class by computing its distances to support vector. The model achieves an accuracy of 94.5 % on the private dataset and 92.8 % on the public dataset for the six classes of defects using 5 % of labeled data volume. In addition, comparative experiments show that our method only requires 5 % of labeled data to achieve accuracy comparable to traditional supervised learning methods. The proposed algorithm addresses the issue of relying on a substantial amount of labeled data in welding process defect classification.

The Use of eXplainable Artificial Intelligence and Machine Learning Operation Principles to Support the Continuous Development of Machine Learning-Based Solutions in Fault Detection and Identification

Machine learning (ML) revolutionized traditional machine fault detection and identification (FDI), as complex-structured models with well-designed unsupervised learning strategies can detect abnormal patterns from abundant data, which significantly reduces the total cost of ownership. However, their opaqueness raised human concern and intrigued the eXplainable artificial intelligence (XAI) concept. Furthermore, the development of ML-based FDI models can be improved fundamentally with machine learning operations (MLOps) guidelines, enhancing reproducibility and operational quality. This study proposes a framework for the continuous development of ML-based FDI solutions, which contains a general structure to simultaneously visualize and check the performance of the ML model while directing the resource-efficient development process. A use case is conducted on sensor data of a hydraulic system with a simple long short-term memory (LSTM) network. Proposed XAI principles and tools supported the model engineering and monitoring, while additional system optimization can be made regarding input data preparation, feature selection, and model usage. Suggested MLOps principles help developers create a minimum viable solution and involve it in a continuous improvement loop. The promising result motivates further adoption of XAI and MLOps while endorsing the generalization of modern ML-based FDI applications with the HITL concept.

Intelligent computing framework to analyze the transmission risk of COVID-19: Meyer wavelet artificial neural networks

The optimum control methods for the epidemiology of the COVID-19 model are acknowledged using a novel advanced intelligent computing infrastructure that joins artificial neural networks with unsupervised learning-based optimizers i.e., Genetic Algorithms (GA) and sequential quadratic programming (SQP). Unsupervised learning strategy is provided which depends on the wavelet basis's sequential deconstruction of stochastic data. The weights or selection values of neural networks are utilizing cumulative algorithms of Meyer wavelet artificial neural networks (MWANNs) optimized with global search Genetic Algorithms (GAs) and Sequential Quadratic Programming (SQP), referred to as MWANNs-GA-SQP and the design technique is utilized to determine the COVID-19 model for five different scenarios employing different step sizes and input intervals. The findings of this research article examined that in order to minimize the total disease transmission at the lowest cost and complexity, safety, focused medical care, and exterior sterilization methods applicability. The provided data is validated through various graphical simulations, which surely authenticate the effectiveness and robustness of the proposed solver. The suggested solver, MWANNs-GA-SQP, is tested in a variety of circumstances to examine that how reliable, safe, and tolerant. Using the proposed MWANNs hubristic intelligent approach, an objective optimization function is created in feed forward neural networking to minimize the mean square error. An investigation of the hybrid GA-SQP is used to confirm the accuracy and dependability of the MWANNs model results. Mean absolute graphs have been constructed to assess the integrity and efficiency of the proposed methodology. The accuracy and reliability of the suggested method are demonstrated by constantly achieving maximum variables of analytical assessment criteria computed for a large appropriate variety of distinct trials.

Artificial intelligence and machine learning in optics: tutorial

Across the spectrum of scientific inquiry and practical applications, the emergence of artificial intelligence (AI) and machine learning (ML) has comprehensively revolutionized problem-solving methodologies. This tutorial explores key aspects of AI/ML and their remarkable role in augmenting the capabilities of optics and photonics technologies. Beginning with fundamental definitions and paradigms, the tutorial progresses to classical machine learning algorithms, with examples employing support vector machines and random forests. Extensive discussion of deep learning encompasses the backpropagation algorithm and artificial neural networks, with examples demonstrating the applications of dense and convolutional neural networks. Data augmentation and transfer learning are examined next as effective strategies for handling scenarios with limited datasets. Finally, the necessity of alleviating the burden of data collection and labeling is discussed, motivating the investigation of unsupervised and semi-supervised learning strategies as well as the utilization of reinforcement learning. By providing a structured exploration of AI/ML techniques, this tutorial equips researchers with the essential tools to begin leveraging AI’s transformative potential within the expansive realm of optics and photonics.

Finding a closest saddle–node bifurcation in power systems: An approach by unsupervised deep learning

We propose a neural network using an unsupervised learning strategy for direct computation of closest saddle–node bifurcations, eliminating the need for labeled training data. Our method not only estimates the worst-case load increase scenarios but also significantly reduces the computational complexity traditionally associated with this task during inference time. Simulation results validate the effectiveness and real-time applicability of our approach, demonstrating its potential as a robust tool for modern power system analysis.

Gas well production optimization: Classifying liquid loading severity in shale gas wells using semi-supervised learning

The liquid loading of shale gas wells presents significant challenges, including a reduction in production capacity, an increase in production costs, and environmental degradation. It is of paramount importance to provide accurate and timely warnings of liquid loading in order to ensure the efficient production operations. Existing physical models often rely on assumptions regarding material properties and fluid behaviors. However, the establishment of precise physical models in complex real-world scenarios represents a significant challenge. Most existing data-driven methods rely on either supervised or unsupervised learning strategies. The former causes a considerable investment of time and human resources to generate labels, while the latter demands exceptionally high standards for data quality. These are challenging to achieve in actual production. Most studies on liquid loading adopt a dichotomous approach, whereby the presence or absence of liquid loading is determined. A paucity of studies has attempted to classify the severity of liquid loading. In this paper, a semi-supervised learning-based liquid loading severity classification model is proposed. The model integrates the benefits of supervised and unsupervised learning, enabling the utilization of the entire dataset with a minimal number of labels. The paper also presents a Dynamic Time Relationship Self-Attention (DTRSA) module, which enhances the model’s anti-interference ability against other anomalies and improves the accuracy of liquid loading severity classification. In order to assess the effectiveness of the proposed model, experiments are conducted on a dataset of 219 shale gas wells in the southwestern region of Sichuan Province, China, within an oil and gas field. The model exhibits an overall classification accuracy of 98.46% in detecting liquid loading, with an earlier warning time than existing physical models.

Temporally-preserving latent variable models: Offline and online training for reconstruction and interpretation of fault data for gearbox condition monitoring

Latent variable models can effectively determine the condition of essential rotating machinery without needing labelled data. These models analyse vibration data via an unsupervised learning strategy. Temporal preservation is necessary to obtain an informative latent manifold for the fault diagnosis task. In a temporal-preserving context, two approaches exist to develop a condition-monitoring methodology: offline and online. For latent variable models, the available training modes are no different. While many traditional methods use offline training, online training can dynamically adjust the latent manifold, possibly leading to better fault signature extraction from the vibration data. This study explores online training using temporal-preserving latent variable models. Within online training, there are two main methods: one focuses on reconstructing data and the other on interpreting the data components. Both are considered to evaluate how they diagnose faults over time. Using two experimental datasets, the study confirms that models from both training modes can detect changes in machinery health and identify faults even under varying conditions. Importantly, the complementarity of offline and online models is emphasised, reassuring their versatility in fault diagnostics. Understanding the implications of the training approach and the available model formulations is crucial for further research in latent variable model-based fault diagnostics. Conflict of Interest Statement The authors declare no conflicts of interest.

LELD: Learn enhancement by learning degradation

Enhancing low-light images improves both the visibility and quality of the images. Existing methods primarily focus on the enhancement process and heavily rely on the supervised learning strategy, where low/normal-light image pairs are used as the training dataset. In this paper, we propose a novel method called Learn Enhancement by Learning Degradation (LELD) to achieve efficient light adjustment and scene fidelity. We use a carefully designed degradation network (DNet) to guide the enhancement network (ENet). Specifically, the role of DNet is transforming normal-light images into low-light images. For better generalization ability, we employ an unsupervised learning strategy and a generative adversarial network framework. The training is totally dependent on unpaired datasets. Inspired by Retinex theory, we propose a fidelity loss to maintain color and detail during the degradation process. The ENet exhibits a straightforward architecture and achieves efficient enhancement. Experimental results demonstrate the advantages of our method over state-of-the-art methods in terms of visual quality and enhancement efficiency.

Ensemble Approach Using k-Partitioned Isolation Forests for the Detection of Stock Market Manipulation

Stock market manipulation, defined as any attempt to artificially influence stock prices, poses significant challenges by causing financial losses and eroding investor trust. The prevalent reliance on supervised learning models for detecting such manipulations, while showing promise, faces notable hurdles due to the dearth of labeled data and the inability to recognize novel manipulation tactics beyond those explicitly labeled. This study ventures into addressing these gaps by proposing a novel detection framework aimed at identifying suspicious hourly manipulation blocks through an unsupervised learning approach, thereby circumventing the limitations of data labeling and enhancing the adaptability to emerging manipulation strategies. Our methodology involves the innovative creation of features reflecting the behavior of stocks across various time windows followed by the segmentation of the dataset into k subsets. This setup facilitates the identification of potential manipulation instances via a voting ensemble composed of k isolation forest models, which have been chosen for their efficiency in pinpointing anomalies and their linear computational complexity—attributes that are critical for analyzing vast datasets. Evaluated against eight real stocks known to have undergone manipulation, our approach demonstrated a remarkable capability to identify up to 89% of manipulated blocks, thus significantly outperforming previous methods that do not utilize a voting ensemble. This finding not only surpasses the detection rates reported in prior studies but also underscores the enhanced robustness and adaptability of our unsupervised model in uncovering varied manipulation schemes. Through this research, we contribute to the field by offering a scalable and efficient unsupervised learning strategy for stock manipulation detection, thereby marking a substantial advancement over traditional supervised methods and paving the way for more resilient financial markets.

Contrastive deep convolutional transform k-means clustering

Deep clustering has gained the immense attention of researchers in recent years. Most of the deep clustering approaches are based on auto-encoders which consist of an encoder-decoder framework. In these approaches, the clustering module is embedded in the latent space of auto-encoders. The auto-encoder based deep clustering approaches require learning of encoder weights as well as decoder weights. Moreover, due to the unsupervised learning strategy, these approaches lack in learning the discriminative features that can help in generating better clusters. This work introduces a novel clustering approach based on Contrastive Deep Convolutional Transform Learning (DCTL) framework. The proposed approach mitigates the problem of lack of supervision in DCTL based K-means clustering approach by embedding the contrastive learning into it. To embed the contrastive learning, the positive pairs and negative pairs of data samples are generated by reconstructing the data samples from the DCTL learnt representation itself and thus eliminates the requirement of data augmentation for embedding contrastive learning. The experimental results on several benchmark facial images datasets demonstrate that the proposed framework gives better clustering performance as compared to the current state-of-the-art deep clustering approaches especially in data constrained scenarios.

GraphMoCo: A graph momentum contrast model for large-scale binary function representation learning

In the field of cybersecurity, the ability to compute similarity scores at the function level for binary code is of utmost importance. Considering that a single binary file may contain an extensive amount of functions, an effective learning framework must exhibit both high accuracy and efficiency when handling substantial volumes of data. Nonetheless, conventional methods encounter several limitations. Firstly, accurately annotating different pairs of functions with appropriate labels poses a significant challenge, thereby making it difficult to employ supervised learning methods without risk of overtraining. Secondly, while SOTA models often rely on pre-trained encoders or fine-grained graph comparison techniques, these approaches suffer from drawbacks related to time and memory consumption. Thirdly, the momentum update algorithm utilized in graph-based contrastive learning models can result in information leakage. Surprisingly, none of the existing articles address this issue. This research focuses on addressing the challenges associated with large-scale Binary Code Similarity Detection (BCSD). To overcome the aforementioned problems, we propose GraphMoCo: a graph momentum contrast model that leverages multimodal structural information for efficient binary function representation learning on a large scale. We adopt an unsupervised learning strategy. Our approach eliminates the need for manual labeling. By leveraging the intrinsic structural information at multiple levels of the binary code, our model could achieve higher accuracy with a simple CNN-based model. By introducing the preshuffle mechanism, the issue of information leakage in graph momentum update algorithm is mitigated. The evaluation results indicate that GraphMoCo exhibits superior performance compared to SOTA approaches in the function pair search task, showing an average improvement of 7% on AUC, and 10% on MRR and Recall@1. Furthermore, GraphMoCo achieves a MAP of 0.93 on the more challenging dataset 2, which comprises a larger function pool. In a real-world scenario, specifically in known vulnerability searching, GraphMoCo achieves a MRR that surpasses existing SOTA models by 5%.

HeGCL: Advance Self-Supervised Learning in Heterogeneous Graph-Level Representation.

Representation learning in heterogeneous graphs with massive unlabeled data has aroused great interest. The heterogeneity of graphs not only contains rich information, but also raises difficult barriers to designing unsupervised or self-supervised learning (SSL) strategies. Existing methods such as random walk-based approaches are mainly dependent on the proximity information of neighbors and lack the ability to integrate node features into a higher-level representation. Furthermore, previous self-supervised or unsupervised frameworks are usually designed for node-level tasks, which are commonly short of capturing global graph properties and may not perform well in graph-level tasks. Therefore, a label-free framework that can better capture the global properties of heterogeneous graphs is urgently required. In this article, we propose a self-supervised heterogeneous graph neural network (GNN) based on cross-view contrastive learning (HeGCL). The HeGCL presents two views for encoding heterogeneous graphs: the meta-path view and the outline view. Compared with the meta-path view that provides semantic information, the outline view encodes the complex edge relations and captures graph-level properties by using a nonlocal block. Thus, the HeGCL learns node embeddings through maximizing mutual information (MI) between global and semantic representations coming from the outline and meta-path view, respectively. Experiments on both node-level and graph-level tasks show the superiority of the proposed model over other methods, and further exploration studies also show that the introduction of nonlocal block brings a significant contribution to graph-level tasks.

Hybrid Unsupervised Learning Strategy for Monitoring Industrial Batch Processes

Industrial production processes, especially in the pharmaceutical industry, are complex systems that require continuous monitoring to ensure efficiency, product quality, and safety. This paper presents a hybrid unsupervised learning strategy (HULS) for monitoring complex industrial processes. Addressing the limitations of traditional Self-Organizing Maps (SOMs), especially in scenarios with unbalanced data sets and highly correlated process variables, HULS combines existing unsupervised learning techniques to address these challenges. To evaluate the performance of the HULS concept, comparative experiments are performed based on a laboratory batch process.

Combining Hough Transform and Fuzzy Unsupervised Learning Strategy in Automatic Segmentation of Large Bowel Obstruction Area from Erect Abdominal Radiographs

Combining Hough Transform and Fuzzy Unsupervised Learning Strategy in Automatic Segmentation of Large Bowel Obstruction Area from Erect Abdominal Radiographs

Statistical similarity matching and filtering for clinical image retrieval by machine learning approach

Clinical image retrieval plays a pivotal role in modern healthcare for diagnostics and research, but prior research has grappled with the challenge of achieving high accuracy due to limited filtering techniques. The proposed method includes statistical distance measurements for similarity comparison and a machine learning technique for image filtering. Throughout this framework, the search area for similarity matching is reduced by first filtering away irrelevant images using the probabilistic outcomes of the Support Vector Machine (SVM) classification as class predictions of search and database images. Resizing is done as part of the preprocessing. Then, using Principal Component Analysis (PCA), the preprocessed data’s textural features, visual characteristics, and low-level features are extracted. The study also suggested an adaptive similarity matching method centered on a linear integration of feature-level similarities on the individual-level level. The precision and ranking order details of the most appropriate images retrieved and predicted by SVMs are considered when calculating the feature weights. The system continually alters weights for every distinctive search to generate beneficial outcomes. The supervised and unsupervised learning strategies are studied to link low-level global image features in the generated PCA-based Eigen Space using their high-level semantic and visual classifications to reduce the semantic gap and enhance retrieval effectiveness. The ground-truth database used in experiments has 1594 unique medical images with 3 different databases. Our method significantly improves the precision and recall rates in image retrieval tasks by combining sophisticated feature extraction, data-driven algorithms, and deep learning models. Research obtained an impressive accuracy of 0.99, demonstrating the effectiveness of our approach. This novel methodology addresses the limitations of prior research and provides a robust and reliable solution for clinicians and researchers in the medical field seeking to access and analyze relevant clinical images.

LIDA‐YOLO: An unsupervised low‐illumination object detection based on domain adaptation

AbstractThe low‐light environment is integral to everyday activities but poses significant challenges in object detection. Due to the low brightness, noise, and insufficient illumination of the acquired image, the model's object detection performance is reduced. Opposing recent studies mainly developing using supervised learning models, this paper suggests LIDA‐YOLO, an approach for unsupervised adaptation of low‐illumination object detectors. The model improves the YOLOv3 by using normal illumination images as the source domain and low‐illumination images as the target domain and achieves object detection in low‐illumination images through an unsupervised learning strategy. Specifically, a multi‐scale local feature alignment and global feature alignment module are proposed to align the overall attributes of the image and feature biases such as background, scene, and target layout are thus reduced. The experimental results of LIDA‐YOLO on the ExDark dataset achieved the highest performance mAP score of 56.65% compared to several current state‐of‐the‐art unsupervised domain adaptation object detection methods. Compared to I3Net, the performance improvement is 4.04%, and compared to OSHOT, the performance improvement is 6.5%. LIDA‐YOLO achieves a performance improvement of 2.7% compared to the supervised baseline method YOLOv3. Overall, the suggested LIDA‐YOLO model requires fewer samples and presents a stronger generalization ability than previous works.

UDATNN: A modeling scheme integrating unsupervised domain adversarial learning and tri-training strategy for variety recognition of maize seeds with domain shift

Constructing an accurate classification model is the key to realizing maize seed varieties identification based on optical sensing techniques. However, the optical information (features) of seeds collected by optical sensing technology is easily affected by the planting environment (e.g., origin, year), which makes the training samples used to construct the model (source domain, SD) and the samples to be recognized (target domain, TD) subject to domain shift (DS), and ultimately undermines the recognition performance of the model. In this study, a modeling scheme, called unsupervised domain adversarial tri-training of neural networks (UDATNN), is proposed for maize variety recognition in the presence of DS. Firstly, an unsupervised domain adversarial learning approach is used to map the raw features of the SD and TD into a low-dimensional feature space in order to achieve feature alignment of the SD and TD, and to improve the feature discriminability of different varieties of seeds. Subsequently, the aligned low-dimensional features are used as inputs of classifiers (random forest) and part of the target domain samples are iteratively selected and given pseudo-labels according to the tri-training strategy. Finally, these samples assigned with pseudo-labels (called updating samples) together with the training samples in SD are used to re-train the classification model constructed based on the unsupervised domain adversarial strategy, for improving recognition accuracy of seed varieties with DS scenario. Hyperspectral images of a total of 4584 maize seeds, including 7 varieties (each variety produced from two or three years) were collected to verify the performance of the proposed scheme. The recognition accuracy of the target domain reaches 91.5%, 94.1%, and 92.6% under 3 different domain shifts, which improves the recognition performance nearly by 8%–22% compared with the no-transfer model and the traditional transfer model. The model performance was further discussed by calculating precision, recall, and F1 score to achieve satisfactory results. The robustness of the model was also verified by discussing the randomness of the update samples and the effect of the number of samples in the source domain on the model performance through 100 random experiments and multiple experimental comparisons. The proposed UDATNN scheme can be used as a new framework to address the nondestructive identification of seed varieties under domain shift conditions.

DEAR-GAN: Degradation-Aware Face Restoration With GAN Prior

With the development of generative adversarial networks (GANs), recent face restoration (FR) methods often utilize pre-trained GAN models ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i.e</i> ., StyleGAN2) as prior to generate rich details. However, these methods usually struggle to balance realness and fidelity when facing various degradation levels. In this paper, we propose a novel DEgradation-Aware Restoration network with GAN prior, dubbed DEAR-GAN, for FR tasks by explicitly learning the degradation representations (DR) to adapt to various degradation. Specifically, an unsupervised degradation representation learning (UDRL) strategy is first developed to extract DR of the input degraded images. Then, a degradation-aware feature interpolation (DAFI) module is proposed to dynamically fuse the two types of informative features ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i.e</i> ., features from degraded images and features from GAN prior network) with flexible adaption to various degradation based on DR. Extensive experiments show that our proposed DEAR-GAN outperforms the state-of-the-art methods for face restoration under multiple degradation and face super-resolution, and demonstrate the effectiveness of feature interpolation, which can be extended to face inpainting to achieve excellent results.

Industrial UAV-Based Unsupervised Domain Adaptive Crack Recognitions: From Database Towards Real-Site Infrastructural Inspections

The defect diagnosis of modern infrastructures is crucial to public safety. In this work, we propose an unsupervised domain adaptive crack recognition framework. To fulfill the unsupervised domain adaptation (UDA) task of cracks recognition in infrastructural inspections, we propose a robust unsupervised domain adaptive learning strategy termed <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Crack-DA</i> to increase the generalization capacity of the model in unseen test circumstances. More specifically, we first propose leveraging the self-supervised depth information to help the learning of semantics. And then using the edge information to suppress non-edge background objects and noises. We propose the data augmentation-based consistency to enhance the performance. More importantly, we proposed to use the disparity in depth to evaluate the domain gap in semantics and explicitly consider the domain gap in network optimization. A database composed of a large number of crack images with detailed pixel-level labels for network training is established. Extensive experiments on UAV-captured highway cracks and real-site UAV inspections of building cracks demonstrate the robustness and effectiveness of our proposed domain adaptive crack recognition approach.

GATE: A guided approach for time series ensemble forecasting

In this article, a new ensemble learning model called GATE is proposed to improve the accuracy and stability of time-series forecasting, which is a crucial aspect of modern engineering practices. Despite the promise of deep learning (DL) models in this area, their performance can be volatile due to the diversity of time series data. To address this, the GATE model combines the strengths of recurrent neural networks (RNN), long short-term memory network (LSTM), and convolution-LSTM (Conv-LSTM) structures and utilizes an unsupervised learning strategy to steer the ensemble output using a guided network. To prevent overfitting in DL models, GATE optimizes the sample loss function and the weight updating function for each individual model within the ensemble structure. A comprehensive evaluation of the proposed GATE method on four real-world datasets is presented. The experimental results unequivocally demonstrate that GATE surpasses state-of-the-art ensemble methods and individual models, exhibiting the best performance in terms of testing errors. Notably, GATE outperforms existing single models in addressing long-term prediction tasks. To validate the effectiveness of GATE, ablation studies is carried out, comparing different ensemble combinations involving two distinct models. Through systematic analysis, these studies provided valuable insights into the performance of various ensemble configurations, further confirming the effectiveness and superiority of the proposed GATE method.