Supervised Learning Approach Research Articles

Bridge progressive damage detection using unsupervised learning and self-attention mechanism

This paper presents a novel unsupervised method for detecting progressive damage in bridge structures. Traditional supervised learning approaches require a large number of damaged samples to train a high-performing detection model, which poses challenges when applied to normal in-service bridges. Since in-service bridges usually lack data from damage scenarios and existing methods fail to achieve real-time detection, unsupervised learning has gained popularity due to its ability to work without damaged samples and offer better real-time performance. In light of this, this paper proposes a novel unsupervised damage detection method that utilizes an unsupervised attention-convolutional auto-encoder to reconstruct real-time vibration signals from bridge. Two damage indicators are employed to evaluate the effectiveness of the reconstruction process, aiming to capture potential changes in the progressive damage of bridges. The results demonstrate that: the attention-convolutional auto-encoder achieves excellent reconstruction performance for vibration signals from intact structures, which reconstruction error converging to 0 and structural similarity approaching 1. However, the reconstruction performance is less satisfactory for vibration signals in damage scenarios, deteriorating further as the damage level increases. By analyzing the variation patterns of vibration signals in attention-convolutional auto-encoders, it was confirmed that the reliability of the proposed evaluation effects by the two different types of damage indicators. Finally, the trend of progressive damage was extracted through these indicators, showcasing the effectiveness of the method in detecting progressive damage. This approach facilitates the identification of potential progressive changes in the normal operation process of bridges, thus enhancing real-time damage detection capabilities and contributing to the knowledge of bridge monitoring.

Correction to “The Machines Take over: A Comparison of Various Supervised Learning Approaches for Automated Scoring of Divergent Thinking Tasks”

Correction to “The Machines Take over: A Comparison of Various Supervised Learning Approaches for Automated Scoring of Divergent Thinking Tasks”

Instance-level loss based multiple-instance learning framework for acoustic scene classification

An acoustic scene is inferred by detecting properties combining diverse sounds and acoustic environments. This study is intended to discover these properties effectively using multiple-instance learning (MIL). MIL, also known as a weakly supervised learning approach, is a strategy for extracting an instance vector from an audio chunk that composes an audio clip and utilizing these unlabeled instances to infer a scene corresponding to the input data. However, many studies pointed out an underestimation problem of MIL. In this study, we propose an enhanced MIL framework more suitable for ASC systems by defining instance-level labels and loss to extract and cluster instances effectively. Furthermore, we design a lightweight convolutional neural network named FUSE comprising frequency-, temporal-sided depthwise, and pointwise convolutional filters. Experimental results show that the confidence and proportion of positive instances significantly increase compared to vanilla MIL, overcoming the underestimation problem and improving the classification accuracy even higher than the supervised learning. The proposed system achieved a performance of 81.1%, 72.3%, and 58.3% on the TAU urban acoustic scenes 2019, 2020 mobile, and 2022 mobile datasets with 139 K parameters, respectively. In particular, it achieves the highest performance among the systems having under the 1 M parameters for the TAU urban acoustic scenes 2019 dataset.

Language inference-based learning for Low-Resource Chinese clinical named entity recognition using language model

Electronic health records (EHRs) have been widely used and are gradually replacing paper records. Therefore, extracting valuable information from EHRs has become the focus and hotspot of current research. Clinical named entity recognition (CNER) is an important task in information extraction. Most current research methods used standard supervised learning approaches to fine-tune pre-trained language models (PLMs), which require a large amount of annotated data for model training. However, in realistic medical scenarios, annotated data are scarce, especially in the healthcare field. The process of annotating data in real clinical settings is time-consuming and labour-intensive. In this paper, a language inference-based learning method (LANGIL) is proposed to study clinical NER tasks with limited annotated samples, i.e., in low-resource clinical scenarios. A method based on prompt learning is designed to reformulate the entity recognition task into a language inference-based task. Differing from the standard fine-tuning method, the approach introduced in this paper does not design the additional network layers that train from scratch. This alleviates the gap between pre-training tasks and downstream tasks, allowing the comprehension capabilities of PLMs to be leveraged under the condition of limited training samples. The experiments on four Chinese clinical named entity recognition datasets showed that LANGIL achieves significant improvements in F1-score compared to the former method.

Inferring Human Intent and Predicting Human Action in Human–Robot Collaboration

Researchers in human–robot collaboration have extensively studied methods for inferring human intentions and predicting their actions, as this is an important precursor for robots to provide useful assistance. We review contemporary methods for intention inference and human activity prediction. Our survey finds that intentions and goals are often inferred via Bayesian posterior estimation and Markov decision processes that model internal human states as unobserved variables or represent both agents in a shared probabilistic framework. An alternative approach is to use neural networks and other supervised learning approaches to directly map observable outcomes to intentions and to make predictions about future human activity based on past observations. That said, due to the complexity of human intentions, existing work usually reasons about limited domains, makes unrealistic simplifications about intentions, and is mostly constrained to short-term predictions. This state of the art provides opportunity for future research that could include more nuanced models of intents, reason over longer horizons, and account for the human tendency to adapt.

The Effects of Comparative Reviews on Product Sales

Consumers increasingly rely on online reviews to make their purchase decisions. Drawing from linguistics and sociology research, the authors posit that comparative reviews, which highlight the similarities and differences between a focal product and its alternatives, may influence consumers’ regulation systems and perceived credibility, thereby affecting product sales. The authors examined 61,480 reviews on e-commerce platforms to explore the effects of comparative reviews and their valence on product sales. By using a supervised learning approach, they identified positive and negative comparative reviews, as well as positive and negative regular reviews, and then applied a two-way fixed-effects model. The results show that comparative reviews positively impacted product sales. Specifically, positive comparative reviews had a greater effect than positive regular reviews, whereas negative comparative reviews had a lesser absolute effect than negative regular reviews on product sales. Moreover, positive comparative reviews exerted a greater absolute effect than negative ones. A follow-up controlled lab study further substantiated the authors’ results and insights. The findings offer new insights and practical guidance for marketers and practitioners in promoting more comparative review posts and optimizing online review presentations.

Weakly Supervised Learning Approach for Implicit Aspect Extraction

Aspect-based sentiment analysis (ABSA) is a process to extract an aspect of a product from a customer review and identify its polarity. Most previous studies of ABSA focused on explicit aspects, but implicit aspects have not yet been the subject of much attention. This paper proposes a novel weakly supervised method for implicit aspect extraction, which is a task to classify a sentence into a pre-defined implicit aspect category. A dataset labeled with implicit aspects is automatically constructed from unlabeled sentences as follows. First, explicit sentences are obtained by extracting explicit aspects from unlabeled sentences, while sentences that do not contain explicit aspects are preserved as candidates of implicit sentences. Second, clustering is performed to merge the explicit and implicit sentences that share the same aspect. Third, the aspect of the explicit sentence is assigned to the implicit sentences in the same cluster as the implicit aspect label. Then, the BERT model is fine-tuned for implicit aspect extraction using the constructed dataset. The results of the experiments show that our method achieves 82% and 84% accuracy for mobile phone and PC reviews, respectively, which are 20 and 21 percentage points higher than the baseline.

Silicon versus Superbug: Assessing Machine Learning's Role in the Fight against Antimicrobial Resistance.

In his 1945 Nobel Prize acceptance speech, Sir Alexander Fleming warned of antimicrobial resistance (AMR) if the necessary precautions were not taken diligently. As the growing threat of AMR continues to loom over humanity, we must look forward to alternative diagnostic tools and preventive measures to thwart looming economic collapse and untold mortality worldwide. The integration of machine learning (ML) methodologies within the framework of such tools/pipelines presents a promising avenue, offering unprecedented insights into the underlying mechanisms of resistance and enabling the development of more targeted and effective treatments. This paper explores the applications of ML in predicting and understanding AMR, highlighting its potential in revolutionizing healthcare practices. From the utilization of supervised-learning approaches to analyze genetic signatures of antibiotic resistance to the development of tools and databases, such as the Comprehensive Antibiotic Resistance Database (CARD), ML is actively shaping the future of AMR research. However, the successful implementation of ML in this domain is not without challenges. The dependence on high-quality data, the risk of overfitting, model selection, and potential bias in training data are issues that must be systematically addressed. Despite these challenges, the synergy between ML and biomedical research shows great promise in combating the growing menace of antibiotic resistance.

A comprehensive strategy for phase detection of high entropy alloys: Machine learning and deep learning approaches

High Entropy Alloys (HEAs) are a novel category of materials with unique physical and mechanical properties, which makes them a promising candidate for various engineering applications. However, the accurate identification of different phases present in HEAs, controlling its superb properties, is still a challenging task. To address this issue, this study proposes a comprehensive approach based on machine learning (ML) and deep learning (DL) to detect the phases present in HEAs. A deep neural network (DNN) was successfully developed that uses unique feature values of a number of HEAs to accurately predict different phases present in the alloy system. The DNN architecture consists of multiple layers that allow it to learn and extract the relevant features from the data, and then classify them into their respective phases. To train and validate the DNN model, a large dataset of HEAs (with known phases and their corresponding feature values) was collected from the literature. This dataset was then used to train the DNN model using a supervised learning approach. The DNN model was also validated using a separate set of HEAs with unknown phases to test its accuracy and reliability. Comparison with known results demonstrated that the DNN model can accurately predict the phases present in HEAs with high accuracy. After implementing all ML and DL algorithms, a voting ensemble (containing six best performing algorithms) was constructed. The final accuracy was obtained as high as 84% which is quite reasonable for classification problem. Further, it was also observed how significant each feature is in determining the phase(s) of HEAs. The proposed approach offers a promising method for phase detection in HEAs, which can lead to a deeper understanding of their properties and facilitate the design of new materials for various applications.

Optimized supervised learning approach to predict Parkinson’s disease with minimal attributes using PPMI Datasets

Optimized supervised learning approach to predict Parkinson’s disease with minimal attributes using PPMI Datasets

Comparing Supervised Learning and Rigorous Approach for Predicting Protein Stability upon Point Mutations in Difficult Targets.

Accurate prediction of protein stability upon a point mutation has important applications in drug discovery and personalized medicine. It remains a challenging issue in computational biology. Existing computational prediction methods, which range from mechanistic to supervised learning approaches, have experienced limited progress over the last few decades. This stagnation is largely due to their heavy reliance on both the quantity and quality of the training data. This is evident in recent state-of-the-art methods that continue to yield substantial errors on two challenging blind test sets: frataxin and p53, with average root-mean-square errors exceeding 3 and 1.5 kcal/mol, respectively, which is still above the theoretical 1 kcal/mol prediction barrier. Rigorous approaches, on the other hand, offer greater potential for accuracy without relying on training data but are computationally demanding and require both wild-type and mutant structure information. Although they showed high accuracy for conserving mutations, their performance is still limited for charge-changing mutation cases. This might be due to the lack of an available mutant structure, often represented by a simplified capped peptide. The recent advances in protein structure prediction methods now make it possible to obtain structures comparable to experimental ones, including complete mutant structure information. In this work, we compare the performance of supervised learning-based methods and rigorous approaches for predicting protein stability on point mutations in difficult targets: frataxin and p53. The rigorous alchemical method significantly surpasses state-of-the-art techniques in terms of both the root-mean-squared error and Pearson correlation coefficient in these two challenging blind test sets. Additionally, we propose an improved alchemical method that employs the pmx double-system/single-box approach to accurately predict the folding free energy change upon both conserving and charge-changing mutations. The enhanced protocol can accurately predict both types of mutations, thereby outperforming existing state-of-the-art methods in overall performance.

Self-supervised learning for clustering of wireless spectrum activity

In recent years, much work has been done on processing of wireless spectrum data involving machine learning techniques in domain-related problems for cognitive radio networks, such as anomaly detection, modulation classification, technology classification and device fingerprinting. Most of the solutions are based on labeled data, created in a controlled manner and processed with supervised learning approaches. However, spectrum data measured in real-world environment is highly nondeterministic, making its labeling a laborious and expensive process, requiring domain expertise, thus being one of the main drawbacks of using supervised learning approaches in this domain. In this paper, we investigate the utilization of self-supervised learning (SSL) for exploring spectrum activities in a real-world unlabeled data. In particular, we assess the performance of SSL models, based on the reference DeepCluster architecture. We carefully consider the current state-of-the-art feature extractors, taking into account the performance and complexity trade-offs. Our findings demonstrate that SSL models achieve superior performance regarding the feature quality and clustering performance compared to baseline feature learning approaches. With SSL models we achieve significant reduction of the feature vectors size by two orders of magnitude, while improving the performance by a factor ranging from 2 to 2.5 across the evaluation metrics, supported by visual assessment. Furthermore, we showcase how adapting the reference SSL architecture to domain-specific data is followed by a substantial reduction in model complexity up to one order of magnitude, without compromising, and in some cases, even improving the clustering performance.

Guided wave-based cross-scene interfacial debonding detection in reinforced concrete structures

The cross-scene problem within the field of guided wave-based interfacial debonding detection in steel-reinforced concrete structures is a significant challenge. This issue arises when the methods initially designed for laboratory-configured structures or numerical models are adapted to real-world scenarios. In response to this challenge, driven by the concept of domain adaptation, a novel method—the discriminative wavelet adversarial adaptation network (DWAAN)—is proposed to assess the health of analogous structures using diagnostic insights from a reference structure. The proposed method utilizes continuous wavelet transform to segregate signal components. A feature extraction network, designed with the convolutional neural network architecture, extracts domain confusion and damage-sensitive features through guided adversarial adaptive training in the wavelet domain. Both the adversarial and statistical alignment are employed during training for comprehensive optimization of data distribution discrepancies. The effectiveness of the proposed method is assessed in both the homogeneous and heterogeneous cross-scene interfacial debonding detection scenarios, revealing an average performance enhancement of 32.5% in distinct homogeneous applications and 26.6% in heterogeneous scenarios, when compared to the traditional supervised learning approach. In addition, the present method demonstrates robust convergence stability, evidenced by the minimized training fluctuations. The proposed method holds promising potential for practical applications in interfacial debonding detection in reinforced concrete structures while addressing the cross-scene challenge.

Automated classification of “cluttered” construction housekeeping images through supervised and self-supervised feature representation learning

Construction housekeeping is crucial for safety, but frequent manual inspections are difficult to maintain. A computer vision approach to automatically monitor housekeeping can overcome these issues. However, it requires labelling large number of “cluttered” construction housekeeping images that are difficult to label, even by experts. Thus, this paper presents an alternative approach that evaluates the use of self-supervised learning feature extraction to classify “cluttered” construction housekeeping images. The most suitable (84% accuracy) backbone architecture for supervised classification of housekeeping images was found to be Swin-transformer. In addition, the experiments show that self-supervised learning approach can perform better (1–4% improvement in prediction accuracy, precision, and recall) than the supervised learning approach in a non-transfer learning context and when the number of training images is reduced.

Self-supervised-RCNN for medical image segmentation with limited data annotation.

Many successful methods developed for medical image analysis based on machine learning use supervised learning approaches, which often require large datasets annotated by experts to achieve high accuracy. However, medical data annotation is time-consuming and expensive, especially for segmentation tasks. To overcome the problem of learning with limited labeled medical image data, an alternative deep learning training strategy based on self-supervised pretraining on unlabeled imaging data is proposed in this work. For the pretraining, different distortions are arbitrarily applied to random areas of unlabeled images. Next, a Mask-RCNN architecture is trained to localize the distortion location and recover the original image pixels. This pretrained model is assumed to gain knowledge of the relevant texture in the images from the self-supervised pretraining on unlabeled imaging data. This provides a good basis for fine-tuning the model to segment the structure of interest using a limited amount of labeled training data. The effectiveness of the proposed method in different pretraining and fine-tuning scenarios was evaluated based on the Osteoarthritis Initiative dataset with the aim of segmenting effusions in MRI datasets of the knee. Applying the proposed self-supervised pretraining method improved the Dice score by up to 18% compared to training the models using only the limited annotated data. The proposed self-supervised learning approach can be applied to many other medical image analysis tasks including anomaly detection, segmentation, and classification.

Application of one-dimensional convolutional neural network in guitar factory quality inspection

After the completion of guitar manufacturing, it is necessary to conduct quality testing and final adjustments to ensure that the guitar meets the factory standards. This stage often requires the involvement of professional musicians. In this project, a supervised learning approach is employed, where guitar vibration signals are inputted and correspond to the output sound quality. Multiple test subjects perform guitar strumming, and the guitars are adjusted to simulate various abnormal conditions for experimental comparison. By classifying the differences in energy and frequency, different guitar factory standards are identified, and the stability of guitar quality is explored. For data analysis, a one-dimensional convolutional neural network (1D CNN) is used as the base model, analyzing the Mel-frequency cepstral coefficients to identify elements of stable guitar quality and tonal characteristics. The research results show that the model can predict variations due to factors, such as environmental temperature, humidity, and worn-out strings, ensuring objective and consistent analysis of each guitar's factory quality. This innovative technology eliminates the need for traditional labor-intensive inspection procedures, enabling the traditional luthier industry to utilize automated methods for rapid factory testing.

Automated reliability calculation of failure rate, lifetime extrapolation and prediction for embedded Metal-Insulator-Metal capacitors using an optimized Time-Dependent-Dielectric-Breakdown model

This study presents an algorithm for automated reliability analysis of embedded Metal-Insulator-Metal (MIM) capacitors with high-k dielectrics. With the proposed algorithm, Time-Dependent-Dielectric-Breakdown (TDDB) data of embedded capacitors of different physical dimensions measured at various stress conditions (temperature and electric field) can be analyzed in a uniform way based on a supervised learning approach. Instead of analyzing each influence parameter separately, the data is combined using automated linear regression, based on decision tree learning and thus defining a multi-dimensional plane. With this approach a simultaneous consideration of all affecting parameters and their interaction on the dataset can be achieved, while removing statistical outliers and extrapolating the reliability behavior of any failure percentage. To support the most common conduction mechanism related to the embedded ΜΙΜ capacitors, Poole-Frenkel emission, a new TDDB model with changed coordinate system is proposed.

Improving Representation Learning for Histopathologic Images with Cluster Constraints.

Recent advances in whole-slide image (WSI) scanners and computational capabilities have significantly propelled the application of artificial intelligence in histopathology slide analysis. While these strides are promising, current supervised learning approaches for WSI analysis come with the challenge of exhaustively labeling high-resolution slides-a process that is both labor-intensive and timeconsuming. In contrast, self-supervised learning (SSL) pretraining strategies are emerging as a viable alternative, given that they don't rely on explicit data annotations. These SSL strategies are quickly bridging the performance disparity with their supervised counterparts. In this context, we introduce an SSL framework. This framework aims for transferable representation learning and semantically meaningful clustering by synergizing invariance loss and clustering loss in WSI analysis. Notably, our approach outperforms common SSL methods in downstream classification and clustering tasks, as evidenced by tests on the Camelyon16 and a pancreatic cancer dataset. The code and additional details are accessible at https://github.com/wwyi1828/CluSiam.

VAX

The use of audio and video modalities for Human Activity Recognition (HAR) is common, given the richness of the data and the availability of pre-trained ML models using a large corpus of labeled training data. However, audio and video sensors also lead to significant consumer privacy concerns. Researchers have thus explored alternate modalities that are less privacy-invasive such as mmWave doppler radars, IMUs, motion sensors. However, the key limitation of these approaches is that most of them do not readily generalize across environments and require significant in-situ training data. Recent work has proposed cross-modality transfer learning approaches to alleviate the lack of trained labeled data with some success. In this paper, we generalize this concept to create a novel system called VAX (Video/Audio to 'X'), where training labels acquired from existing Video/Audio ML models are used to train ML models for a wide range of 'X' privacy-sensitive sensors. Notably, in VAX, once the ML models for the privacy-sensitive sensors are trained, with little to no user involvement, the Audio/Video sensors can be removed altogether to protect the user's privacy better. We built and deployed VAX in ten participants' homes while they performed 17 common activities of daily living. Our evaluation results show that after training, VAX can use its onboard camera and microphone to detect approximately 15 out of 17 activities with an average accuracy of 90%. For these activities that can be detected using a camera and a microphone, VAX trains a per-home model for the privacy-preserving sensors. These models (average accuracy = 84%) require no in-situ user input. In addition, when VAX is augmented with just one labeled instance for the activities not detected by the VAX A/V pipeline (~2 out of 17), it can detect all 17 activities with an average accuracy of 84%. Our results show that VAX is significantly better than a baseline supervised-learning approach of using one labeled instance per activity in each home (average accuracy of 79%) since VAX reduces the user burden of providing activity labels by 8x (~2 labels vs. 17 labels).

Multiple instance learning from similarity-confidence bags

Multiple instance learning (MIL) is a classic weakly supervised learning approach, in which samples are grouped into bags that may contain varying numbers of instances. A bag is designated as positive if it contains at least one positive instance; otherwise, it is considered negative. Previous studies have consistently assumed that the bag labels are completely known. In fact, labeling every bag can be extremely challenging or even unfeasible due to the exorbitant expenses in terms of time and labor. Fortunately, it is much easier to obtain the similarity confidence, which represents the probability of two bags sharing the same label. How to employ it in MIL is worthy of study. Inspired by the above study, we present the first attempt to investigate MIL from similarity-confidence bags. Therefore, this paper proposes a new framework for training bag-level classifiers that adheres to the principle of empirical risk minimization. Moreover, we theoretically derive a generalization error bound to guarantee model convergence. Finally, we implement risk correction to mitigate potential over-fitting problem and provide theoretical consistency. Numerical experiments on eight datasets further validate the effectiveness of the proposed bag-level classifier.