Robust Learning Research Articles

LW-BPNN: A Novel Feature Extraction Method for Rolling Bearing Fault Diagnosis

Efficiently diagnosing bearing faults is of paramount importance to enhance safety and reduce maintenance costs for rotating machinery. This paper introduces a novel bearing fault diagnosis method (LW-BPNN), which combines the rich properties of Legendre multiwavelet bases with the robust learning capabilities of a BP neural network (BPNN). The proposed method not only addresses the limitations of traditional deep networks, which rely on manual feature extraction and expert experience but also eliminates the complexity associated with designing and training deep network architectures. To be specific, only two statistical parameters, root mean square (RMS) and standard deviation (SD), are calculated on different Legendre multiwavelet decomposition levels to thoroughly represent more salient and comprehensive fault characteristics by using several scale and wavelet bases with various regularities. Then, the mapping relation between the extracted features and the health conditions of the bearing is automatically learned by the simpler BPNN classifier rather than the complex deep network structure. Finally, a few experiments on a popular bearing dataset are implemented to verify the effectiveness and robustness of the presented method. The experimental findings illustrate that the proposed method exhibits a high degree of precision in diagnosing various fault patterns. It outperforms other methods in terms of diagnostic accuracy, making it a viable and promising solution for real-world industrial applications in the field of rotating machinery.

Dynamic Loss For Robust Learning.

Label noise and class imbalance are common challenges encountered in real-world datasets. Existing approaches for robust learning often focus on addressing either label noise or class imbalance individually, resulting in suboptimal performance when both biases are present. To bridge this gap, this work introduces a novel meta-learning-based dynamic loss that adapts the objective functions during the training process to effectively learn a classifier from long-tailed noisy data. Specifically, our dynamic loss consists of two components: a label corrector and a margin generator. The label corrector is responsible for correcting noisy labels, while the margin generator generates per-class classification margins by capturing the underlying data distribution and the learning state of the classifier. In addition, we employ a hierarchical sampling strategy that enriches a small amount of unbiased metadata with diverse and challenging samples. This enables the joint optimization of the two components in the dynamic loss through meta-learning, allowing the classifier to effectively adapt to clean and balanced test data. Extensive experiments conducted on multiple real-world and synthetic datasets with various types of data biases, including CIFAR-10/100, Animal-10N, ImageNet-LT, and Webvision, demonstrate that our method achieves state-of-the-art accuracy. The code for our approach will soon be made publicly available.

GAN-assisted Road Segmentation from Satellite Imagery

Geo-information extraction from satellite imagery has become crucial to carry out large-scale ground surveys in a short amount of time. With the increasing number of commercial satellites launched into orbit in recent years, high-resolution RGB color remote sensing imagery has attracted a lot of attention. However, because of the high cost of image acquisition and even more complicated annotation procedures, there are limited high-resolution satellite datasets available. Compared to close-range imagery datasets, existing satellite datasets have a much lower number of images and cover only a few scenarios (cities, background environments, etc. ). They may not be sufficient for training robust learning models that fit all environmental conditions or be representative enough for training regional models that optimize for local scenarios. Instead of collecting and annotating more data, using synthetic images could be another solution to boost the performance of a model. This study proposes a GAN-assisted training scheme for road segmentation from high-resolution RGB color satellite images, which includes three critical components: a) synthetic training sample generation, b) synthetic training sample selection, and c) assisted training strategy. Apart from the GeoPalette and cSinGAN image generators introduced in our prior work, this paper in detail explains how to generate new training pairs using OpenStreetMap (OSM) and introduces a new set of evaluation metrics for selecting synthetic training pairs from a pool of generated samples. We conduct extensive quantitative and qualitative experiments to compare different image generators and training strategies. Our experiments on the downstream road segmentation task show that 1) our proposed metrics are more aligned with the trained model performance compared to commonly used GAN evaluation metrics such as the Fréchet inception distance (FID); and 2) by using synthetic data with the best training strategy, the model performance, mean Intersection over Union (mean IoU), is improved from 60.92% to 64.44%, when 1,000 real training pairs are available for learning, which reaches a similar level of performance as a model that is standard-trained with 4,000 real images (64.59%), i.e. , enabling a 4-fold reduction in real dataset size.

Influence of learning motivation on high school students' mathematical creative thinking ability

This research aims to investigate the impact of learning motivation on students' mathematical creativity. Employing correlational research techniques with a quantitative approach, the study focuses on the entire Class X population at MIPA Rajagaluh High School. The purposive sampling method was applied to select 30 students from Class X MIPA 6. Research tools included tests on creative thinking and surveys to gauge students' enthusiasm for learning. Data analysis involved correlation, linearity, and normalcy tests. The results revealed a noteworthy correlation between learning motivation and enhancing students' mathematical creative thinking skills. Learning motivation has a statistically significant influence on mathematical creative thinking abilities. These results underscore the importance of cultivating and sustaining students' learning motivation. It is recommended that teachers actively provide encouragement and guidance to nurture robust learning motivation among students.

Modeling user choice behavior under data corruption: Robust learning of the latent decision threshold model

Recent years have witnessed the emergence of many new mobile apps and user-centered systems that interact with users by offering choices with rewards. These applications have been promising to address challenging societal problems such as congestion in transportation and behavior changes for healthier lifestyles. Considerable research efforts have been devoted to model the user behaviors in these new applications. However, as real-world user data is often prone to data corruptions, the success of these models hinges on a robust learning method. Building on the recently proposed Latent Decision Threshold model, this article shows that, among the existing robust learning frameworks, the L 0-norm-based framework can outperform other state-of-the-art methods in terms of prediction accuracy and model estimation. And based on the L 0-norm framework, we further develop a user screening algorithm to identify potential bad actors.

Development of A Learning Model for Blended Collaborative Knowledge Construction in Vocational Education

Vocational Education readies students for specific careers and industries. A robust learning approach is crucial to meet modern workforce demands and foster knowledge exchange. This article introduces the origination of a Learning Model for Blended Collaborative Knowledge Construction in Vocational Education. This model merges methods from blended learning with collaborative strategies for constructing knowledge, fuelling student engagement, skills acquisition, and shared knowledge creation. The research utilizes the Delphi method to accumulate convincing proof for learning models prioritizing blended collaboration in knowledge generation. The sample pool comprises 60 students spanning two Zigong Vocational Training Institute classes, gathered via selective sampling. Data collection involved pre and post-testing of the student’s performance in the experimental and control groups, reinforced by the satisfaction survey from the experimental group. Data analysis employed statistical tools like percentages, means, standard deviations, and the t-test for related samples. The study found that pretest scores and SPSS data analysis showed that the f-test significant value is .702, which exceeds .05. This implies that, under the assumption of equal variance, the mean equivalence t-test findings for the first row of data, as shown in the table above, show that the experimental class averages 81.02 and the control class 81.05. The experimental and control groups' pretest scores are not significantly different, as the computed p-value of .984 is larger than .05. Thus, teaching practice is reasonable. The experimental group's student satisfaction measure averaged 4.27, ranging from 4.10 to 4.43, indicating that students are content with their academics. The control group's student satisfaction measure averaged 2.87, ranging from 2.50 to 3.10, indicating low satisfaction with learning. Students in a vocational education setting were compared using a hybrid collaborative knowledge creation learning model, demonstrating its potential to enhance student satisfaction and outcomes.

Robust learning of parsimonious deep neural networks

We propose a simultaneous learning and pruning algorithm capable of identifying and eliminating irrelevant structures in a neural network during the early stages of training. Simultaneous learning and pruning presents serious challenges such as premature pruning of units that can lead to poor performance. Our method is capable of overcoming such challenges and is robust, i.e., it gives consistent pruning levels and prediction accuracy regardless of weight initialization or the size of the starting network. Thus, it allows for substantial computational cost savings during training, besides that of inference, and it can enable the training of very deep networks when transfer learning to obtain fully trained deep networks that can be pruned after training is not possible. Our approach is based on variational inference principles using Gaussian scale mixture priors on the neural network weights. The variational posterior distribution of Bernoulli random variables multiplying the units/filters is learned, similarly to adaptive dropout. We construct a novel hyper-prior distribution over the prior parameters to impose properties crucial for their optimal selection and the overall robustness of our algorithm. It is shown in the context of our algorithm that the parameters of the posterior distributions practically converge to either 0 or 1, establishing a deterministic final network. Convergence is proved analytically based on dynamical systems theory and from the theoretical results, practical pruning conditions are established. The proposed algorithm is evaluated on the MNIST, CIFAR-10 and ImageNet data sets and the commonly used fully connected, convolutional and residual architectures LeNet, VGG16 and ResNet. The simulations show that our method typically achieves better pruning levels while maintaining test-accuracy on par with state-of the-art methods for structured pruning in a manner robust with respect to network initialization and initial size.

Time Series Prediction for Energy Consumption of Computer Numerical Control Axes Using Hybrid Machine Learning Models

The prediction of energy-related time series for computer numerical control (CNC) machine tool axes is an essential enabler for the shift towards autonomous and intelligent production. In particular, a precise prediction of energy consumption is needed to determine the environmental impact of a product and the optimization of its production. For this purpose, a novel approach for predicting high-frequency time series of numerically controlled axes based on the program code to be executed is presented. The method involves simulative preprocessing of the input NC code to determine each axis’s acceleration, velocity, and process force. Combined with the material removal rate, these variables are input for a machine learning (ML) model that delivers axis-specific high-frequency time series predictions. Compared to common approaches, it is thus possible to make predictions for the variable energy consumption of machine tools for any tool path or target resolution in the time domain. Experiments show that this approach achieves a high precision when a robust learning data basis is available. For the X-, Y-, and Z-axis, errors of 0.2%, −1.09%, and 0.09% for aircut and of 0.15%, −3.55%, and 0.08% for material removal can be achieved. The potentials for further improvement are identified systematically.

A novel approach for identifying customer groups for personalized demand-side management services using household socio-demographic data

Demand-side management (DSM) is crucial to smart energy systems. This paper presents a data-driven approach for understanding the relationship between energy consumption patterns and household characteristics to better provide DSM services. The proposed method uses a robust learning fuzzy c-Means clustering algorithm to automatically generate the optimal number of customer groups for DSM, and then uses symmetric uncertainty techniques to identify the identified load patterns and socio-demographic characteristics as the features for training a deep learning model. The model obtained can be used to predict the possibility of DSM group membership for a given household. This approach can be applied even in situations where smart meter data are not available, such as when new customers are added to the system or when residents change, or due to privacy concerns. The proposed model is evaluated comprehensively, including prediction accuracy, comparison with other baselines, and case studies for DSM. The results demonstrate the usefulness of weekly energy consumption data and associated household socio-demographic information for distinguishing between different consumer groups, the effectiveness of the proposed model, and the potential for targeted DSM strategies such as time-of-use pricing, energy efficiency measures, and demand response programs.

GMRLNet: A graph-based manifold regularization learning framework for placental insufficiency diagnosis on incomplete multimodal ultrasound data.

Multimodal analysis of placental ultrasound (US) and microflow imaging (MFI) could greatly aid in the early diagnosis and interventional treatment of placental insufficiency (PI), ensuring a normal pregnancy. Existing multimodal analysis methods have weaknesses in multimodal feature representation and modal knowledge definitions and fail on incomplete datasets with unpaired multimodal samples. To address these challenges and efficiently leverage the incomplete multimodal dataset for accurate PI diagnosis, we propose a novel graph-based manifold regularization learning (MRL) framework named GMRLNet. It takes US and MFI images as input and exploits their modality-shared and modality-specific information for optimal multimodal feature representation. Specifically, a graph convolutional-based shared and specific transfer network (GSSTN) is designed to explore intra-modal feature associations, thus decoupling each modal input into interpretable shared and specific spaces. For unimodal knowledge definitions, graph-based manifold knowledge is introduced to describe the sample-level feature representation, local inter-sample relations, and global data distribution of each modality. Then, an MRL paradigm is designed for inter-modal manifold knowledge transfer to obtain effective cross-modal feature representations. Furthermore, MRL transfers the knowledge between both paired and unpaired data for robust learning on incomplete datasets. Experiments were conducted on two clinical datasets to validate the PI classification performance and generalization of GMRLNet. State-of-the-art comparisons show the higher accuracy of GMRLNet on incomplete datasets. Our method achieves 0.913 AUC and 0.904 balanced accuracy (bACC) for paired US and MFI images, as well as 0.906 AUC and 0.888 bACC for unimodal US images, illustrating its application potential in PI CAD systems.

Distributionally Robust Learning With Stable Adversarial Training

Machine learning algorithms with empirical risk minimization are vulnerable under distributional shifts due to the greedy adoption of all the correlations found in training data. There is an emerging literature on tackling this problem by minimizing the worst-case risk over an uncertainty set. However, existing methods mostly construct ambiguity sets by treating all variables equally regardless of the stability of their correlations with the target, resulting in the overwhelmingly-large uncertainty set and low confidence of the learner. In this paper, we propose a novel Stable Adversarial Learning (SAL) algorithm that leverages heterogeneous data sources to construct a more practical uncertainty set and conduct differentiated robustness optimization, where covariates are differentiated according to the stability of their correlations with the target. We theoretically show that our method is tractable for stochastic gradient-based optimization and provide the performance guarantees for our method. Empirical studies on both simulation and real datasets validate the effectiveness of our method in terms of uniformly good performance across unknown distributional shifts.

Ensemble Provably Robust Learn-to-Optimize Approach for Security-Constrained Unit Commitment

Security-constrained unit commitment (SCUC) is the basis for power systems and markets operation, which is solved periodically via mixed-integer programming (MIP) with limited input data changes to historical solved instances. This paper proposes an ensemble provably robust learn-to-optimize approach (EPR-L2O) for transforming the MIP-based SCUC into the tractable convex problem with predictable solving time by identifying the integer part of MIP, denoted by unit status integer solution. It formulates the SCUC model for unit status integer solution generation, prunes the generated integer solutions, clusters the units based on their operation status, and proposes the provably robust learning approach to training the unit status integer solution identifiers, which are provably robust against the input adversarial perturbation. After learning, the integer solutions from different identifiers are assembled to identify the integer variables of SCUC. Case study verifies that: 1) EPR-L2O achieves 6.91, 19.71, and 22.14 times acceleration in the 30-bus, 118-bus, and 300-bus systems than the Branch-and-Bound; 2) compared to cross-entropy-based learning, EPR-L2O reduces the MIP gap by 0.031%, 0.21%, and 0.0084% in the 30-bus, 118-bus, and 300-bus systems, indicating its operation economy and robustness.

Enhancing hyperspectral remote sensing image classification using robust learning technique

Advanced sensor tech integrates into diverse applications, including remote sensing, robotics, and IoT. Combining artificial intelligence (AI) with sensors enhances their capabilities, creating smart sensors, revolutionizing remote sensing and Internet of Things (IoT). This synergy forms a potent technology in the field. This study carries out a comprehensive analysis of the progress made in Hyperspectral sensors and AI-based classification techniques that are employed in remote sensing fields that utilize hyperspectral images. The classification of images obtained from Hyperspectral Sensors (HSS) has emerged as a prominent research subject within the domain of remote sensing. HSS offer a wealth of information across numerous spectral bands, supporting diverse applications such as land cover classification, environmental monitoring, agricultural assessment, change detection, and more. However, the abundance of data present in HSS also poses the challenge called the curse of dimensionality. The reduction of data dimensionality is crucial before applying any machine learning model to achieve optimal results. The present study introduces a new hybrid strategy combining the Back-Propagation algorithm with a variable adaptive momentum (BPVAM) and principal component analysis (PCA) for the purpose of classifying hyperspectral images. PCA is first applied to obtain an optimal set of discriminative features by eliminating highly correlated and redundant features. These features are then fed into the BPVAM model for classification. The addition of the momentum term in the weight update equation of the backpropagation algorithm helped achieve faster convergence with high accuracy. The proposed model was subjected to evaluation through experiments conducted on two benchmark datasets. These results indicated that the hybrid model based on BPVAM with PCA is an efficient technique for HSS classification.

Statistical word segmentation succeeds given the minimal amount of exposure.

One of the first tasks in language acquisition is word segmentation, a process to extract word forms from continuous speech streams. Statistical approaches to word segmentation have been shown to be a powerful mechanism, in which word boundaries are inferred from sequence statistics. This approach requires the learner to represent the frequency of units from syllable sequences, though accounts differ on how much statistical exposure is required. In this study, we examined the computational limit with which words can be extracted from continuous sequences. First, we discussed why two occurrences of a word in a continuous sequence is the computational lower limit for this word to be statistically defined. Next, we created shortsyllable sequences that contained certain words either two or four times. Learners were presented with these syllable sequences one at a time, immediately followed by a test of the novel words from these sequences. We found that, with the computationally minimal amount of two exposures, words were successfully segmented from continuous sequences. Moreover, longer syllable sequences providing four exposures to words generated more robust learning results. The implications of these results are discussed in terms of how learners segment and store the word candidates from continuous sequences.

Robust training of median dendritic artificial neural networks for time series forecasting

Although artificial neural network models have produced very successful results in the time series forecasting problem in recent years, an outlier or outliers in the data set adversely affect the forecasting performance of the artificial neural network models. Dendritic neuron model artificial neural networks which are the most similar neural network model to an artificial neural network model are also adversely affected by outliers in the data set like many artificial neural network models in the literature. In this study, to prevent the dendritic neuron model artificial neural networks from being affected by the outliers in the data set; a robust learning algorithm based on Talwar's m estimator, median statistics to prevent the effect of outliers in the inputs, and a new data pre-processing method are used together in a network structure. In addition, the training of the proposed artificial neural network model is carried out with the symbiotic organism search algorithm. To evaluate the performance of the proposed method, analyses are carried out over the closing prices of the time series of Spain, Italy and German stock exchanges in certain years. According to the results of the analysis of the time series of the relevant stock exchanges, both in their original state and by injecting outliers into the time series, the proposed method has superior forecasting performance even when the time series contains outliers and does not contain outliers.

Federated Deep Learning for Wireless Capsule Endoscopy Analysis: Enabling Collaboration Across Multiple Data Centers for Robust Learning of Diverse Pathologies

Wireless capsule endoscopy (WCE) is a revolutionary diagnostic method for small bowel pathology. The manual perusal of the resulting lengthy and redundant videos is cumbersome. Automated analysis of WCE video frames is an intricate data modeling task because of the diverse representations of anomalies caused by inappropriate capture conditions. Deep neural networks require training to learn diverse pathological manifestations utilizing heterogeneous data collected from multiple institutions. However, the accessibility of WCE data poses privacy concerns for multiple centers. The efficient learning of heterogeneous data distributed over multiple institutions in a privacy-preserving fashion has become a challenge hampering the adoption of AI-based diagnoses in clinical practice. Prior studies have contrived extensive data augmentation and the generation of synthetic images from the same center. However, models trained at one center are at risk of a lack of generalization for a global deployment. Federated learning (FL) is a novel paradigm in which models learn from distributed data and share knowledge without accessing the data themselves. This study proposes an FL framework for multiple anomaly classifications of WCE frames, elaborating on the potential of collaborative learning from multiple data centers on the edge. Our empirical results prove that the proposed decentralized approach can learn the generalized features of WCE frames. Validating heterogeneous test sets revealed a 10–12% improvement in performance for decentralized models based on FL compared to the best-case performance of centralized models, demonstrating the potential of the federated framework to support multiple anomaly classification of WCE frames while preserving data privacy across various clinical setups.

Stacking integrated learning model via ELM and GRU with mixture correntropy loss for robust state of health estimation of lithium-ion batteries

Accurate estimation of the state of health (SOH) is crucial for the safe and stable operation of lithium-ion batteries. However, since the labeled values may contain non-Gaussian noise which may lead to a decrease in the effectiveness of traditional data-driven based estimation methods. Therefore, a novel robust stacking integrated learning model (ILM) is proposed to enable accurate SOH estimation under non-Gaussian noises (or outliers) conditions. The mixture correntropy loss (MCL) is used in original extreme learning machine (ELM) and the gated recurrent unit (GRU) frameworks to develop novel robust learning models, i.e MCL-ELM and MCL-GRU, and they are used as the sub-models of the proposed ILM, which takes into account the generalization performance of the model and the overall trend of SOH over the time series. In addition, the entropy weighting method that can well reflect the differences between the sub-models is utilized to reduce the complexity of the stacked ILM. Furthermore, the local tangent space alignment is used to capture the local relationships between different loops to enhance the effectiveness of the extracted health features. Several numerical experiments are performed under different cases to validate the estimation effect of the proposed model by using two publicly available datasets, and the experimental results demonstrate that the maximum RMSE, MAE, and MAPE values of the proposed model are 1.391%, 0.932%, and 1.480%, respectively, under non-Gaussian noises conditions.

The Development of Artificial Intelligence in Career Initiation Education and Implications for China

Artificial intelligence (AI) is currently exerting a significant impact on the development of career guidance education, facilitating personalized guidance and data-driven decision-making for students. The historical and evolutionary trajectory of AI-driven career guidance education can be traced back to its early stages as assistive functionalities, which have now advanced to encompass robust learning applications, such as multimedia and interactive features, machine learning, and natural language processing. Notably, AI has transcended its conventional role in vocational development and expanded into the realms of social and emotional learning. The complexity of AI research in international contexts necessitates consideration of various factors, including cognitive development, parental involvement and supervision, and cultural backgrounds. Despite certain limitations in utilizing AI for career exploration, it has brought numerous impacts and insights. These primarily manifest in the areas of data-driven decision-making and the outlook for career exploration, the demand for cultural sensitivity in AI-driven career guidance, and the provision of personalized career guidance through artificial intelligence in education.

Conv-eRVFL: Convolutional Neural Network Based Ensemble RVFL Classifier for Alzheimer's Disease Diagnosis.

As per the latest statistics, Alzheimer's disease (AD) has become a global burden over the following decades. Identifying AD at the intermediate stage became challenging, with mild cognitive impairment (MCI) utilizing credible biomarkers and robust learning approaches. Neuroimaging techniques like magnetic resonance imaging (MRI) and positron emission tomography (PET) are practical research approaches that provide structural atrophies and metabolic variations. With the help of MRI and PET scans, metabolic and structural changes in AD patients can be visible even ten years before the disease's onset. This paper proposes a novel wavelet packet transform-based structural and metabolic image fusion approach using MRI and PET scans. An eight-layer trained CNN extracts features from multiple layers and these features are fed to an ensemble of non-iterative random vector functional link (RVFL) models. The RVFL network incorporates the s-membership fuzzy function as an activation function that helps overcome outliers. Lastly, outputs of all the customized RVFL classifiers are averaged and fed to the RVFL classifier to make the final decision. Experiments are performed over Alzheimer's Disease Neuroimaging Initiative (ADNI) dataset, and classification is made over CN vs. AD vs. MCI. The model performance obtained is decent enough to prove the effectiveness of the fusion-based ensemble approach.

Confidence-aware self-supervised learning for dense monocular depth estimation in dynamic laparoscopic scene

This paper tackles the challenge of accurate depth estimation from monocular laparoscopic images in dynamic surgical environments. The lack of reliable ground truth due to inconsistencies within these images makes this a complex task. Further complicating the learning process is the presence of noise elements like bleeding and smoke. We propose a model learning framework that uses a generic laparoscopic surgery video dataset for training, aimed at achieving precise monocular depth estimation in dynamic surgical settings. The architecture employs binocular disparity confidence information as a self-supervisory signal, along with the disparity information from a stereo laparoscope. Our method ensures robust learning amidst outliers, influenced by tissue deformation, smoke, and surgical instruments, by utilizing a unique loss function. This function adjusts the selection and weighting of depth data for learning based on their given confidence. We trained the model using the Hamlyn Dataset and verified it with Hamlyn Dataset test data and a static dataset. The results show exceptional generalization performance and efficacy for various scene dynamics, laparoscope types, and surgical sites.