Classifier Training Research Articles

Self-learning salp swarm algorithm for global optimization and its application in multi-layer perceptron model training

Optimization problems are common across various fields, and one effective solution is the swarm intelligence algorithm.It is essential for the algorithm to deliver high-quality solutions for problems with varying characteristics. However, most existing swarm intelligence rely on fixed and monotonic search strategies, which limits their ability to handle the diverse and complex situations encountered when solving real-world optimization problems with unknown fitness landscapes. To extend the applicability of swarm intelligence and thus offer users an efficient black-box optimizer for various applications, a novel self-learning mechanism is proposed and applied to the Salp Swarm Algorithm (SSA) to develop the self-learning salp swarm algorithm (SLSSA) in this paper. In SLSSA, four distinct search strategies, including a novel multiple food sources search strategy, are adopted to strengthen the search agents’ abilities to conquer various difficulties in the search space. To improve the efficiency of the search process, the self-learning strategy dynamically determines the execution probability of each search strategy according to the quality of solutions it produced previously. Moreover, a parameter setting method is proposed in this paper, which eliminates the need for a trial-and-error approach and allows for straightforward configuration of the parameters that optimize the performance of SLSSA. In comparison with several highly regarded state-of-the-art peer algorithms, the performance of SLSSA in solving the CEC2014 benchmark functions was thoroughly examined. Subsequently, SLSSA was applied to train multi-layer perceptron classifiers and test on the UCI machine-learning datasets. The experimental results and analysis on benchmark functions and multi-layer perceptron classifier training problems demonstrate that SLSSA outperforms the competing algorithms in terms of solution accuracy, stability, and overall convergence speed. Moreover, computational time comparisons reveal that SLSSA achieves significant performance improvement with only a marginal increase in time cost compared to the original SSA.

Cross-instrument optical coherence tomography-angiography (OCTA)-based prediction of age-related macular degeneration (AMD) disease activity using artificial intelligence

This study investigates the efficacy of predicting age-related macular degeneration (AMD) activity through deep neural networks (DNN) using a cross-instrument training dataset composed of Optical coherence tomography-angiography (OCTA) images from two different manufacturers. A retrospective cross-sectional study analyzed 2D vascular en-face OCTA images from Heidelberg Spectralis (1478 samples: 1102 training, 276 validation, 100 testing) and Optovue Solix (1003 samples: 754 training, 189 validation, 60 testing). OCTA scans were labeled based on clinical diagnoses and adjacent B-scan OCT fluid information, categorizing activity into normal, dry AMD, active wet AMD, and wet AMD in remission. Experiments explored cross-instrument disease classification using separate and combined datasets for training the DNN. Testing involved 100 Heidelberg and 60 Optovue samples. Training on Heidelberg data alone yielded 73% accuracy on Heidelberg images and 60% on Optovue images. Training on Optovue data alone resulted in 34% accuracy on Heidelberg and 85% on Optovue images. Combined training data from both instruments achieved 78% accuracy on Heidelberg and 76% on Optovue test sets. Results indicate that cross-instrument classifier training demonstrates high classification prediction accuracy, making cross-instrument training viable for future clinical applications. This implies that vascular morphology in OCTA can predict disease progression.

ATGAN: attention-based temporal GAN for EEG data augmentation in personal identification

With the development of brain–computer interfaces in recent years, deep learning models have been widely used in EEG-based personal identification. The training of complex models requires a large amount of data, but the acquisition of EEG signals is very time-consuming and energy-consuming. EEG data augmentation can reduce the overhead of EEG data acquisition and provide the data volume required by various complex models, which is conducive to applying EEG identification in practice. The generative adversarial network (GAN) has been applied to augment EEG data features. But the augmentation of temporal EEG signals is less studied. Therefore, this paper proposes a method of temporal EEG data augmentation based on an attention mechanism time series generative adversarial network (ATGAN), which can expand the original EEG dataset to facilitate the training of downstream deep learning classifiers. The experiment is conducted on the BCI Competition IV 2a dataset. The ATGAN model is trained with the temporal EEG signals after preprocessing, and the generated augmentation data by ATGAN is used to expand the original dataset. The accuracy of the EEG-based identification task on the augmented dataset is 7.78% higher than that on the original dataset, which proves the effectiveness of the proposed method.

Lymph node metastasis prediction from in-situ lung squamous cell carcinoma histopathology images using deep learning.

Lung squamous cell carcinoma (LUSC), a subtype of non-small cell lung cancer, represents a significant portion of lung cancer cases with distinct histologic patterns impacting prognosis and treatment. The current pathological assessment methods face limitations such as inter-observer variability, necessitating more reliable techniques. This study seeks to predict lymph node metastasis in LUSC using deep learning models applied to histopathology images of primary tumors, offering a more accurate and objective method for diagnosis and prognosis.Whole slide images (WSIs) from the Outdo-LUSC and TCGA-LUSC cohorts were used to train and validate deep-learning models. Multi-instance learning was applied, with patch-level predictions aggregated into WSI-level outcomes. The study employed the ResNet-18 network, transfer learning, and rigorous data preprocessing. To represent WSI features, innovative techniques like patch likelihood histogram (PLH) and bag of words (BoW) were used, followed by training of machine learning classifiers, including the ExtraTrees algorithm.The diagnostic model for lymph node metastasis showed strong performance, particularly using the ExtraTrees algorithm, as demonstrated by receiver operating characteristic (ROC) curves and Grad-CAM visualizations. The signature generated by the ExtraTrees algorithm, named LN_ISLUSCH (lymph node status-related in-situ lung squamous cell carcinoma histopathology), achieved an area under the curve (AUC) of 0.941 (95% CI: 0.926-0.955) in the training set and 0.788 (95% CI: 0.748-0.827) in the test set. Kaplan–Meier analyses confirmed that the LN_ISLUSCH model was a significant prognostic factor (p = 0.02).This study underscores the potential of artificial intelligence in enhancing diagnostic precision in pathology. The LN_ISLUSCH model stands out as a promising tool for predicting lymph node metastasis and prognosis in LUSC. Future studies should focus on larger and more diverse cohorts and explore the integration of additional omics data to further refine predictive accuracy and clinical utility.

Enhancing Intent Classifier Training with Large Language Model-generated Data

ABSTRACT Intent classification is essential in Natural Language Processing, serving applications like virtual assistants and customer service by categorizing user inputs into predefined classes. Despite its importance, the effectiveness of intent classifiers is often constrained by the scarcity of labeled data, as acquiring substantial, annotated datasets is costly and impractical. Data augmentation addresses this by expanding datasets with modified or synthetic examples, a common practice in computer vision but more complex in NLP due to the discrete nature of language. Traditional NLP data augmentation methods have been explored but exhibit limitations. This paper investigates the use of Large Language Models for generating labeled data to enhance intent classification. We explore whether LLM-generated data can effectively augment training sets, comparing its impact on intent classifier performance against traditional augmentation methods. Our study reveals that LLMs can generate diverse and realistic data, potentially improving classifier accuracy in low-data scenarios, thereby providing valuable insights into leveraging generative AI for NLP tasks in real-world applications.

SmartScanPCOS: A feature-driven approach to cutting-edge prediction of Polycystic Ovary Syndrome using Machine Learning and Explainable Artificial Intelligence

PolyCystic Ovarian Syndrome (PCOS) poses significant challenges to women's reproductive health due to its diagnostic complexity arising from a variety of symptoms, including hirsutism, anovulation, pain, obesity, hyperandrogenism, and oligomenorrhea, necessitating multiple clinical tests. Leveraging Artificial Intelligence (AI) in healthcare offers several benefits that can significantly impact patient care, streamline operations, and improve medical outcomes overall. This study presents an Explainable Artificial Intelligence (XAI)-driven PCOS smart predictor, structured as a hierarchical ensemble consisting of two tiers of Random Forest classifiers following extensive analysis of seven conventional classifiers and two additional stacking ensemble classifiers. An open-source data set comprising numerical parametric features linked to PCOS for classifier training was used. Moreover, to identify essential features for PCOS prediction three feature selection methods: Threshold-driven Optimized Principal Component Analysis (TOPCA), Optimized Salp Swarm (OSSM), and Threshold-driven Optimized Mutual Information Method (TOMIM) were fine-tuned through thresholding and improvisation to detect diverse attribute sets with varying numbers and combinations. Notably, the two-level Random Forest classifier model outperformed others with a remarkable 99.31 % accuracy by employing the top 17 features selected through the Threshold-driven Optimized Mutual Information Method (TOMIM) along with anoverallaccuracy of 99.32 % with 8 fold cross validation for 25 runs. The Smart predictor, constructed using Shapash - a Python library for Explainable Artificial Intelligence - was utilized to deploy the two-level Random Forest classifier model. Ensuring transparency and result reliability, visualizations from robust Explainable AI libraries were employed at different prediction stages for all considered classifiers in this study.

Classification of BPG-Based Lossy Compressed Noisy Images

Acquired remote sensing images can be noisy. This fact has to be taken into account in their lossy compression and classification. In particular, a specific noise filtering effect is usually observed due to lossy compression and this can be positive for classification. Classification can be also influenced by methodology of classifier learning. In this paper, we consider peculiarities of lossy compression of three-channel noisy images by better portable graphics (BPG) encoder and their further classification. It is demonstrated that improvement of data classification accuracy is not observed if a given image is compressed in the neighborhood of optimal operation point (OOP) and the classifier training is performed for the noisy image. Performance of neural network based classifier is studied. As demonstrated, its training for compressed remote sensing data is able to provide certain benefits compared to training for noisy (uncompressed) data. Examples for Sentinel data used in simulations are offered.

A Sign Language Recognition using Improved Grey Wolf Optimization based neural networks

Sign language was developed to enable the deaf and hard of hearing community to communicate with society and convey information. It is the primary means by which they can interact with each other, as well as with the general population. The automatic gesture recognition system described in this paper uses Gray Wolf Optimization (GWO) to identify the most relevant features from a set of gesture images, with the aim of improving model performance. The main steps of the system are explained, including image preprocessing, feature extraction, GWO-based feature selection, classifier training, and evaluation. The proposed model achieved an accuracy of 99.9%.

Sequential label shift detection in classification data: An application to dengue fever.

Classifiers have been developed to help diagnose dengue fever in patients presenting with febrile symptoms. However, classifier predictions often rely on the assumption that new observations come from the same distribution as training data. If the population prevalence of dengue changes, as would happen with a dengue outbreak, it is important to raise an alarm as soon as possible, so that appropriate public health measures can be taken and also so that the classifier can be re-calibrated. In this paper, we consider the problem of detecting such a change in distribution in sequentially-observed, unlabeled classification data. We focus on label shift changes to the distribution, where the class priors shift but the class conditional distributions remain unchanged. We reduce this problem to the problem of detecting a change in the one-dimensional classifier scores, leading to simple nonparametric sequential changepoint detection procedures. Our procedures leverage classifier training data to estimate the detection statistic, and converge to their parametric counterparts in the size of the training data. In simulated outbreaks with real dengue data, we show that our method outperforms other detection procedures in this label shift setting.

Acoustic Feature Extraction Based on Wavelet Transform for Industrial Induction Motor-Driven Belt Conveyor Condition Monitoring

The industrial induction motor-driven belt conveyor is an essential component in manufacturing facilities. Any unexpected shutdown can lead to significant disruptions, resulting in financial losses amounting to thousands of dollars per hour. Unfortunately, efficient mechanisms for monitoring the conveyor's condition are often lacking. Therefore, it is crucial to ensure early, precise, and effective detection of malfunctions in belt conveyors. This necessitates the identification of distinctive anomalies stemming from initial damage to the rotating machinery and motor components. This paper presents a non-invasive acoustic monitoring technique designed specifically for industrial belt conveyors. The method employed relies on wavelet transform-based feature extraction, offering notable advantages in terms of classification accuracy, time efficiency, and the quantity of feature vectors required for classifier training.

Design of MATLAB-based Radiomics Classifier Training Simulator Powered by Pyradiomics

Design of MATLAB-based Radiomics Classifier Training Simulator Powered by Pyradiomics

A novel and efficient risk minimisation-based missing value imputation algorithm

Missing value imputation (MVI) is a key task in data science, in which learning models are built from incomplete data. In contrast to externally driven MVI algorithms, this study proposes a novel risk minimisation-based MVI algorithm (RM-MVI) that considers both the internal characteristics of missing data and the external performance for specific classification applications. RM-MVI is technically designed for labelled data and is applied in two stages: filling with structural risk minimisation (SRM) and refining with empirical risk minimisation (ERM). In the filling stage, an autoencoder with a single hidden layer is trained on the original dataset without missing values. Missing values are first initialised with random numbers, and the imputation values are then preliminarily optimised based on the derived updating rule to minimise the structural risk-oriented objective function. After the imputation values have been preliminarily optimised in the filling stage, a neural-network-based classifier is trained in the refining stage to optimise the imputation values sophisticatedly by reducing the empirical risk. Experiments were conducted on several benchmark datasets to validate the feasibility, rationality, and effectiveness of the proposed RM-MVI algorithm. The results show that (1) the optimisation processes of the imputation values corresponding to the SRM and ERM are convergent so that the optimised imputation values can be obtained; (2) SRM can ensure distribution consistency of the imputation values that are preliminarily optimised in the filling stage, while ERM can optimise the imputation values sophisticatedly in the refining stage, which is more helpful for classifier training; and (3) the RM-MVI algorithm can yield considerably better MVI performance on benchmark datasets than 11 well-known MVI algorithms, such as a 26% higher distribution consistency ratio and 2% to 5% higher testing accuracies for 6 classifiers on average. This demonstrates that RM-MVI is a viable approach for addressing MVI problems.

Learning-based CU partition prediction for fast panoramic video intra coding

In recent years, Virtual Reality (VR) technology has been applied in various aspects of life. As an indispensable component of VR, panoramic video offers viewers immersive experiences, thereby attracting widespread attention. The high resolution of panoramic video poses significant challenges for its storage and transmission, requiring efficient video compression. The Versatile Video Coding (VVC) exhibits superior coding efficiency, albeit at the expense of a several dozen-fold increase in encoding complexity. In this paper, we propose a fast Coding Unit (CU) partition algorithm for intra-coding based on Support Vector Machine (SVM), which selects appropriate features for online training of multiple classifiers based on frame texture characteristics. Additionally, in response to distortion issues in panoramic video coding, we divide video frames into different regions and adjust the Quantization Parameter (QP) values of CUs within each region to improve the encoding efficiency. Experimental results show that the proposed method achieves an average encoding time reduction of 46.21%, with an average BDBR increase of only 1.01%, achieving a good trade-off between time-saving and encoding efficiency.

Self-Supervised Representation Distribution Learning for Reliable Data Augmentation in Histopathology WSI Classification.

Multiple instance learning (MIL) based whole slide image (WSI) classification is often carried out on the representations of patches extracted from WSI with a pre-trained patch encoder. The performance of classification relies on both patch-level representation learning and MIL classifier training. Most MIL methods utilize a frozen model pre-trained on ImageNet or a model trained with self-supervised learning on histopathology image dataset to extract patch image representations and then fix these representations in the training of the MIL classifiers for efficiency consideration. However, the invariance of representations cannot meet the diversity requirement for training a robust MIL classifier, which has significantly limited the performance of the WSI classification. In this paper, we propose a Self-Supervised Representation Distribution Learning framework (SSRDL) for patch-level representation learning with an online representation sampling strategy (ORS) for both patch feature extraction and WSI-level data augmentation. The proposed method was evaluated on three datasets under three MIL frameworks. The experimental results have demonstrated that the proposed method achieves the best performance in histopathology image representation learning and data augmentation and outperforms state-of-the-art methods under different WSI classification frameworks. The code is available at https://github.com/lazytkm/SSRDL.

Personalized whole-brain activity patterns predict human corticospinal tract activation in real-time.

Transcranial magnetic stimulation (TMS) interventions could feasibly treat stroke-related motor impairments, but their effects are highly variable. Brain state-dependent TMS approaches are a promising solution to this problem, but inter-individual variation in lesion location and oscillatory dynamics can make translating them to the poststroke brain challenging. Personalized brain state-dependent approaches specifically designed to address these challenges are therefore needed. As a first step towards this goal, we tested a novel machine learning-based EEG-TMS system that identifies personalized brain activity patterns reflecting strong and weak corticospinal tract (CST) output (strong and weak CST states) in healthy adults in real-time. Participants completed a single-session study that included the acquisition of a TMS-EEG-EMG training dataset, personalized classifier training, and real-time EEG-informed single pulse TMS during classifier-predicted personalized CST states. MEP amplitudes elicited in real-time during personalized strong CST states were significantly larger than those elicited during personalized weak and random CST states. MEP amplitudes elicited in real-time during personalized strong CST states were also significantly less variable than those elicited during personalized weak CST states. Personalized CST states lasted for ~1-2 seconds at a time and ~1 second elapsed between consecutive similar states. Individual participants exhibited unique differences in spectro-spatial EEG patterns between personalized strong and weak CST states. Our results show for the first time that personalized whole-brain EEG activity patterns predict CST activation in real-time in healthy humans. These findings represent a pivotal step towards using personalized brain state-dependent TMS interventions to promote poststroke CST function.

Enhancing Inter-Patient Performance for Arrhythmia Classification with Adversarial Learning Using Beat-Score Maps

Research on computer-aided arrhythmia classification is actively conducted, but the limited generalization capacity constrains its applicability in practical clinical settings. One of the primary challenges in deploying such techniques in real-world scenarios is the inter-patient variability and the consequent performance degradation. In this study, we leverage our previous innovation, the n-beat-score map (n-BSM), to introduce an adversarial framework to mitigate the issue of poor performance in arrhythmia classification within the inter-patient paradigm. The n-BSM is a 2D representation of the ECG signal, capturing its constituent beat characteristics through beat-score vectors derived from a pre-trained beat classifier. We employ adversarial learning to eliminate patient-dependent features during the training of the beat classifier, thereby generating the patient-independent n-BSM (PI-BSM). This approach enables us to concentrate primarily on the learning characteristics associated with beat type rather than patient-specific features. Through a beat classifier pre-trained with adversarial learning, a series of beat-score vectors are generated for the beat segments that make up a given ECG signal. These vectors are then concatenated chronologically to form a PI-BSM. Utilizing PI-BSMs as the input, an arrhythmia classifier is trained to differentiate between distinct types of rhythms. This approach yields a 14.27% enhancement in the F1-score in the MIT-BIH arrhythmia database and a 4.97% improvement in cross-database evaluation using the Chapman–Shaoxing 12-lead ECG database.

Hard sample mining-enabled supervised contrastive feature learning for wind turbine pitch system fault diagnosis

Abstract The presence of multiple failure severities in the wind turbine pitch system due to the long-term wear and tear poses challenges in accurately classifying the system’s health condition, thus increasing maintenance costs or damage risks. This paper proposes a novel method based on hard sample mining (HSM)-enabled supervised contrastive learning to address this problem. The proposed method leverages the powerful feature extraction capabilities of supervised contrastive learning to extract discriminative features from highly imbalanced data. Additionally, the method incorporates a cosine similarity-based HSM framework that constructs hard sample pairs within mini-batches during both the representation learning and classifier training phases, thereby improving the diagnostic performance of the model for hard samples. The proposed method achieves macro G-mean of 0.9991 and 0.9971 on two real datasets containing data on wind turbine pitch system cog belt fractures. These results indicate significantly superior fault diagnosis performance compared to existing methods, highlighting its potential for enhancing the reliability and efficiency of fault diagnosis in wind turbine pitch systems.

EEG Analyses of visual cue effects on executed movements

BackgroundIn electroencephalographic (EEG) or electrocorticographic (ECoG) experiments, visual cues are commonly used for timing synchronization but may inadvertently induce neural activity and cognitive processing, posing challenges when decoding self-initiated tasks. New methodTo address this concern, we introduced four new visual cues (Fade, Rotation, Reference, and Star) and investigated their impact on brain signals. Our objective was to identify a cue that minimizes its influence on brain activity, facilitating cue-effect free classifier training for asynchronous applications, particularly aiding individuals with severe paralysis. Results22 able-bodied, right-handed participants aged 18–30 performed hand movements upon presentation of the visual cues. Analysis of time-variability between movement onset and cue-aligned data, grand average MRCP, and classification outcomes revealed significant differences among cues. Rotation and Reference cue exhibited favorable results in minimizing temporal variability, maintaining MRCP patterns, and achieving comparable accuracy to self-paced signals in classification. Comparison with existing methodsOur study contrasts with traditional cue-based paradigms by introducing novel visual cues designed to mitigate unintended neural activity. We demonstrate the effectiveness of Rotation and Reference cue in eliciting consistent and accurate MRCPs during motor tasks, surpassing previous methods in achieving precise timing and high discriminability for classifier training. ConclusionsPrecision in cue timing is crucial for training classifiers, where both Rotation and Reference cue demonstrate minimal variability and high discriminability, highlighting their potential for accurate classifications in online scenarios. These findings offer promising avenues for refining brain-computer interface systems, particularly for individuals with motor impairments, by enabling more reliable and intuitive control mechanisms.

Adaptive Evolutionary Computing Ensemble Learning Model for Sentiment Analysis

Standard machine learning and deep learning architectures have been widely used in the field of sentiment analysis, but their performance is unsatisfactory if the input texts are short (e.g., social media posts). Specifically, the accuracy of standard machine learning methods crucially depends on the richness and completeness of the features used to represent the texts, and in the case of short messages, it is often difficult to obtain high-quality features. Conversely, methods based on deep learning can achieve better expressiveness, but these methods are computationally demanding and often suffer from over-fitting. This paper proposes a new adaptive evolutionary computational integrated learning model (AdaECELM) to overcome the problems encountered by traditional machine learning and deep learning models in sentiment analysis for short texts. AdaECELM consists of three phases: feature selection, sub classifier training, and global integration learning. First, a grid search is used for feature extraction and selection of term frequency-inverse document frequency (TF-IDF). Second, cuckoo search (CS) is introduced to optimize the combined hyperparameters in the sub-classifier support vector machine (SVM). Finally, the training set is divided into different feature subsets for sub-classifier training, and then the trained sub-classifiers are integrated and learned using the AdaBoost integrated soft voting method. Extensive experiments were conducted on six real polar sentiment analysis data sets. The results show that the AdaECELM model outperforms the traditional ML comparison methods according to evaluation metrics such as accuracy, precision, recall, and F1-score in all cases, and we report an improvement in accuracy exceeding 4.5%, the second-best competitor.

Output Regularization With Cluster-Based Soft Targets.

While supervised learning of over-parameterized neural networks achieved state-of-the-art performance in image classification, it tends to over-fit the labeled training samples to give inferior generalization ability. Output regularization deals with over-fitting by using soft targets as additional training signals. Although clustering is one of the most fundamental data analysis tools for discovering general-purpose and data-driven structures, it has been ignored in existing output regularization approaches. In this article, we leverage this underlying structural information by proposing Cluster-based soft targets for Output Regularization (CluOReg). This approach provides a unified way for simultaneous clustering in embedding space and neural classifier training with cluster-based soft targets via output regularization. By explicitly calculating a class relationship matrix in the cluster space, we obtain classwise soft targets shared by all samples in each class. Results of image classification experiments under various settings on a number of benchmark datasets are provided. Without resorting to external models or designed data augmentation, we get consistent and significant reductions in classification error compared with other approaches, demonstrating that cluster-based soft targets effectively complement the ground-truth label.