Samples Of Different Classes Research Articles

Meta-Hybrid: Integrate Meta-Learning to Enhance Class Imbalance Graph Learning

The class imbalance problem is a significant challenge in node classification tasks. Since majority class samples dominate imbalanced data, the model tends to favor the majority class, resulting in insufficient ability to identify minority classes. Evaluation indicators such as accuracy may not fully reflect the model’s performance. To solve these undesirable effects, we propose a framework for synthesizing minority class samples, GraphSHX, to balance the number of samples of different classes, and integrate the XGBoost model for node classification prediction during the training process. Conventional graph neural networks (GNNs) yielded unsatisfactory results, possibly due to the limited number of newly generated nodes. Therefore, we introduce a meta-mechanism to deal with small-sample problems, and employ the meta-learning approach to enhance performance on small-sample tasks by learning from a large number of tasks. An empirical evaluation of node classification on six publicly available datasets demonstrated that our balanced data set method outperforms existing optimal loss repair methods and synthetic node methods. The addition of the XGBoost model and meta-learning improves the accuracy by more than 5% to 10%, with the overall accuracy of the improved model being 15% higher than that of the baseline method.

A dual algorithmic approach to deal with multiclass imbalanced classification problems

Many real-world applications involve multiclass classification problems, and often the data across classes is not evenly distributed. Due to this disproportion, supervised learning models tend to classify instances towards the class with the maximum number of instances, which is a severe issue that needs to be addressed. In multiclass imbalanced data classification, machine learning researchers try to reduce the learning model's bias towards the class with a high sample count. Researchers attempt to reduce this unfairness by either balancing the data before the classifier learns it, modifying the classifier's learning phase to pay more attention to the class with a minimum number of instances, or a combination of both. The existing algorithmic approaches find it difficult to understand the clear boundary between the samples of different classes due to unfair class distribution and overlapping issues. As a result, the minority class recognition rate is poor. A new algorithmic approach is proposed that uses dual decision trees. One is used to create an induced dataset using a PCA based grouping approach and by assigning weights to the data samples followed by another decision tree to learn and predict from the induced dataset. The distinct feature of this algorithmic approach is that it recognizes the data instances without altering their underlying data distribution and is applicable for all categories of multiclass imbalanced datasets. Five multiclass imbalanced datasets from UCI were used to classify the data using our proposed algorithm, and the results revealed that the duo-decision tree approach pays better attention to both the minor and major class samples.

MSO‐DETR: Metric space optimization for few‐shot object detection

AbstractIn the metric‐based meta‐learning detection model, the distribution of training samples in the metric space has great influence on the detection performance, and this influence is usually ignored by traditional meta‐detectors. In addition, the design of metric space might be interfered with by the background noise of training samples. To tackle these issues, we propose a metric space optimisation method based on hyperbolic geometry attention and class‐agnostic activation maps. First, the geometric properties of hyperbolic spaces to establish a structured metric space are used. A variety of feature samples of different classes are embedded into the hyperbolic space with extremely low distortion. This metric space is more suitable for representing tree‐like structures between categories for image scene analysis. Meanwhile, a novel similarity measure function based on Poincaré distance is proposed to evaluate the distance of various types of objects in the feature space. In addition, the class‐agnostic activation maps (CCAMs) are employed to re‐calibrate the weight of foreground feature information and suppress background information. Finally, the decoder processes the high‐level feature information as the decoding of the query object and detects objects by predicting their locations and corresponding task encodings. Experimental evaluation is conducted on Pascal VOC and MS COCO datasets. The experiment results show that the effectiveness of the authors’ method surpasses the performance baseline of the excellent few‐shot detection models.

Novel Triplet Loss-Based Domain Generalization Network for Bearing Fault Diagnosis with Unseen Load Condition

In the real industrial manufacturing process, due to the constantly changing operational loads of equipment, it is difficult to collect data from all load conditions as the source domain signal for fault diagnosis. Therefore, the appearance of unseen load vibration signals in the target domain presents a challenge and research hotspot in fault diagnosis. This paper proposes a triplet loss-based domain generalization network (TL-DGN) and then applies it to an unseen domain bearing fault diagnosis. TL-DGN first utilizes a feature extractor to construct a multi-source domain classification loss. Furthermore, it measures the distance between class data from different domains using triplet loss. The introduced triplet loss can narrow the distance between samples of the same class in the feature space and widen the distance between samples of different classes based on the action of the cross-entropy loss function. It can reduce the dependency of the classification boundary on bearing operational loads, resulting in a more generalized classification model. Finally, two comparative experiments with fault diagnosis models without triplet loss and other classification models demonstrate that the proposed model achieves superior fault diagnosis performance.

Electroencephalography (EEG) based epilepsy diagnosis via multiple feature space fusion using shared hidden space-driven multi-view learning.

Epilepsy is a chronic, non-communicable disease caused by paroxysmal abnormal synchronized electrical activity of brain neurons, and is one of the most common neurological diseases worldwide. Electroencephalography (EEG) is currently a crucial tool for epilepsy diagnosis. With the development of artificial intelligence, multi-view learning-based EEG analysis has become an important method for automatic epilepsy recognition because EEG contains difficult types of features such as time-frequency features, frequency-domain features and time-domain features. However, current multi-view learning still faces some challenges, such as the difference between samples of the same class from different views is greater than the difference between samples of different classes from the same view. In view of this, in this study, we propose a shared hidden space-driven multi-view learning algorithm. The algorithm uses kernel density estimation to construct a shared hidden space and combines the shared hidden space with the original space to obtain an expanded space for multi-view learning. By constructing the expanded space and utilizing the information of both the shared hidden space and the original space for learning, the relevant information of samples within and across views can thereby be fully utilized. Experimental results on a dataset of epilepsy provided by the University of Bonn show that the proposed algorithm has promising performance, with an average classification accuracy value of 0.9787, which achieves at least 4% improvement compared to single-view methods.

An efficient convolutional neural network-based classifier for an imbalanced oral squamous carcinoma cell dataset

<span lang="EN-US">Imbalanced datasets pose a major challenge for the researchers while addressing machine learning tasks. In these types of datasets, samples of different classes are not in equal proportion rather the gap between the numbers of individual class samples is significantly large. Classification models perform better for datasets having equal proportion of data tuples in both the classes. But, in reality, the medical image datasets are skewed and hence are not always suitable for a model to achieve improved classification performance. Therefore, various techniques have been suggested in the literature to overcome this challenge. This paper applies oversampling technique on an imbalanced dataset and focuses on a customized convolutional neural network model that classifies the images into two categories: diseased and non-diseased. Outcome of the proposed model can assist the health experts in the detection of oral cancer. The proposed model exhibits 99% accuracy after data augmentation. Performance metrics such as precision, recall and F1-score values are very close to 1. In addition, statistical test is performed to validate the statistical significance of the model. It has been found that the proposed model is an optimised classifier in terms of number of network layers and number of neurons.</span>

Deep metric learning assisted by intra-variance in a semi-supervised view of learning

Deep metric learning aims to construct an embedding space where samples belonging to the same class are closely grouped together, while samples from different classes are widely separated. Many existing deep metric learning methods focus on maximizing the difference between inter-class features, and semantic related information is obtained by increasing the distance between samples of different classes in the embedding space. However, by compressing all positive samples together and creating large margins between different classes, the local structure between similar samples is inadvertently disrupted. This disregard for the intra-class variance presents in the local structure results in an embedding space that exhibits lower generalizability when faced with unseen classes. Consequently, the network tends to overfit the training set and performs poorly on the test set, potentially leading to crashes during evaluation. Taking this into account, this paper introduces a self-supervised generative assisted ranking framework that offers a semi-supervised perspective on learning intra-class variance within traditional supervised deep metric learning. Specifically, this paper employs various intensities and diversities to synthesize samples based on positive examples, aiming to simulate the complex variations between similar samples. Then, an intra-class ranking loss function is devised based on the principles of self-supervised learning. This loss function constrains the ordering relationship of synthesized samples according to their generation intensity, enabling the network to capture subtle intra-class differences and maintain the intra-class distribution. With the adoption of this approach, a more realistic embedding space can be achieved, preserving both the global and local structures of samples.

Highly Discriminative Driver Distraction Detection Method Based on Swin Transformer

Driver distraction detection not only helps to improve road safety and prevent traffic accidents, but also promotes the development of intelligent transportation systems, which is of great significance for creating a safer and more efficient transportation environment. Since deep learning algorithms have very strong feature learning abilities, more and more deep learning-based driver distraction detection methods have emerged in recent years. However, the majority of existing deep learning-based methods are optimized only through the constraint of classification loss, making it difficult to obtain features with high discrimination, so the performance of these methods is very limited. In this paper, to improve the discrimination between features of different classes of samples, we propose a high-discrimination feature learning strategy and design a driver distraction detection model based on Swin Transformer and the highly discriminative feature learning strategy (ST-HDFL). Firstly, the features of input samples are extracted through the powerful feature learning ability of Swin Transformer. Then, the intra-class distance of samples of the same class in the feature space is reduced through the constraint of sample center distance loss (SC loss), and the inter-class distance of samples of different classes is increased through the center vector shift strategy, which can greatly improve the discrimination of different class samples in the feature space. Finally, we have conducted extensive experiments on two publicly available datasets, AUC-DD and State-Farm, to demonstrate the effectiveness of the proposed method. The experimental results show that our method can achieve better performance than many state-of-the-art methods, such as Drive-Net, MobileVGG, Vanilla CNN, and so on.

Locality Cross-domain Discriminant Analysis for Membranous Nephropathy Recognition using Microscopic Hyperspectral Imaging.

Cross-domain methods have been proposed to learn the domain invariant knowledge that can be transferred from the source domain to the target domain. Existing cross-domain methods attempt to minimize the distribution discrepancy of the domains. However, these methods fail to explore the domain invariant subspace due to the samples of different classes between two domains may overlap in the new subspace. They consider the features in the original space data that may be unnecessary or irrelevant to the final classification, and neglect to preserve the local manifold structure between two domains. To solve these problems, a novel feature extraction method called Locality Cross-domain Discriminant Analysis (LCDA) is proposed. LCDA first aligns the distributions and avoids overlap between two domains. Then, LCDA exploits the local manifold structure to maintain the discriminative capability of the low-dimensional projection matrices. Finally, a robust constraint is utilized to preserve the robustness of the projection matrices. The proposed LCDA not only avoids overlap between different classes but also explores the local manifold information. Experiment results on the medical membranous nephropathy hyperspectral dataset demonstrate that the proposed LCDA has better performance than other relevant feature extraction methods.

Heterogeneous Domain Adaptation With Generalized Similarity and Dissimilarity Regularization.

Heterogeneous domain adaptation (HDA) aims to address the transfer learning problems where the source domain and target domain are represented by heterogeneous features. The existing HDA methods based on matrix factorization have been proven to learn transferable features effectively. However, these methods only preserve the original neighbor structure of samples in each domain and do not use the label information to explore the similarity and separability between samples. This would not eliminate the cross-domain bias of samples and may mix cross-domain samples of different classes in the common subspace, misleading the discriminative feature learning of target samples. To tackle the aforementioned problems, we propose a novel matrix factorization-based HDA method called HDA with generalized similarity and dissimilarity regularization (HGSDR). Specifically, we propose a similarity regularizer by establishing the cross-domain Laplacian graph with label information to explore the similarity between cross-domain samples from the identical class. And we propose a dissimilarity regularizer based on the inner product strategy to expand the separability of cross-domain labeled samples from different classes. For unlabeled target samples, we keep their neighbor relationship to preserve the similarity and separability between them in the original space. Hence, the generalized similarity and dissimilarity regularization is built by integrating the above regularizers to facilitate cross-domain samples to form discriminative class distributions. HGSDR can more efficiently match the distributions of the two domains both from the global and sample viewpoints, thereby learning discriminative features for target samples. Extensive experiments on the benchmark datasets demonstrate the superiority of the proposed method against several state-of-the-art methods.

CSTS: Exploring Class-Specific and Task-Shared Embedding Representation for Few-Shot Learning.

Few-shot learning (FSL) is a challenging yet promising technique that aims to discriminate objects based on a few labeled examples. Learning a high-quality feature representation is key with few-shot data, and many existing models attempt to extract general information from the sample or task levels. However, the common sample-level means of feature representation limits the models generalizability to different tasks, while task-level representation may lose class characteristics due to excessive information aggregation. In this article, we synchronize the class-specific and task-shared information from the class and task levels to obtain a better representation. Structure-based contrastive learning is introduced to obtain class-specific representations by increasing the interclass distance. A hierarchical class structure is constructed by clustering semantically similar classes using the idea of granular computing. When guided by a class structure, it is more difficult to distinguish samples in different classes that have similar characteristics than those with large interclass differences. To this end, structure-guided contrastive learning is introduced to study class-specific information. A hierarchical graph neural network is established to transfer task-shared information from coarse to fine. It hierarchically infers the target sample based on all samples in the task and yields a more general representation for FSL classification. Experiments on four benchmark datasets demonstrate the advantages of our model over several state-of-the-art models.

DyCR: A Dynamic Clustering and Recovering Network for Few-Shot Class-Incremental Learning.

Few-shot class-incremental learning (FSCIL) aims to continually learn novel data with limited samples. One of the major challenges is the catastrophic forgetting problem of old knowledge while training the model on new data. To alleviate this problem, recent state-of-the-art methods adopt a well-trained static network with fixed parameters at incremental learning stages to maintain old knowledge. These methods suffer from the poor adaptation of the old model with new knowledge. In this work, a dynamic clustering and recovering network (DyCR) is proposed to tackle the adaptation problem and effectively mitigate the forgetting phenomena on FSCIL tasks. Unlike static FSCIL methods, the proposed DyCR network is dynamic and trainable during the incremental learning stages, which makes the network capable of learning new features and better adapting to novel data. To address the forgetting problem and improve the model performance, a novel orthogonal decomposition mechanism is developed to split the feature embeddings into context and category information. The context part is preserved and utilized to recover old class features in future incremental learning stages, which can mitigate the forgetting problem with a much smaller size of data than saving the raw exemplars. The category part is used to optimize the feature embedding space by moving different classes of samples far apart and squeezing the sample distances within the same classes during the training stage. Experiments show that the DyCR network outperforms existing methods on four benchmark datasets. The code is available at: https://github.com/zichengpan/DyCR.

A Large-Class Few-Shot Learning Method Based on High-Dimensional Features

Large-class few-shot learning has a wide range of applications in many fields, such as the medical, power, security, and remote sensing fields. At present, many few-shot learning methods for fewer-class scenarios have been proposed, but little research has been performed for large-class scenarios. In this paper, we propose a large-class few-shot learning method called HF-FSL, which is based on high-dimensional features. Recent theoretical research shows that if the distribution of samples in a high-dimensional feature space meets the conditions of compactness within the class and the dispersion between classes, the large-class few-shot learning method has a better generalization ability. Inspired by this theory, the basic idea is use a deep neural network to extract high-dimensional features and unitize them to project the samples onto a hypersphere. The global orthogonal regularization strategy can then be used to make samples of different classes on the hypersphere that are as orthogonal as possible, so as to achieve the goal of sample compactness within the class and the dispersion between classes in high-dimensional feature space. Experiments on Omniglot, Fungi, and ImageNet demonstrate that the proposed method can effectively improve the recognition accuracy in a large-class FSL problem.

Intrusion detection for Industrial Internet of Things based on deep learning

Intrusion detection technology can actively detect abnormal behaviors in the network and is important to the security of Industrial Internet of Things (IIOT). However, there are some issues with the current intrusion detection technology for IIOT, such as extreme imbalance in the number of samples of different classes in the dataset, redundant and meaningless features in the samples, and the inability of traditional intrusion detection methods to meet the requirements of detection accuracy in the increasingly complex IIOT. In view of the extreme imbalance of classes, this paper applies the hierarchical clustering algorithm to the under-sampling technology, which reduces the number of majority samples while reducing the loss of information of majority samples, and solves the problem of missing detection and false detection of minority samples caused by sample imbalance. In order to avoid feature redundancy and interference, this paper proposes an optimal feature selection algorithm based on greedy thought. This algorithm can obtain the optimal feature subset of each type of data in the data set, thus eliminating redundant and interfering features. Aiming at the problem of insufficient detection ability of traditional detection methods, this paper proposes a deep neural network intrusion detection model based on the parallel connection of global and local subnetworks. This model obtains the overall benchmark detection of the dataset through the deep neural network, and then strengthens the detection effect of each subclass through the parallel connection of subnetworks, greatly improving the performance of the intrusion detection algorithm. The experimental results show that the method described in this paper can improve the intrusion detection for IIOT.

Oversampling method based on GAN for tabular binary classification problems

Data-imbalanced problems are present in many applications. A big gap in the number of samples in different classes induces classifiers to skew to the majority class and thus diminish the performance of learning and quality of obtained results. Most data level imbalanced learning approaches generate new samples only using the information associated with the minority samples through linearly generating or data distribution fitting. Different from these algorithms, we propose a novel oversampling method based on generative adversarial networks (GANs), named OS-GAN. In this method, GAN is assigned to learn the distribution characteristics of the minority class from some selected majority samples but not random noise. As a result, samples released by the trained generator carry information of both majority and minority classes. Furthermore, the central regularization makes the distribution of all synthetic samples not restricted to the domain of the minority class, which can improve the generalization of learning models or algorithms. Experimental results reported on 14 datasets and one high-dimensional dataset show that OS-GAN outperforms 14 commonly used resampling techniques in terms of G-mean, accuracy and F1-score.

Compound-Specific 1D 1H NMR Pulse Sequence Selection for Metabolomics Analyses.

NMR-based metabolomics approaches have been used in a wide range of applications, for example, with medical, plant, and marine samples. One-dimensional (1D) 1H NMR is routinely used to find out biomarkers in biofluids such as urine, blood plasma, and serum. To mimic biological conditions, most NMR studies have been carried out in an aqueous solution where the high intensity of the water peak is a major problem in obtaining a meaningful spectrum. Different methods have been used to suppress the water signal, including 1D Carr-Purcell-Meiboom-Gill (CPMG) presat, consisting of a T2 filter to suppress macromolecule signals and reduce the humped curve in the spectrum. 1D nuclear Overhauser enhancement spectroscopy (NOESY) is another method for water suppression that is used routinely in plant samples with fewer macromolecules than in biofluid samples. Other common 1D 1H NMR methods such as 1D 1H presat and 1D 1H ES have simple pulse sequences; their acquisition parameters can be set easily. The proton with presat has just one pulse and the presat block causes water suppression, while other 1D 1H NMR methods including those mentioned above have more pulses. However, it is not well known in metabolomics studies because it is used only occasionally and in a few types of samples by metabolomics experts. Another effective method is excitation sculpting to suppress water. Herein, we evaluate the effect of method selection on signal intensities of commonly detected metabolites. Different classes of samples including biofluid, plant, and marine samples were investigated, and recommendations on the advantages and limitations of each method are presented.

Local sensitive discriminative broad learning system for hyperspectral image classification

Recently, broad learning system (BLS) has been widely used in hyperspectral image (HSI) classification with its excellent learning performance and generalization ability. However, BLS only focuses on the separability of various samples, ignoring the relative relationship between samples and the discriminative information. To some extent, it limits the performance of BLS. Therefore, we propose a local sensitive discriminative broad learning system (LSDBLS) for HSI classification. LSDBLS considers the discriminative information of labeled samples and the local manifold structure of data samples by introducing local sensitive discriminant analysis, and construct intra-class and inter-class graphs by labeled samples to representation the relative relationship between data samples. On this basis, the intra-class graph and the inter-class graph are introduced into the objective function of the broad learning system. By minimizing the intra-class graph and maximizing the inter-class graph, the samples of the same class are aggregated as much as possible, and the samples of different classes are as much as possible, so as to enhances the discriminative ability of LSDBLS for data features. Experimental results on three HSI datasets show that LSDBLS achieves good performance.

Growing Hierarchical Self-Organising Representation Map (GHSORM)

The Growing Hierarchical Self-Organising Representation Map (GHSORM) is a model fusing the denoising autoencoder, used to better represent a dataset, and the Growing Hierarchical Self-Organising Map, used for organizing and projecting the input data in clusters of varying detail. It is shown here that the GHSORM is instrumental in sub-grouping clusters that are not fully separable by a single SOM. This combined approach is first tested and illustrated on the problem of clustering handwritten digits where a modification of the Activation Maximisation method for use at the SOM output layer demonstrates the benefit of hierarchical growth in the GHSORM. In particular, the SOM Node Activation Maximisation method is used to visually represent the best approximation of each of the SOM nodes at the output layer. This demonstrates the improvement in representing difficult to separate digits in the hierarchical case. To test and measure the efficacy of the GHSORM hierarchical model in class and sub-class separation the method is applied to complex digital gene expression data. A cancer dataset, comprising of gene expression data that has samples of different classes and sub-classes, is used for this purpose. The GHSORM demonstrates robust capabilities to cluster and sub-cluster the different classes and subclasses of cancer, where the results are superior to both linear methods, currently in use, as well as the methods of its constituent algorithms.

Method comparison of sample pretreatment and discovery of differential compositions of natural flavors and fragrances for quality analysis by using chemometric tools

Natural flavors and fragrances or their extracts have been widely used in a large variety of areas, including food, cosmetic, and tobacco industrial processes, among others. The compositions and intrinsic attributes of flavors and fragrances were related to many factors, such as species, geographical origin, planting environment, storage condition, processing method, and so on. This not only increased the difficulty in analyzing the product quality of flavors and fragrances, but also challenged the idea of “quality-by-design (QbD)”. This work proposed an integrated strategy for precise discovery of differential compounds among different classes and subsequent quality analysis of complex samples through flavors and fragrances used in tobacco industry as examples. Three pretreatment methods were first inspected to effectively characterize the sample compositions, including direct injection (DI), thermal desorption (TD), and stir bar sorptive extraction (SBSE)-TD, coupled with gas chromatography-mass spectrometry (GC–MS) analysis to obtain characteristic information of samples of flavors and fragrances. Then, principal component analysis (PCA) was applied to discover the relation and difference between chromatographic fingerprints and peak table data once significant components were recognized in a holistic manner. Model population analysis (MPA) was then used to quantitatively extract the characteristic chemicals representing the quality differences among different classes of samples. Some differential marker compounds were discovered for difference analysis, including benzyl alcohol, latin acid, l-menthol acid, decanoic acid ethyl ester, vanillin, trans-o-coumaric acid, benzyl benzoate, and so on. Furthermore, partial least squares-discriminant analysis (PLS-DA) and support vector machine (SVM) were respectively applied to construct multivariate models for evaluation of quality differences and variations. It was found that the accuracy attains to 100% for sample classification. With the help of optimal sample pretreatment technique and chemometric methods, the strategy for quality analysis and difference discovery proposed in this work can be widely delivered to more areas of complex plants with good interpretability and high accuracy.

A Novel Generative Adversarial Network Model Based on GC-MS Analysis for the Classification of Taif Rose

Rose oil production is believed to be dependent on only a few genotypes of the famous rose Rosa damascena. The aim of this study was to develop a novel GC-MS fingerprint based on the need to expand the genetic resources of oil-bearing rose for industrial cultivation in the Taif region (Saudi Arabia). Gas chromatography-mass spectrometry (GC-MS) is a widely used analytical technique for determining the volatile composition of distilled rose oil from flower data. Because biosample availability, prohibitive costs, and ethical concerns limit observations in agricultural research, we aimed to enhance the quality of analysis by combining real observations with samples generated in silico. This study proposes a novel artificial intelligence model based on generative adversarial neural networks (GANs) to classify Taif rose cultivars using raw GC-MS data. We employed a variant of the GAN known as conditional stacked GANs (cSGANs) to predict Taif rose’s oil content and other latent characteristics without the need to conduct laboratory tests. A hierarchical stack of conditional GANs is used in this algorithm to generate images. A cluster model was developed based on the dataset provided, to quantify the diversity that should be implemented in the proposed model. The networks were trained using the cross-entropy and minimax loss functions. The accuracy of the proposed model was assessed by measuring losses as a function of the number of epochs. The results prove the ability of the proposed model to perfectly generate new real samples of different classes based on the GC-MS fingerprint.