Small Sample Size Problem Research Articles

A Revised Formation of Trace Ratio LDA for Small Sample Size Problem.

Linear discriminant analysis (LDA) is a classic tool for supervised dimensionality reduction. Because the projected samples can be classified effectively, LDA has been successfully applied in many applications. Among the variants of LDA, trace ratio LDA (TR-LDA) is a classic form due to its explicit meaning. Unfortunately, when the sample size is much smaller than the data dimension, the algorithm for solving TR-LDA does not converge. The so-called small sample size (SSS) problem severely limits the application of TR-LDA. To solve this problem, we propose a revised formation of TR-LDA, which can be applied to datasets with different sizes in a unified form. Then, we present an optimization algorithm to solve the proposed method, explain why it can avoid the SSS problem, and analyze the convergence and computational complexity of the optimization algorithm. Next, based on the introduced theorems, we quantitatively elaborate on when the SSS problem will occur in TR-LDA. Finally, the experimental results on real-world datasets demonstrate the effectiveness of the proposed method.

Two-Stage Hyperspectral Image Classification Using Few Labeled Samples

Hyperspectral image classification (HIC) has attracted considerable attention in the last two decades, and significant progress has been made. However, the small sample size problem of HIC is still challenging. This letter presents a two-stage HIC approach that achieves high classification accuracies with few labeled samples. For a given hyperspectral image, the spatial features are first extracted by local binary patterns (LBP). Spatial features and spectral features for each pixel are then stacked into feature vectors. These vectors are fed into SVM to finish the first classification stage. Based on the preliminary classification results, a superpixel segmentation method is introduced for selecting some superpixels which include training samples and all test pixels assigned to some class. These selected superpixels with their labels obtained by SVM are then added to training samples. According to the enlarged training sample set, Random Multi-Graph (RMG) is finally utilized to classify the remaining samples. Experimental results on three benchmark HSI datasets demonstrate that the proposed LBP and RMG-based two-stage method (LBP-RMG2) significantly outperforms several state-of-the-art algorithms with a few labeled samples.

Using meta-learning to establish a highly transferable driving speed prediction model from the visual road environment

Driving speed prediction is of great significance for the realization of autonomous driving and intelligent transportation systems. The effectiveness of the existing driving speed prediction models often relies on a large number of training data, and their performance will drop sharply with the change in the usage environment. To solve these problems, this study adopted a Meta-learning algorithm to propose a highly transferable driving speed prediction model based on the visual road environment. The data used in this study were the visual road environment pictures and corresponding driving speed from three different types of scenarios (i.e., rural roads, urban roads, and highways) in a naturalistic driving experiment and a public KITTI dataset. The analysis of a Random forest model showed that the influence degree of visual road environment elements on driving speed was quite distinct in different scenarios. Then, a Meta-learning-based driving speed prediction model was presented to address two issues, including the small sample size and transferability. To examine the performance of this new model, a driving speed prediction model based on a convolutional neural network (CNN) was also used for comparison. The results showed that the prediction accuracy of Meta-learning was significantly improved compared to CNN in the face of the small sample size problem and when transferred to new scenarios. The findings in this study can contribute to the optimization of the driving speed prediction model and the improvement of the visual road environment design.

A transferred hybrid surrogate model integrating Gaussian membership virtual sample generation for small sample prediction: Applications in metal tube bending

The high-performance virtual sample generation (VSG) method has been extensively introduced to solve the problem of small sample sizes. Data distribution information is a key element of current VSG methods at the data-driven level. Herein, we propose an improved VSG method with a Gaussian distribution and explore the relationship between the Gaussian function and data expansion. To obtain more feasible virtual samples, information expanded based on the Gaussian membership function (GMIE) was established. For further improvement, a hybrid surrogate model based on transfer (THSM) is proposed, which differs from the general hybrid surrogate model (HSM) methods that only mix single models. Using a prevailing evaluation method, our proposed method, which combines a Gaussian membership function with a hybrid surrogate model, outperforms other competing approaches in 12 numerical cases owing to its feasibility and efficacy. Additionally, the proposed approach is applied to a metal tube rotary draw bending (RDB) prediction problem to illustrate its ability to support complex engineering designs.

EEG-Based Emotion Recognition Using Trainable Adjacency Relation Driven Graph Convolutional Network

In recent years, there has been a growing research interest in using deep learning to resolve the issue of Electroencephalogram (EEG)-based emotion recognition. Current research emphasizes exploiting the useful information from each single EEG channel or each individual set of multi-channel EEG, but overlooks the correlation information among different multi-channel EEG sets. To explore such discriminative correlation information, we propose a novel and effective method“, Trainable Adjacency Relation Driven Graph Convolutional Network (TARDGCN)", which contains two complementary modules, TAR and GCN. TAR optimizes the local pair-wise positions of multi-channel EEG sets, which helps form an improved graphic representation for GCN to learn the global correlation among these sets for classification. The proposed method is capable of dealing with the problem of small sample size but large data variation in this issue. Our experimental results conducted on the databases DREAMER and DEAP in the subject-dependent and subject-independent modes show that TARDGCN outperforms the state-of-the-art approaches in classifying all of valence, arousal, and dominance.

Exponential Local Fisher Discriminant Analysis with Sparse Variables Selection: A Novel Fault Diagnosis Scheme for Industry Application

Local Fisher discriminant analysis (LFDA) has been widely applied to dimensionality reduction and fault classification fields. However, it often suffers from small sample size (SSS) problem and incorporates all process variables without emphasizing the key faulty ones, thus leading to degraded fault diagnosis performance and poor model interpretability. To this end, this paper develops the sparse variables selection based exponential local Fisher discriminant analysis (SELFDA) model, which can overcome the two limitations of basic LFDA concurrently. First, the responsible faulty variables are identified automatically through the least absolute shrinkage and selection operator, and the current optimization problem are subsequently recast as an iterative convex optimization problem and solved by the minimization-maximization method. After that, the matrix exponential strategy is implemented on LFDA, it can essentially overcome the SSS problem by ensuring that the within-class scatter matrix is always full-rank, thus more practical in real industrial practices, and the margin between different categories is enlarged due to the distance diffusion mapping, which is benefit for the enhancement of classification accuracy. Finally, the Tennessee Eastman process and a real-world diesel working process are employed to validate the proposed SELFDA method, experimental results prove that the SELFDA framework is more excellent than the other approaches.

SFAMNet: A scene flow attention-based micro-expression network

Tremendous progress has been made in facial Micro-Expression (ME) spotting and recognition; however, most works have focused on either spotting or recognition tasks on the 2D videos. Until recently, the estimation of the 3D motion field (a.k.a scene flow) for the ME has only become possible after the release of the multi-modal ME dataset. In this paper, we propose the first Scene Flow Attention-based Micro-expression Network, namely SFAMNet. It takes the scene flow computed using the RGB-D flow algorithm as the input and predicts the spotting confidence score and emotion labels. Specifically, SFAMNet is an attention-based end-to-end multi-stream multi-task network devised to spot and recognize the ME. Besides that, we present a data augmentation strategy to alleviate the small sample size problem during network learning. Extensive experiments are performed on three tasks: (i) ME spotting; (ii) ME recognition; and (iii) ME analysis on the multi-modal CAS(ME)3 dataset. Empirical results indicate that depth is vital in capturing the ME information and the effectiveness of the proposed approach. Our source code is publicly available at https://github.com/genbing99/SFAMNet.

An anti-noise algorithm based on locally linear embedding and weighted XGBoost for fault diagnosis of T/R module

Due to the noise accompanied with fault signals, it is challenging to identify the discriminant information and the local geometric feature from the complex fault data for enhancing fault diagnosis accuracy. To address this challenge, this work proposed an anti-noise algorithm based on locally linear embedding integrated with diffusion distance and maximum correntropy criterion (DMLLE). In DMLLE, diffusion distance was adopted instead of the Euclidean distance for neighborhood construction. Meanwhile, the optimal weights were updated to reveal local geometry information based on the loss function of the maximum correntropy criterion. Subsequently, DMLLE is eventually developed to further restrain noise embedding into raw signals and obtain low dimensional features. Furthermore, weighted extreme gradient boosting is used to map the low dimensional features to the types of faults, which easily implements fault pattern recognition. Finally, two synthetic manifold datasets and fault data acquired from the transmit/receive (T/R) module are used to validate the performance of the proposed diagnosis methodology. Compared with the existing methods, the proposed diagnosis methodology generates a smoother flow structure by preserving the local neighborhood of the dataset with noises and realizes a higher accuracy of 94.41% on the T/R module dataset, which outperforms 3%–9% better than other classification models. Therefore, it can be concluded that the proposed diagnosis methodology can effectively extract intrinsic fault features by weakening the influence aroused by noise and achieve superior accuracy in fault diagnosis by addressing the problem of small sample size.

Tomato Production Prediction Based on Deep Learning Algorithm-Cascade-PSPNET and Bayes

Accounting the problem of small sample size in tomato yield statistics, a dual-fusion prediction analysis model based on Bayes theory and deep learning algorithm Cascade-PSPNET is proposed. The tomato yield prediction is conducted based on remote sensing image time series analysis and multi-source information fusion theory with Bayesian theory and credibility weighting. First, the area of tomato planting is analyzed through semantic segmentation algorithms of remote sensing images during the tomato planting process. The yield is predicted by variables such as planting area fluctuation, disaster cycle fluctuation, etc. Through analyzing the reduced yield affected by disasters in different time periods of remote sensing images during planting process, the value chain of tomato industry is calculated by comprehensively analyzing price-value system, inflation coefficient, and unit area yield. At the same time, the annual patent application volume is used to predict the change of tomato yield year by year according to Bayesian theory, and the relationship between annual patent application volume and tomato yield year by year under different confidence levels is analyzed. The results show that it is feasible to use the Bayes method with semantic segmentation algorithms of remote sensing images to predict tomato yield. Next, the experiment of fusion prediction between two prediction models in the same target area will be carried out for verification.

Deep learning‐based methods for detecting defects in cast iron parts and surfaces

AbstractThe large size tolerance and positional differences of burrs in cast iron blanks make it easy for traditional teaching polishing paths to cause overcutting or undercutting. Rapid and accurate identification of burrs and real‐time correction of polishing trajectories are key technical issues for achieving high‐precision polishing. Here, a deep learning‐based method for defect detection in cast iron parts and surfaces is proposed. Firstly, a self‐made dataset of cast iron parts and surface defects is created and annotated, and a variety of data augmentation methods are used to expand the number of samples in the original dataset, alleviating the problem of small sample size. Then, the coordinate attention mechanism is introduced into the backbone network to allocate more attention to the defect target. Finally, the bidirectional weighted feature pyramid network (BiFPN) is used in the feature fusion network to replace the original path aggregation network, improving the model's ability to fuse features of different sizes. Experimental results show that compared with the original model, the mean average precision (mAP) is increased by 3.1%, and the average precision (AP) in defect classification is increased by 7.6%, with an FPS of 112, achieving accurate and efficient real‐time detection of cast iron parts and surface defects.

Transferable driver facial expression recognition based on joint discriminative correlation alignment network with enhanced feature attention

AbstractFacial expression is closely related to the emotions of drivers, thus facilitating safe driving detection in advanced driving assistance system (ADAS). Recently, deep learning techniques have become prevalent for facial expression recognition. However, for driving scenarios, the facial expression recognition is mainly challenged by the problems of small sample size as well as effective feature representation. To address the above issues, this paper puts forward a transfer learning‐based method for driver facial expression recognition by fully exploiting the other sources of facial expression data that may obey different distributions. Specifically, an enhanced feature attention module is firstly devised such that affluent features with multi‐scales can be extracted and refined based on the spatial and channel attention mechanisms. Then, a joint correlation alignment loss is presented ensuring that the samples in the source and target domains are transformed into the shared common subspace to reduce difference of both the marginal and conditional distributions. Multiple transfer learning tasks on real‐world data are carried out to evaluate the proposed method. The experimental results show that our model achieves better recognition accuracy for driver facial expression compared with several traditional and deep learning‐based transfer learning algorithms.

Hybrid-supervised bidirectional transfer networks for computer-aided diagnosis

Medical imaging techniques have been widely used for diagnosis of various diseases. However, the imaging-based diagnosis generally depends on the clinical skill of radiologists. Computer-aided diagnosis (CAD) can help radiologists improve diagnostic accuracy as well as the consistency and reproducibility. Although convolutional neural network (CNN) has shown its feasibility and effectiveness in CAD, it generally suffers from the problem of small sample size when training CAD models. Nowadays, self-supervised learning (SSL) has shown its effectiveness in the field of medical image analysis, especially when there are only limited training samples. However, the backbone of downstream task sometimes cannot be well pre-trained in the conventional SSL framework due to the limitation of the pretext task and fine-tuning mechanism. In this work, an improved SSL framework, named Hybrid-supervised Bidirectional Transfer Networks (HBTN), is proposed to improve the performance of CAD models. Specifically, a novel Gray-Scale Image Mapping (GSIM) task is developed, which still takes the widely used image restoration task in SSL as the pretext task, but further embeds the class label information into it to improve discriminative feature learning of its corresponding network model. The proposed HBTN then integrates two different network architectures, i.e. the image restoration network for the pretext task and the classification network for the downstream task, into a unified hybrid-supervised learning (HSL) framework. It jointly trains both networks and collaboratively transfers the knowledge between each other. Consequently, the performance of downstream network is thus improved. The proposed HBTN is evaluated on two medical image datasets for CAD tasks. The experimental results indicate that HBTN outperforms the conventional SSL algorithms for CAD with limited training samples.

A ppb-level online detection system for gas concentrations in CS2/SO2 mixtures based on UV-DOAS combined with VMD-CNN-TL model

Carbon disulfide (CS2) and sulfur dioxide (SO2) are the main decomposition products of sulfur hexafluoride (SF6). Detecting the concentrations of these products plays a key role in early fault warning in SF6 electrical equipment. In this study, we report an online ppb-level detection system for a mixture of CS2 and SO2 based on ultraviolet differential optical absorption spectroscopy (UV-DOAS) combined with VMD-CNN-TL model. First, important features are extracted and recombined from the differential absorption spectra by variational modal decomposition (VMD) to obtain the combined differential absorption spectra (CDAS) data and mixed-gas differential absorption spectra (MDAS) data. A multilayer 1D convolutional neural network (CNN) model is then pre-trained on CDAS. Next, the parameters of the model are fine-tuned using transfer learning (TL) based on MDAS. TL solves the small sample size problem in detecting mixed-gas concentrations using neural network models. Lab-based results indicate that the system enables stable detection of CS2 (3.81–179.24 ppb) and SO2 (19.21–942.73 ppb) with mean absolute errors of 0.51 ppb and 1.12 ppb, respectively, which are the best results reported so far. Furthermore, the feasibility of the sensor system is verified for detection of gas mixtures at ppb-level by field testing.

Techniques to produce and evaluate realistic multivariate synthetic data

Data modeling requires a sufficient sample size for reproducibility. A small sample size can inhibit model evaluation. A synthetic data generation technique addressing this small sample size problem is evaluated: from the space of arbitrarily distributed samples, a subgroup (class) has a latent multivariate normal characteristic; synthetic data can be generated from this class with univariate kernel density estimation (KDE); and synthetic samples are statistically like their respective samples. Three samples (n = 667) were investigated with 10 input variables (X). KDE was used to augment the sample size in X. Maps produced univariate normal variables in Y. Principal component analysis in Y produced uncorrelated variables in T, where the probability density functions were approximated as normal and characterized; synthetic data was generated with normally distributed univariate random variables in T. Reversing each step produced synthetic data in Y and X. All samples were approximately multivariate normal in Y, permitting the generation of synthetic data. Probability density function and covariance comparisons showed similarity between samples and synthetic samples. A class of samples has a latent normal characteristic. For such samples, this approach offers a solution to the small sample size problem. Further studies are required to understand this latent class.

Modified locally joint sparse marginal embedding and wasserstein generation adversarial network for bearing fault diagnosis

Rolling bearings are essential parts for manufacturing machines. Vast quantities of features are often extracted from measured signals to comprehensively reflect the conditions of bearings, which may cause high dimensionality, information redundancy, and time consumption. In addition, it is extremely difficult, expensive, and time-consuming to collect samples with label information during the bearing fault diagnosis in real-world scenarios. In this study, a novel bearing defect diagnosis method for small sample size is proposed based on modified local joint sparse marginal embedding (MLJSME) and Wasserstein generative adversarial networks (WGANs). MLJSME can effectively extract intrinsic sparse discriminant features of high-dimensional dataset by preserving both global and local structures. Graph embedding and Gaussian kernel function are adopted to preserve the locality structure of dataset. The global structure and discriminate information are preserved by maximum margin criterion which can also avoid small sample-size problem. Moreover, joint sparsity is applied to preserve the sparse property and improve the robustness to noise and outliers. An abundance of artificial samples can be obtained with WGAN and a few labeled samples. Firstly, a high-dimensional feature dataset consisting of time-domain and frequency-domain features is extracted from original vibration signals, then MLJSME is utilized to extract sensitive low-dimensional features, and a small number of low-dimensional features are fed into WGAN to generate a large number of artificial samples that used to train the classifier, and the bearing fault types can be finally identified. The effectiveness and feasibility of the proposed method is validated by analyzing the different experimental cases.

The Guiding Power of Media on Public Opinion in the Internet Age -- Focus on the Construction of National Image

The change in media communication methods has led to a transformation in the audience and the subject and object of information. The diversity of information sources has made everyone become a producer and influencer, which has resulted in a multitude of voices and the multilateralism of public opinion on the Internet, posing a greater challenge to the media guidance power. The focus of this paper is on the impact of media guidance on public opinion, especially in how countries use public opinion to construct their national image. It begins with a literature review of previous researches on media changes, the relationship between media guidance power and public opinion, and the self-definition and other-definition of national images. Then, news articles from China and the United States are selected to analyze the performance of media in shaping their national images through content analysis and comparative analysis, and summarize the generality and feasibility of media guidance of public opinion. There is a problem of small sample size as quantitative analysis is unfeasible due to the timeliness and large number of news items. It is hoped that better solutions can be explored in future research.

Two-Dimensional Exponential Sparse Discriminant Local Preserving Projections

The two-dimensional discriminant locally preserved projections (2DDLPP) algorithm adds a between-class weighted matrix and a within-class weighted matrix into the objective function of the two-dimensional locally preserved projections (2DLPP) algorithm, which overcomes the disadvantage of 2DLPP, i.e., that it cannot use the discrimination information. However, the small sample size (SSS) problem still exists, and 2DDLPP processes the whole original image, which may contain a large amount of redundant information in the retained features. Therefore, we propose a new algorithm, two-dimensional exponential sparse discriminant local preserving projections (2DESDLPP), to address these problems. This integrates 2DDLPP, matrix exponential function and elastic net regression. Firstly, 2DESDLPP introduces the matrix exponential into the objective function of 2DDLPP, making it positive definite. This is an effective method to solve the SSS problem. Moreover, it uses distance diffusion mapping to convert the original image into a new subspace to further expand the margin between labels. Thus more feature information will be retained for classification. In addition, the elastic net regression method is used to find the optimal sparse projection matrix to reduce redundant information. Finally, through high performance experiments with the ORL, Yale and AR databases, it is proven that the 2DESDLPP algorithm is superior to the other seven mainstream feature extraction algorithms. In particular, its accuracy rate is 3.15%, 2.97% and 4.82% higher than that of 2DDLPP in the three databases, respectively.

Robust Exponential Graph Regularization Non-Negative Matrix Factorization Technology for Feature Extraction

Graph regularized non-negative matrix factorization (GNMF) is widely used in feature extraction. In the process of dimensionality reduction, GNMF can retain the internal manifold structure of data by adding a regularizer to non-negative matrix factorization (NMF). Because Ga NMF regularizer is implemented by local preserving projections (LPP), there are small sample size problems (SSS). In view of the above problems, a new algorithm named robust exponential graph regularized non-negative matrix factorization (REGNMF) is proposed in this paper. By adding a matrix exponent to the regularizer of GNMF, the possible existing singular matrix will change into a non-singular matrix. This model successfully solves the problems in the above algorithm. For the optimization problem of the REGNMF algorithm, we use a multiplicative non-negative updating rule to iteratively solve the REGNMF method. Finally, this method is applied to AR, COIL database, Yale noise set, and AR occlusion dataset for performance test, and the experimental results are compared with some existing methods. The results indicate that the proposed method is more significant.

Digital Addiction Intervention for Children and Adolescents: A Scoping Review.

Digital devices play a significant role in the learning and living of children and adolescents, whose overuse or addiction has become a global concern. This scoping review seeks to synthesize existing studies to investigate relevant interventions and their effects on digital addiction in children (ages 0-18). To understand the latest advances, we have identified 17 studies published in international peer-reviewed journals between 2018-2022. The findings revealed that, first, most interventions for digital addiction in children and adolescents were cognitive-behavioral therapies (CBT) or CBT-based interventions, which could improve anxiety, depression, and related symptoms of digital addiction. Second, rather than directly targeting addictive behaviors, some family-based interventions aim to strengthen family functions and relationships. Finally, digital-based interventions, such as website-based, application-based, and virtual reality interventions, are promising in adolescent digital addiction interventions. However, these studies shared the same limitations: small sample sizes, short intervention durations, no control group, and nonrandomized assignments. The small sample size problem is difficult to solve by offline intervention. Meanwhile, online digital-based intervention is still in its infancy, resulting in limited generalizability of the findings and the inability to popularize digital intervention. Accordingly, future intervention studies should integrate various assessments and interventions to form an integrated platform to provide interventions for addicted children and adolescents worldwide.

Application of Virtual Sample Generation and Screening in Process Parameter Optimization of Botanical Medicinal Materials.

The small sample problem widely exists in the fields of the chemical industry, chemistry, biology, medicine, and food industry. It has been a problem in process modeling and system optimization. The aim of this study is to focus on the problems of small sample size in modeling, the process parameters in the ultrasonic extraction of botanical medicinal materials can be obtained by optimizing the extraction rate model. However, difficulty in data acquisition results in problem of small sample size in modeling, which eventually reduces the accuracy of modeling prediction. A virtual sample generation method based on full factorial design (FFD) is proposed to solve the problem ofa small sample size. The experiments are first conducted according to the Box- Behnken Design (BBD) to obtain small-size samples, and the response surface function is established accordingly. Then, virtual sample inputs are obtained by the FFD, and the corresponding virtual sample outputs are calculated by the response surface function. Furthermore, a screening method of virtual samples is proposed based on an extreme learning machine (ELM). The connection weights of ELM are used for further optimization and screening of the generated virtual samples. The results show that virtual sample data can effectively expand the sample size. The precision of the model trained on semi-synthetic samples (small-size experimental simples and virtual samples) is higher than the model trained merely on small-size experimental samples. The virtual sample generation and screening methods proposed in this paper can effectively solve the modeling problem of small samples. The reliable process parameters can be obtained by optimizing the model trained by the semi-synthetic samples.