Small Sample Size Problem Research Articles

Data/Model Jointly Driven High-Quality Case Generation for Power System Dynamic Stability Assessment

For data-driven dynamic stability assessment (DSA) in power systems, learning cases collected from actual historical records appear to be more reliable than those obtained from numerical simulations with an inevitable reality gap. However, due to the scarceness of transient events in practical systems, historical case sets generally encounter the small sample size and class-imbalance problems. To tackle these challenging issues, this article proposes a novel data/model jointly driven framework to generate high-quality cases for power system DSA applications. Model-driven numerical simulations are first utilized for rough case generation, based upon which case refinement is then intelligently carried out via cycle generative adversarial network (CycleGAN) learning. In this data-driven manner, the CycleGAN is able to produce refined cases highly resembling actual historical ones. A long short-term memory-based semisupervised learning scheme is further designed to reliably label all the refined cases. Numerical tests are comprehensively carried out on the realistic Guangdong Power Grid in South China. With only a small and skewed historical case set initially provided, the proposed framework is able to generate highly realistic cases to augment the set and mitigate the class-imbalance issue. These synthetic cases further help derive a more discerning DSA model, which contributes to enhanced reliability and adaptability of online DSA in practical power grids.

Similarity-Principle-Based Machine Learning Method for Clinical Trials and Beyond

With recent success in supervised learning, artificial intelligence (AI) and machine learning (ML) can play a vital role in precision medicine. Deep learning neural networks have been used in drug discovery when larger data is available. However, applications of machine learning in clinical trials with small sample size (around a few hundreds) are limited. We propose a Similarity-Principle-Based Machine Learning (SBML) method, which is applicable for small and large sample size problems. In SBML, the attribute-scaling factors are introduced to objectively determine the relative importance of each attribute (predictor). The gradient method is used in learning (training), that is, updating the attribute-scaling factors. We evaluate SBML when the sample size is small and investigate the effects of tuning parameters. Simulations show that SBML achieves better predictions in terms of mean squared errors for various complicated nonlinear situations than full linear models, optimal and ridge regressions, mixed effect models, support vector machine and decision tree methods.

Forecast of Logistics Demand in the Pearl River Delta Region Based on PCA-GA-SVM Model

Regional economic development is highly correlated with the change of regional logistics. This paper selects the freight volume as the representative index to reflect the development of regional logistics, and constructs the prediction index system of regional logistics demand. Accordingly, the principal component analysis method is used to reduce the data dimension of the prediction index, and the complexity of the prediction model. Further, the support vector regression model is optimized by genetic algorithm which is constructed by using the advantages of support vector machine algorithm in dealing with nonlinear and small sample size problems. The empirical analysis shows that the prediction model based on PCA-GA-SVM has very good prediction accuracy, it can provide valuable reference for regional logistics development and management.

Facial Recognition Attendance System Using Deep Learning

Abstract: Face authentication is one of the critical unsolved problems in computer science, a lot of time and energy is spent to invent a robust solution for it. Face authentication can play a significant role in security, Biometric verification, auto detection of criminals in a crowd, etc. The paper presents a unique model that authenticates the faces in the image, and securely shares the images. Firstly, the deep learning algorithm identifies all the faces in the image, extracts the invariant features of each face. The invariant feature of the face is studied to compute the emotions. The model captures the friends list of the user and securely shares the image to the known persons in the image.The main purpose of this project is to enhance the relationship between the student and the teaching staff. Due to pandemic every student are forced to attend the online classes, even though some students are tend to bunk the classes by attending those session by use some other else. In order to stop this we developed an windows application to take facial attendance and attending the class. The current old system has a lot of ambiguity that caused inaccurate and inefficient of attendance taking. Many problems arise when the authority is unable to enforce the regulation that exists in the old system. Thus, by means of technology, this project will resolve the flaws existed in the current system while bringing attendance taking to a whole new level by automating most of the tasks. The technology working behind will be the face recognition system. In the old system they use LDA. Although LDA is one of the most common data reduction techniques, it suffers from two main problems: The Small Sample Size (SSS) and linearity problems. We improvised the facial recognition using LBPH – (Local Binary Pattern Histograms) - Local binary patterns (LBP) is a type of visual descriptor used for classification in computer vision. This algorithm is so fast and reliable and the result prediction we have utilized a regression algorithm.

Oblique and rotation double random forest

Random Forest is an ensemble of decision trees based on the bagging and random subspace concepts. As suggested by Breiman, the strength of unstable learners and the diversity among them are the ensemble models’ core strength. In this paper, we propose two approaches known as oblique and rotation double random forests. In the first approach, we propose rotation based double random forest. In rotation based double random forests, transformation or rotation of the feature space is generated at each node. At each node different random feature subspace is chosen for evaluation, hence the transformation at each node is different. Different transformations result in better diversity among the base learners and hence, better generalization performance. With the double random forest as base learner, the data at each node is transformed via two different transformations namely, principal component analysis and linear discriminant analysis. In the second approach, we propose oblique double random forest. Decision trees in random forest and double random forest are univariate, and this results in the generation of axis parallel split which fails to capture the geometric structure of the data. Also, the standard random forest may not grow sufficiently large decision trees resulting in suboptimal performance. To capture the geometric properties and to grow the decision trees of sufficient depth, we propose oblique double random forest. The oblique double random forest models are multivariate decision trees. At each non-leaf node, multisurface proximal support vector machine generates the optimal plane for better generalization performance. Also, different regularization techniques (Tikhonov regularization, axis-parallel split regularization, Null space regularization) are employed for tackling the small sample size problems in the decision trees of oblique double random forest. The proposed ensembles of decision trees produce trees with bigger size compared to the standard ensembles of decision trees as bagging is used at each non-leaf node which results in improved performance. The evaluation of the baseline models and the proposed oblique and rotation double random forest models is performed on benchmark 121 UCI datasets and real-world fisheries datasets. Both statistical analysis and the experimental results demonstrate the efficacy of the proposed oblique and rotation double random forest models compared to the baseline models on the benchmark datasets.

Neighbourhood income inequalities in mental health in Barcelona 2001-2016: a Bayesian smoothed estimate

ObjectiveObtaining reliable health estimates at the small area level (such as neighbourhoods) using survey data usually poses the problem of small sample sizes. To overcome this limitation, we explored smoothing techniques in order to estimate poor mental health prevalence at the neighbourhood level and analyse its profile by income in Barcelona city (Spain). MethodA Bayesian smoothing model with a logit-normal transformation was applied to four repeated cross-sectional waves of the Barcelona health survey for 2001, 2006, 2011 and 2016. Mental health status was identified from the 12-item General Health Questionnaire. Income inequalities were analysed with neighbourhood income in quantiles for each year and trends in the pooled analysis. ResultsThe prevalence of poor mental health ranged from 14.6% in 2001 to 18.9% in 2016. The yearly difference between neighbourhoods was 12.4% in 2001, 16.7% in 2006, 14.2% in 2011, and 20.0% in 2016. The odds ratio and 95% credible interval (95%CI) of experiencing poor mental health was 1.40 times higher (95%CI: 1.02-1.91) in less advantaged neighbourhoods than in more advantaged neighbourhoods in 2001, 1.61 times higher (95%CI: 1.01-2.59) in 2006 and 2.31 times higher (95%CI: 1.57-3.40) in 2016. ConclusionsThis study shows that the Bayesian smoothed techniques allows detection of inequalities in health in neighbourhoods and monitoring of interventions against them. In Barcelona, mental health problems are more prevalent in low-income neighbourhoods and raised in 2016.

Jointly sparse least square support vector machine

Least square support vector machine (LS-SVM) is extended from support vector machine (SVM) for binary classification problems. However, it may suffer from the small sample size (SSS) problem when the sample size is much smaller than the number of features. Motivated by the dimensionality reduction and feature selection methods, we introduce L2,1-norm into LS-SVM to design a novel classification algorithm called jointly sparse LS-SVM (JS-LSSVM). JS-LSSVM minimizes the L2,1-norm regularization on the projection matrix with orthogonal constraint, which is used to project the samples into an optimal low-dimensional subspace, where the derived LS-SVM can obtain the best performance. This projection matrix releases the least square problem in primal space and allows us to select features with joint sparsity. Besides, we propose an iterative algorithm to solve the optimization problem, which guarantees the convergence of JS-LSSVM. The experiments also show the superior performance of JS-LSSVM on many datasets. The proposed method has at least 1% improvement to the conventional methods.

A Bayesian Generalized Regression Approach for estimating parameters in the Item Sum Technique of sensitive surveys

Most respondents find it difficult to respond to sensitive questions in surveys especially providing information that require one to disclose behaviors that are possibly against social norms. This study involves solving the problem of non-response or falsification of information in a sample survey as a result of sensitive questions using the Item Sum Technique (IST). The IST is a recent indirect method for collecting data for continuous sensitive characteristics and it is a quantitative variant of the Item Count Technique (ICT). The calibration method has previously been used to estimate the IST, but the method is design-based and only useful for sufficiently large domains. There are situations where the domain sample size is not adequate to meet the needs of the precision of the estimates, hence the smallarea estimation which is model-based may be needed. The Bayesian method is a model-based estimation method that tackles the problem of small area sample size. Results from simulation indicate the efficiency of the Bayesian Generalized Regression (BGREG) method for both small and large domain samples as compared with the Generalized Regression (GREG) and calibration method using three different criteria, AIC, BIC and MSE. A small sample survey on the use of substance abuse and internet gambling among university students from Nigeria was further used to show that the BGREG outperformed the GREG and Calibration methods using the AIC, BIC and MSE criterion.

Self-Distilled Supervised Contrastive Learning for diagnosis of breast cancers with histopathological images

Convolutional neural networks (CNN) and its variants have been widely used for developing the histopathological image based computer-aided diagnosis (CAD). However, the annotated data are scarce in clinical practice, and limited training samples generally cannot well train the CNN model, resulting in degraded predictive performance. To this end, we propose a novel Self-Distilled Supervised Contrastive Learning (SDSCL) algorithm to improve the diagnostic performance of a CNN-based CAD for breast cancer. In particular, the original histopathological images are first decomposed into H and E stain views, which are served as the augmented sample pairs in the supervised contrastive learning (SCL). Due to the complementary characteristic of the two stain views, this data-driven SCL guide the CNN model to efficiently learn the discriminative features, alleviating the problem of small sample size. Furthermore, self-distillation is embedded into the SCL framework, in which the CNN model jointly distills itself and conducts SCL to further improve feature representation. The proposed SDSCL is evaluated on two public breast histopathological datasets, which outperforms all the compared algorithms. Its average classification accuracy, precision, recall, and F1 scores are 94.28%, 94.64%, 94.58%, 94.34%, respectively, on Bioimaging dataset, and 80.44%, 81.92%, 80.57%, 80.10% on Databiox dataset. The experimental results on two datasets indicate that SDSCL has the potential for the histopathological image based CAD.

Data augmentation and machine learning techniques for control strategy development in bio-polymerization process

Machine learning has been increasingly used in biochemistry. However, in organic chemistry and other experiment-based fields, data collected from real experiments are inadequate and the current coronavirus disease (COVID-19) pandemic has made the situation even worse. Such limited data resources may result in the low performance of modeling and affect the proper development of a control strategy. This paper proposes a feasible machine learning solution to the problem of small sample size in the bio-polymerization process. To avoid overfitting, the variational auto-encoder and generative adversarial network algorithms are used for data augmentation. The random forest and artificial neural network algorithms are implemented in the modeling process. The results prove that data augmentation techniques effectively improve the performance of the regression model. Several machine learning models were compared and the experimental results show that the random forest model with data augmentation by the generative adversarial network technique achieved the best performance in predicting the molecular weight on the training set (with an R2 of 0.94) and on the test set (with an R2 of 0.74), and the coefficient of determination of this model was 0.74.

A novel virtual sample generation method based on a modified conditional Wasserstein GAN to address the small sample size problem in soft sensing

In the modern chemical industry process, soft sensing has been widely used. However, the lack of valid and sufficient data has made it difficult to apply advanced soft sensor modeling methods to realistic scenarios. In this paper, a novel soft sensing method based on deep learning is proposed to handle the problem of small data. Aiming at handling the issue of small sample size, a novel virtual sample generation method embedding a deep neural network as a regressor into conditional Wasserstein generative adversarial networks with gradient penalty (rCWGAN) is presented. In rCWGAN, conditional variables are introduced to make the training supervised and a dual training algorithm is specially designed. With the advanced structure and the designed training algorithm, rCWGAN has powerful sample generation capabilities and can well predict quality variables. Finally, an experiment on the purified terephthalic acid (PTA) solvent system is carried out for the validation of the presented rCWGAN. The simulation results indicate that the presented rCWGAN has good sample approximation ability and acceptable prediction accuracy with the small sample size task.

Performance Comparison of Two Cryosection Embedding Agents Used for Desorption Electrospray Ionization Mass Spectrometry Imaging

To evaluate the potential effect of embedding with carboxylmethyl cellulose (CMC) and embedding with optimal cutting temperature (OCT) compound followed by washing with PBS (OCT-W) on the analysis of breast cancer tissue samples with desorption electrospray ionization mass spectrometry imaging (DESI-MSI). DESI-MSI of fresh frozen (FF) tissue samples, OCT-embedded samples, CMC-embedded samples, and OCT-W samples from the same breast cancer tumor tissue were performed. The ratio of maximum abundance ion was used to assess the reproducibility of DESI-MSI analysis. In addition, the effects of the treatment of each group were examined by comparing the characteristic ion species and the ion signal intensity detected by DESI-MSI. DESI-MSI of continuous sections of FF samples showed that the coefficient of variation (CV) of the pair-to-pair ratios of m/ z 281.25, m/ z 309.28 and m/ z 279.23 ions, the three ions with the highest intensity in the tumor region, were 19.61%, 20.74% and 10.18%, respectively. The characteristic ion species detected by DESI-MSI of CMC embedded tissue and the OCT-W tissue were almost the same, compared with those of the FF tumor tissue. However, ion species detected in OCT embedded samples were less than 50% of the FF samples. In terms of ion signal intensity, the CMC embedded tissue was not affected overall, while the signal of most of the characteristic ions of the OCT-W group showed decreased intensity (P<0.05). FF tissue sections and CMC-embedded samples can be used for DESI-MSI routine analysis. OCT embedding affects the feasibility of sample analysis whether or not the sample undergoes washing with PBS. CMC embedding agent is recommended if the tissue sections need to be fixated and supported due to small sample size, fragility, or other problems.

Automatic fall risk assessment with Siamese network for stroke survivors using inertial sensor‐based signals

Fall is a major threat to stroke survivors with the problems of gait and balance disorders in the rehabilitation phase following severe consequences on quality of life and a heavy burden to their families. Many solutions have been proposed to assess fall risk for elders based on inertial sensor-based signals, however, there still exists a great challenge of transferring them from elderly populations to the stroke-survivors populations as gait disorder patterns are significant difference between elders and stroke survivors. In this study, we conduct a pilot study to collect inertial sensor-based signals from stroke survivors when they performed the timed up and go test, and build an automatic fall risk assessment model with the architecture of Siamese network, with a merit of mitigating the problem of small sample size. Specifically, the proposed automatic fall risk assessment model consists of two parallel convolutional neural networks, each of which is composed of three convolutional layers, two max-pooling layers, and three fully connected layers. To utilize the space relation among accelerator-based and gyroscope-based signals, two-dimensional discrete wavelet transform extracts image-like features, wavelet coefficients, from inertial sensor-based signals as the input. Experimental results show that the proposed fall risk assessment model has achieved a promising results, which outperform cutting-edge methods with a big margin. The proposed fall risk assessment model with low computational complexity and limited memory consuming can be deployed on an embedded system to provide fall risk assessment service for stroke survivors in point-of-care environments or community settings.

Symmetry GAN Detection Network: An Automatic One-Stage High-Accuracy Detection Network for Various Types of Lesions on CT Images

Computed tomography (CT) is the first modern slice-imaging modality. Recent years have witnessed its widespread application and improvement in detecting and diagnosing related lesions. Nonetheless, there are several difficulties in detecting lesions in CT images: (1) image quality degrades as the radiation dose is reduced to decrease radiational injury to the human body; (2) image quality is frequently hampered by noise interference; (3) because of the complicated circumstances of diseased tissue, lesion pictures typically show complex shapes; (4) the difference between the orientated object and the background is not discernible. This paper proposes a symmetry GAN detection network based on a one-stage detection network to tackle the challenges mentioned above. This paper employs the DeepLesion dataset, containing 10,594 CT scans (studies) of 4427 unique patients. The symmetry GANs proposed in this research consist of two distinct GAN models that serve different functions. A generative model is introduced ahead of the backbone to increase the input CT image series to address the typical problem of small sample size in medical datasets. Afterward, GAN models are added to the attention extraction module to generate attention masks. Furthermore, experimental data indicate that this strategy has significantly improved the model’s robustness. Eventually, the proposed method reaches 0.9720, 0.9858, and 0.9833 on P, R, and mAP, on the validation set. The experimental outcome shows that the suggested model outperforms other comparison models. In addition to this innovation, we are inspired by the innovation of the ResNet model in terms of network depth. Thus, we propose parallel multi-activation functions, an optimization method in the network width. It is theoretically proven that by adding coefficients to each base activation function and performing a softmax function on all coefficients, parallel multi-activation functions can express a single activation function, which is a unique ability compared to others. Ultimately, our model outperforms all comparison models in terms of P, R, and mAP, achieving 0.9737, 0.9845, and 0.9841. In addition, we encapsulate the model and build a related iOS application to make the model more applicable. The suggested model also won the second prize in the 2021 Chinese Collegiate Computing Competition.

Multivariate Small Area Modelling for Measuring Micro Level Earning Inequality: Evidence from Periodic Labour Force Survey of India.

The economy of India is growing continuously with its gross domestic product increasing rapidly than most of the developing countries. Nonetheless an increase in national gross domestic product is not revealing the earning parity at micro level in the country. The earning inequality in a country like India has adversely obstructed under privileged in accessing basic needs such as health and education. The Periodic labour force survey (PLFS) conducted by the National Statistical Office of India generates estimates on earning status at state and national level for both rural and urban sectors separately. However, due to a small sample size problem that leads to high sampling variability, these surveys cannot be used directly to produce reliable estimates at micro level such as district or further disaggregate levels. As earnings are often unevenly distributed among the subgroups of comparatively small areas, disaggregate level statistics are inevitably needed in the country for target specific policy planning and monitoring to reduce the earning disparity. Nonetheless, owing to unavailability of estimates at district level, the analysis and spatial mapping related to earning inequality are limited to the national and state level. As a result, the existing variability in disaggregate level earning distribution are often unavailable. This article describes multivariate small area estimation (SAE) to generate precise and representative district-wise model-based estimates of inequality in earning distribution in rural and urban areas of Uttar Pradesh state in India by linking the latest round of PLFS 2018–2019 data and the 2011 Indian Population Census data. The diagnostic measures demonstrate that the district-wise estimates of earning generated by multivariate SAE method are reliable and representative. The spatial maps produced in this analysis reveal district level inequality in earning distribution in the state of Uttar Pradesh. These disaggregate level estimates and spatial mapping of earning distribution are directly pertinent to measuring and monitoring the sustainable development goal 10 of inequality reduction within countries. These expected to offer evidence to executive policy-makers and experts for recognizing the areas demanding additional consideration. This study will definitely provide added advantage to the newly launched schemes of Government of India for fund distribution along with the better monitoring of these schemes.

Bayesian information fusion method for reliability analysis with failure‐time data and degradation data

AbstractDegradation data and failure‐time data are two types of data commonly used in reliability analysis. Both types of data are collected from different sources for reliability analysis of complex products. For highly reliable products, however, it is often difficult to collect sufficient useful data for reliability analysis with high accuracy, which poses the challenge for small sample size problems, that is, single‐type data with few samples. In this paper, three novel Bayesian information fusion models are first proposed to characterize the inherent relationship between the failure‐time data and the degradation data, and further to integrate the heterogeneous data to obtain accurate reliability analysis results under small sample size. Then, a model selection method is developed to choose appropriate model from the Wiener process, gamma process, and IG process models. Finally, the reliability analysis is completed based on the parameter estimation of the Bayesian information fusion model with the aid of the MCMC method. An industrial example is presented to demonstrate the effectiveness of the proposed Bayesian information fusion method.

Daily Activity Recognition and Tremor Quantification from Accelerometer Data for Patients with Essential Tremor Using Stacked Denoising Autoencoders

Human activity recognition (HAR) has received more and more attention, which is able to play an important role in many fields, such as healthcare and intelligent home. Thus, we have discussed an application of activity recognition in the healthcare field in this paper. Essential tremor (ET) is a common neurological disorder that can make people with this disease rise involuntary tremor. Nowadays, the disease is easy to be misdiagnosed as other diseases. We have combined the essential tremor and activity recognition to recognize ET patients’ activities and evaluate the degree of ET for providing an auxiliary analysis toward disease diagnosis by utilizing stacked denoising autoencoder (SDAE) model. Meanwhile, it is difficult for model to learn enough useful features due to the small behavior dataset from ET patients. Thus, resampling techniques are proposed to alleviate small sample size and imbalanced samples problems. In our experiment, 20 patients with ET and 5 healthy people have been chosen to collect their acceleration data for activity recognition. The experimental results show the significant result on ET patients activity recognition and the SDAE model has achieved an overall accuracy of 93.33%. What’s more, this model is also used to evaluate the degree of ET and has achieved the accuracy of 95.74%. According to a set of experiments, the model we used is able to acquire significant performance on ET patients activity recognition and degree of tremor assessment.

Network Traffic Prediction Based on LSTM and Transfer Learning

The increasing amount of traffic in recent years has led to increasingly complex network problems. To be able to improve overall network performance and increase network utilization, it is valuable to take measures to capture future trends in network traffic. In traditional machine learning, to guarantee the accuracy and high reliability of the models obtained through training, there are two basic assumptions: (1) the training samples used for learning and the new test samples satisfy the condition of independent identical distribution; and (2) there must be enough training samples to learn a good model. However, time-series data are not easily accessible in real life, and even after putting in a lot of time and effort to collect them, the data may be unavailable due to confidentiality. In this paper, a neural network model based on LSTM and transfer learning is proposed to address the problem of small sample size in network traffic prediction. Knowledge in the source domain is transferred to the target domain using transfer learning, and a prediction model with good performance is constructed with a small amount of target domain data. The results show that the performance of the transfer learning model improves by more than 40% over the direct training model when using the same samples for predicting 10,000 rows of data, resulting in better performance of the network traffic prediction task.

Semisupervised Feature Extraction of Hyperspectral Image Using Nonlinear Geodesic Sparse Hypergraphs

Recently, the sparse representation (SR)-based graph embedding method has been extensively used in feature extraction (FE) tasks, but it is hard to reveal the complex manifold structure and multivariate relationship of samples in the hyperspectral image (HSI). Meanwhile, the small size sample problem in HSI data also limits the performance of the traditional SR approach. To tackle this problem, this article develops a new semisupervised FE algorithm called a geodesic-based sparse manifold hypergraph (GSMH). The presented method first utilizes the geodesic distance to measure the nonlinear similarity between samples lying on manifold space and further constructs the manifold neighborhood of each sample. Then, a geodesic-based neighborhood SR (GNSR) model is designed to explore the multivariate sparse correlations of different manifold neighborhoods. Considering the multivariate sparse manifold correlations among samples, a pair of semisupervised hypergraphs (HGs) is constructed to effectively incorporate the labeled and unlabeled training information in the embedding process and obtain the nonlinear discriminative feature representation for HSI. Experimental results on three HSI datasets indicate that the proposed method not only achieves satisfying FE performance with limited labeled training samples but also shows superiority compared with other state-of-the-art methods.

Exponential Multi-Modal Discriminant Feature Fusion for Small Sample Size

Multi-modal Canonical Correlation Analysis (MCCA) is an important information fusion method, and some discriminant variations of MCCA have been proposed. However, the variations suffer from the Small Sample Size (SSS) problem and the absence of cross-modal discriminant scatters. Thus we propose a novel exponential multi-modal discriminant feature fusion method for a small amount of training samples, i.e. exponential multi-modal discriminant correlation analysis. In the method, we construct a discriminative integration scatter of all the modalities by constraining the aggregation towards cross-modal discriminative centroids. Besides, the method gives a decomposition-based matrix exponential strategy. The strategy can solve the SSS problem and improve the robustness of noises, and we further provide corresponding theoretical proofs and some intuitive analysis. The method can learn correlation fusion features with well discriminative power from a small amount of samples. Encouraging experimental results show the effectiveness and robustness of our method.