Original Training Samples Research Articles

Artificial immune systems for data augmentation

We study object detection models and observe that their respective architectures are vulnerable to image distortions such as noise, compression, blur, or snow. We propose alleviating this problem by training the models with antibodies generated using Artificial Immune Systems (AIS) from original training samples (antigens). These antibodies are AIS-distorted antigens at the pixel level through cycles of “select, clone, mutate, select” until an affinity to the antigen is achieved. We then add the antibodies to the antigens, train the models, validate and test them under 15 distortions, and show that our data augmentation approach (AISbod) significantly improved their accuracy without altering their architecture or inference speed. For example, the DINO object detector under the COCO dataset improves by 4% under clean samples, by 6.50% on average over all 15 distortions, by 2.15% under snow, and by 27.60% under impulse noise. Our simulations show that our method performs better under distortions and clean samples than related defense methods and is more consistent across datasets and object detection models. For instance, our method is, on average, 70% better than the closest related method across 15 distortions for the evaluated models under COCO. Moreover, we show that our approach to image classification and object tracking models significantly improves accuracy under distortions. We provide the code of our method and the DINO model trained using our method at https://github.com/moforio/AISbod.

Interactive experience of cross-cultural Japanese communication e-learning intelligent system based on computer information security

In the current situation of widespread digital media, e-learning has become the mainstream mode of online learning. Its intelligent and entertaining learning experience can enhance students' interest in learning. This article analyzes the interactive experience of cross-cultural Japanese communication e-learning intelligent system. The e-learning intelligent system can provide network platform support for the learning process, allowing students and teachers to engage in interactive learning through computer systems. At the same time, the system will also recommend learning materials that students are interested in. Adding some encryption techniques to the system in this article is mainly aimed at protecting the security of data. In order to build an intelligent communication system for cross-cultural Japanese, the main parameters of data information are generally used as the basis for experiments, and relevant coding principles are formulated. For some neural networks, the larger the stored values of the original training samples, the higher the sample size level of information parameters after successful decoding. The experimental results indicate that using data security encryption technology in computer information systems can effectively improve the overall performance of cross-cultural Japanese intelligent systems.

Improving Photometric Redshift Estimates with Training Sample Augmentation

Large imaging surveys will rely on photometric redshifts (photo-z's), which are typically estimated through machine-learning methods. Currently planned spectroscopic surveys will not be deep enough to produce a representative training sample for Legacy Survey of Space and Time (LSST), so we seek methods to improve the photo-z estimates that arise from nonrepresentative training samples. Spectroscopic training samples for photo-z's are biased toward redder, brighter galaxies, which also tend to be at lower redshift than the typical galaxy observed by LSST, leading to poor photo-z estimates with outlier fractions nearly 4 times larger than for a representative training sample. In this Letter, we apply the concept of training sample augmentation, where we augment simulated nonrepresentative training samples with simulated galaxies possessing otherwise unrepresented features. When we select simulated galaxies with (g-z) color, i-band magnitude, and redshift outside the range of the original training sample, we are able to reduce the outlier fraction of the photo-z estimates for simulated LSST data by nearly 50% and the normalized median absolute deviation (NMAD) by 56%. When compared to a fully representative training sample, augmentation can recover nearly 70% of the degradation in the outlier fraction and 80% of the degradation in NMAD. Training sample augmentation is a simple and effective way to improve training samples for photo-z's without requiring additional spectroscopic samples.

Generative-Based Fusion Mechanism for Multi-Modal Tracking

Generative models (GMs) have received increasing research interest for their remarkable capacity to achieve comprehensive understanding. However, their potential application in the domain of multi-modal tracking has remained unexplored. In this context, we seek to uncover the potential of harnessing generative techniques to address the critical challenge, information fusion, in multi-modal tracking. In this paper, we delve into two prominent GM techniques, namely, Conditional Generative Adversarial Networks (CGANs) and Diffusion Models (DMs). Different from the standard fusion process where the features from each modality are directly fed into the fusion block, we combine these multi-modal features with random noise in the GM framework, effectively transforming the original training samples into harder instances. This design excels at extracting discriminative clues from the features, enhancing the ultimate tracking performance. Based on this, we conduct extensive experiments across two multi-modal tracking tasks, three baseline methods, and four challenging benchmarks. The experimental results demonstrate that the proposed generative-based fusion mechanism achieves state-of-the-art performance by setting new records on GTOT, LasHeR and RGBD1K. Code will be available at https://github.com/Zhangyong-Tang/GMMT.

A Deep Convolutional Neural Network for Time Series Classification with Intermediate Targets

Deep Convolutional Neural Networks (CNNs) have been successfully used in different applications, including image recognition. Time series data, which are generated in many applications, such as tasks using sensor data, have different characteristics compared to image data, and accordingly, there is a need for specific CNN structures to address their processing. This paper proposes a new CNN for classifying time series data. It is proposed to have new intermediate outputs extracted from different hidden layers instead of having a single output to control weight adjustment in the hidden layers during training. Intermediate targets are used to act as labels for the intermediate outputs to improve the performance of the method. The intermediate targets are different from the main target. Additionally, the proposed method artificially increases the number of training instances using the original training samples and the intermediate targets. The proposed approach converts a classification task with original training samples to a new (but equivalent) classification task that contains two classes with a high number of training instances. The proposed CNN for Time Series classification, called CNN-TS, extracts features depending the distance of two time series. CNN-TS was evaluated on various benchmark time series datasets. The proposed CNN-TS achieved 5.1% higher overall accuracy compared to the CNN base method (without an intermediate layer). Additionally, CNN-TS achieved 21.1% higher average accuracy compared to classical machine-learning methods, i.e., linear SVM, RBF SVM, and RF. Moreover, CNN-TS was on average 8.43 times faster in training time compared to the ResNet method.

Hyperspectral images classification of small sample based on the strategy of sample enlargement by superpixel pair method

ABSTRACT The available training samples for hyperspectral image (HSI) are limited, which will lead to the failure to improve the classification accuracy. Aiming at increasing the number of available training samples, a superpixel pair (SSP) method is proposed to solve the problem of small samples. Combining the advantages of pixel pair (PP) and superpixel, the number of original training samples are augmented multiple times. The SSP method mainly includes three steps: first, according to the regional clustering-based spatial preprocessing (RCSPP) algorithm and its improved algorithms, superpixel segmentation is carried out to realize the first sample enlargement; second, the first expanded training samples are paired to form SSP; last, SVM and deep CNN are used to train and learn the SSP, and the testing pixels obtain the final result through the voting mechanism. Experimental results based on three HSI data sets demonstrate that the proposed method can achieve better classification performance than some existing algorithms to solve the problem of small samples and very small samples classification.

Face Recognition Method under Adaptive Image Matching and Dictionary Learning Algorithm.

In this research, a robust face recognition method based on adaptive image matching and a dictionary learning algorithm was proposed. A Fisher discriminant constraint was introduced into the dictionary learning algorithm program so that the dictionary had certain category discrimination ability. The purpose was to use this technology to reduce the influence of pollution, absence, and other factors on face recognition and improve the recognition rate. The optimization method was used to solve the loop iteration to obtain the expected specific dictionary, and the selected specific dictionary was used as the representation dictionary in adaptive sparse representation. In addition, if a specific dictionary was placed in a seed space of the original training data, the mapping matrix can be used to represent the mapping relationship between the specific dictionary and the original training sample, and the test sample could be corrected according to the mapping matrix to remove the contamination in the test sample. Moreover, the feature face method and dimension reduction method were used to process the specific dictionary and the corrected test sample, and the dimensions were reduced to 25, 50, 75, 100, 125, and 150, respectively. In this research, the recognition rate of the algorithm in 50 dimensions was lower than that of the discriminatory low-rank representation method (DLRR), and the recognition rate in other dimensions was the highest. The adaptive image matching classifier was used for classification and recognition. The experimental results showed that the proposed algorithm had a good recognition rate and good robustness against noise, pollution, and occlusion. Health condition prediction based on face recognition technology has the advantages of being noninvasive and convenient operation.

A Generalized Zero-Shot Quantization of Deep Convolutional Neural Networks Via Learned Weights Statistics

Quantizing the floating-point weights and activations of deep convolutional neural networks to fixed-point representation yields reduced memory footprints and inference time. Recently, efforts have been afoot towards zero-shot quantization that does not require original unlabelled training samples of a given task. These best-published works heavily rely on the learned batch normalization (BN) parameters to infer the range of the activations for quantization. In particular, these methods are built upon either empirical estimation framework or the data distillation approach, for computing the range of the activations. However, the performance of such schemes severely degrades when presented with a network that does not accommodate BN layers. In this line of thought, we propose a generalized zero-shot quantization (GZSQ) framework that neither requires original data nor relies on BN layer statistics. We have utilized the data distillation approach and leveraged only the pre-trained weights of the model to estimate enriched data for range calibration of the activations. To the best of our knowledge, this is the first work that utilizes the distribution of the pretrained weights to assist the process of zero-shot quantization. The proposed scheme has significantly outperformed the existing zero-shot works, e.g., an improvement of ~ 33% in classification accuracy for MobileNetV2 and several other models that are w & w/o BN layers, for a variety of tasks. We have also demonstrated the efficacy of the proposed work across multiple open-source quantization frameworks. Importantly, our work is the first attempt towards the post-training zero-shot quantization of futuristic unnormalized deep neural networks.

A mixture of probabilistic predictable feature analysis for multi-mode dynamic process monitoring

In modern industrial processes, the demand for safety and reliability is increasingly prioritized, spawning quantities of related research on process monitoring models. It is significant for process monitoring to consider multi-mode operating conditions simultaneously. This work proposes an efficient method based on multi-mode probabilistic predictable feature analysis (MPPFA) for complex industrial process monitoring. A strategy combining deep auto-encoder (DAE) and Gaussian mixture model (GMM) is first designed to extract low-dimensional features and identify possible running modes. In line with the obtained features, original training samples are classified into corresponding Gaussian distributions with reasonable probabilities. Further, local PPFA models are established based on the separated training batches. Five monitoring statistics, including TP2, SPEP, and DIP for PPFAs, TD2 and SPED for DAE, are defined to detect abnormalities. The proposed method is first applied to a three-phase flow facility, and its superiority over comparison methods has been verified. The effectiveness of the proposed method is further demonstrated through an experiment on a practical coal pulverising system.

Data Augmentation in 3D Object Detection for self-driving vehicles: the role of original and augmented training samples

Safe self-driving vehicles require precise 3D Object Identification. LiDAR sensors are key in accomplishing such a task, as LiDARs produce high-definition point clouds. Such point clouds are then processed by 3D Object Detection models to finally detect objects.Most Object Detection models require massive amounts of data to be trained. Gathering and processing this data is an expensive and time-consuming task, which is why the information taken from each sample must be fully harnessed. Such can be done through Data Augmentation. Data Augmentation contributes significantly for improving performance, being at least as relevant as the advances in the Object Detection models themselves.A few studies have been reported regarding the effectiveness of Data Augmentation. However, the role played by original and augmented samples has been neglected. This work reports the first-ever detailed quantification of the impact that the inclusion of original and augmented samples in a dataset has in 3D Object Detection in the context of autonomous driving. The obtained results show that although a good augmentation strategy is crucial to the model’s performance, it is only as good as the quality of the original samples allows it to be.

Time series monitoring and prediction of coal mining subsidence based on multitemporal InSAR technology and GSM-HW model

The surface subsidence caused by coal mining will cause serious environmental problems, and an effective monitoring and prediction system is indispensable. Aiming at this phenomenon, a mining subsidence monitoring and dynamic prediction model combining time series stacking differential interferometric synthetic aperture radar (TSS-DInSAR) technology, small baseline subset InSAR technology (SBAS-InSAR), and golden section method-Holt–Winters (GSM-HW) model was proposed. First, based on the nine scene images of Sentinel-1A satellite in the study area, the time series subsidence of 811 work-face in Guobei coal mine from April 5 to July 10, 2021, was obtained using TSS-DInSAR and SBAS-InSAR monitoring technologies, respectively. Then, the original training samples of the GSM-HW model were generated by combining the cumulative surface subsidence results monitored by TSS-DInSAR and SBAS-InSAR, and the monitoring and prediction of surface subsidence were realized using this model. The experimental results show that the application of TSS-DInSAR and SBAS-InSAR monitoring technology to the impact of surface mining in mining areas has certain reliability, and the GSM-HW prediction model can make up for the deficiency of a single HW model in parameter optimization. The maximum fitting accuracy and prediction accuracy of the model for the 10 monitoring pilot sites are 96.9% and 98.4%, respectively, which can provide a reference for the design of surface monitoring and prediction system in the mining area.

MLysPRED: graph-based multi-view clustering and multi-dimensional normal distribution resampling techniques to predict multiple lysine sites.

Posttranslational modification of lysine residues, K-PTM, is one of the most popular PTMs. Some lysine residues in proteins can be continuously or cascaded covalently modified, such as acetylation, crotonylation, methylation and succinylation modification. The covalent modification of lysine residues may have some special functions in basic research and drug development. Although many computational methods have been developed to predict lysine PTMs, up to now, the K-PTM prediction methods have been modeled and learned a single class of K-PTM modification. In view of this, this study aims to fill this gap by building a multi-label computational model that can be directly used to predict multiple K-PTMs in proteins. In this study, a multi-label prediction model, MLysPRED, is proposed to identify multiple lysine sites using features generated from human protein sequences. In MLysPRED, three kinds of multi-label sequence encoding algorithms (MLDBPB, MLPSDAAP, MLPSTAAP) are proposed and combined with three encoding strategies (CHHAA, DR and Kmer) to convert preprocessed lysine sequences into effective numerical features. A multidimensional normal distribution oversampling technique and graph-based multi-view clustering under-sampling algorithm were first proposed and incorporated to reduce the proportion of the original training samples, and multi-label nearest neighbor algorithm is used for classification. It is observed that MLysPRED achieved an Aiming of 92.21%, Coverage of 94.98%, Accuracy of 89.63%, Absolute-True of 81.46% and Absolute-False of 0.0682 on the independent datasets. Additionally, comparison of results with five existing predictors also indicated that MLysPRED is very promising and encouraging to predict multiple K-PTMs in proteins. For the convenience of the experimental scientists, 'MLysPRED' has been deployed as a user-friendly web-server at http://47.100.136.41:8181.

Application of improved virtual sample and sparse representation in face recognition

AbstractSparse representation plays an important role in the research of face recognition. As a deformable sample classification task, face recognition is often used to test the performance of classification algorithms. In face recognition, differences in expression, angle, posture, and lighting conditions have become key factors that affect recognition accuracy. Essentially, there may be significant differences between different image samples of the same face, which makes image classification very difficult. Therefore, how to build a robust virtual image representation becomes a vital issue. To solve the above problems, this paper proposes a novel image classification algorithm. First, to better retain the global features and contour information of the original sample, the algorithm uses an improved non‐linear image representation method to highlight the low‐intensity and high‐intensity pixels of the original training sample, thus generating a virtual sample. Second, by the principle of sparse representation, the linear expression coefficients of the original sample and the virtual sample can be calculated, respectively. After obtaining these two types of coefficients, calculate the distances between the original sample and the test sample and the distance between the virtual sample and the test sample. These two distances are converted into distance scores. Finally, a simple and effective weight fusion scheme is adopted to fuse the classification scores of the original image and the virtual image. The fused score will determine the final classification result. The experimental results show that the proposed method outperforms other typical sparse representation classification methods.

Graph Empirical Mode Decomposition-Based Data Augmentation Applied to Gifted Children MRI Analysis.

Gifted children and normal controls can be distinguished by analyzing the structural connectivity (SC) extracted from MRI data. Previous studies have improved classification accuracy by extracting several features of the brain regions. However, the limited size of the database may lead to degradation when training deep neural networks as classification models. To this end, we propose to use a data augmentation method by adding artificial samples generated using graph empirical mode decomposition (GEMD). We decompose the training samples by GEMD to obtain the intrinsic mode functions (IMFs). Then, the IMFs are randomly recombined to generate the new artificial samples. After that, we use the original training samples and the new artificial samples to enlarge the training set. To evaluate the proposed method, we use a deep neural network architecture called BrainNetCNN to classify the SCs of MRI data with and without data augmentation. The results show that the data augmentation with GEMD can improve the average classification performance from 55.7 to 78%, while we get a state-of-the-art classification accuracy of 93.3% by using GEMD in some cases. Our results demonstrate that the proposed GEMD augmentation method can effectively increase the limited number of samples in the gifted children dataset, improving the classification accuracy. We also found that the classification accuracy is improved when specific features extracted from brain regions are used, achieving 93.1% for some feature selection methods.

Transformer- and Generative Adversarial Network-Based Inpatient Traditional Chinese Medicine Prescription Recommendation: Development Study.

BackgroundTraditional Chinese medicine (TCM) practitioners usually follow a 4-step evaluation process during patient diagnosis: observation, auscultation, olfaction, inquiry, pulse feeling, and palpation. The information gathered in this process, along with laboratory test results and other measurements such as vital signs, is recorded in the patient’s electronic health record (EHR). In fact, all the information needed to make a treatment plan is contained in the EHR; however, only a seasoned TCM physician could use this information well to make a good treatment plan as the reasoning process is very complicated, and it takes years of practice for a medical graduate to master the reasoning skill. In this digital medicine era, with a deluge of medical data, ever-increasing computing power, and more advanced artificial neural network models, it is not only desirable but also readily possible for a computerized system to mimic the decision-making process of a TCM physician.ObjectiveThis study aims to develop an assistive tool that can predict prescriptions for inpatients in a hospital based on patients’ clinical EHRs.MethodsClinical health records containing medical histories, as well as current symptoms and diagnosis information, were used to train a transformer-based neural network model using the corresponding physician’s prescriptions as the target. This was accomplished by extracting relevant information, such as the patient’s current illness, medicines taken, nursing care given, vital signs, examinations, and laboratory results from the patient’s EHRs. The obtained information was then sorted chronologically to produce a sequence of data for the patient. These time sequence data were then used as input to a modified transformer network, which was chosen as a prescription prediction model. The output of the model was the prescription for the patient. The ultimate goal is for this tool to generate a prescription that matches what an expert TCM physician would prescribe. To alleviate the issue of overfitting, a generative adversarial network was used to augment the training sample data set by generating noise-added samples from the original training samples.ResultsIn total, 21,295 copies of inpatient electronic medical records from Guang’anmen Hospital were used in this study. These records were generated between January 2017 and December 2018, covering 6352 types of medicines. These medicines were sorted into 819 types of first-category medicines based on their class relationships. As shown by the test results, the performance of a fully trained transformer model can have an average precision rate of 80.58% and an average recall rate of 68.49%.ConclusionsAs shown by the preliminary test results, the transformer-based TCM prescription recommendation model outperformed the existing conventional methods. The extra training samples generated by the generative adversarial network help to overcome the overfitting issue, leading to further improved recall and precision rates.

Meta-Learning-Based Incremental Few-Shot Object Detection

Recent years have witnessed meaningful progress in the task of few-shot object detection. However, most of the existing models are not capable of incremental learning with a few samples, <i>i.e.</i>, the detector can&#x2019;t detect novel-class objects by using only a few samples of novel classes (without revisiting the original training samples) while maintaining the performances on base classes. This is largely because of catastrophic forgetting, which is a general phenomenon in few-shot learning that the incorporation of the unseen information (<i>e.g.</i>, novel-class objects) will lead to a serious loss of the knowledge learnt before (<i>e.g.</i>, base-class objects). In this paper, a new model is proposed for incremental few-shot object detection, which takes CenterNet as the fundamental framework and redesigns it by introducing a novel meta-learning method to make the model adapted to unseen knowledge while overcoming forgetting to a great extent. Specifically, a meta-learner is trained with the base-class samples, providing the object locator of the proposed model with a good weight initialization, and thus the proposed model can be fine-tuned easily with few novel-class samples. On the other hand, the filters correlated to base classes are preserved when fine-tuning the proposed model with the few samples of novel classes, which is a simple but effective solution to mitigate the problem of forgetting. The experiments on the benchmark MS COCO and PASCAL VOC datasets demonstrate that the proposed model outperforms the state-of-the-art methods by a large margin in the detection performances on base classes and all classes while achieving best performances when detecting novel-class objects in most cases. The project page can be found in <uri>https://mic.tongji.edu.cn/e6/d5/c9778a190165/page.htm</uri>.

Using a multi-convolutional neural network to automatically identify small-sample tea leaf diseases

Tea leaf disease is a factor leading to the decline of tea quality and yield. In this study, a multi-convolutional neural network (CNN) model, MergeModel, combined with diseased leaf segmentation and a weight initialization method, was used for automatic identification of tea leaf diseases in small samples. Diseased leaf segmentation could reduce the impact of complex backgrounds on the identification results. The integration of multiple CNN modules enabled MergeModel to extract a variety of discriminative features. The identification ability of MergeModel was improved compared with a single neural network. The weight initialization method encoded the diseased leaf features into the convolution filter, which helped the model to concentrate on learning important features at the beginning of training. In addition, this study used a small number of diseased tea leaf images as the original training samples and generated new training samples through the unconditional generation model, namely, SinGAN, for data augmentation. Experimental results showed that MergeModel could effectively distinguish diseased tea leaves from healthy tea leaves and identify common tea leaf diseases such as tea white scab, tea leaf blight, tea red scab, and tea sooty mould. Compared with the existing methods, the proposed method had higher accuracy in identifying tea leaf diseases in small samples.

An Efficient Deep Unsupervised Domain Adaptation for Unknown Malware Detection

As an innovative way of communicating information, the Internet has become an indispensable part of our lives. However, it also facilitates a more widespread attack of malware. With the assistance of modern cryptanalysis, emerging malware having symmetric properties, such as encryption and decryption, pack and unpack, presents new challenges to effective malware detection. Currently, numerous malware detection approaches are based on supervised learning. The biggest challenge is that the existing systems rely on a large amount of labeled data, which is usually difficult to gain. Moreover, since the newly emerging malware has a different data distribution from the original training samples, the detection performance of these systems will degrade along with the emergence of new malware. To solve these problems, we propose an Unsupervised Domain Adaptation (UDA)-based malware detection method by jointly aligning the distribution of known and unknown malware. Firstly, the distribution divergence between the source and target domain is minimized with the help of symmetric adversarial learning to learn shared feature representations. Secondly, to further obtain semantic information of unlabeled target domain data, this paper reduces the class-level distribution divergence by aligning the class center of labeled source and pseudo-labeled target domain data. Finally, we mainly use a residual network with a self-attention mechanism to extract more accurate feature information. A series of experiments are performed on two public datasets. Experimental results illustrate that the proposed approach outperforms the existing detection methods with an accuracy of 95.63% and 95.04% in detecting unknown malware on two datasets, respectively.

Application of Deep Learning in Civil Engineering Management.

Construction safety issues are of great significance in civil engineering management. In this paper, the entry point is the recognition of workers wearing helmets during the construction process, and the recognition performance is improved by combining deep learning and traditional classifiers to achieve intelligent recognition of construction safety clothing. In the specific process, the deep residual networks (ResNet) and sparse representation-based classification (SRC) are used as basic classifiers to classify samples with unknown categories. The results of the two decisions are fused and the reliability of the fused decision is determined. Afterwards, the reliable test samples are added to the original training samples to update the classifier, so as to obtain more reliable recognition results. The proposed method is tested and verified with actual measured data. The experimental results show the effectiveness and robustness of the proposed method.

Predicting Classification Accuracy of Unlabeled Datasets Using Multiple Deep Neural Networks

In machine learning problems, we usually assume that the validation accuracy is a good estimation of prediction accuracy for datasets without ground truth. In reality, this assumption may not hold. Therefore, we propose an approach to estimate the prediction accuracy of a target model on unlabeled datasets. The proposed approach uses multiple target homogeneous models to assign each unlabeled sample a confidence value, based on the number of models agreeing on the predicted label. With the confidence values, the prediction accuracy of the target model on the datasets can be estimated. In the experiments, the target model is a convolutional neural network (CNN) model, and the homogeneous models only differ in initial weights. The experiments are conducted with datasets from a wide variety of music genres. The estimation performance of the proposed approach is compared with the reversed testing qualities (RTQ) and the ensemble average qualities (EAQ) approaches. The RTQ approach was proposed to estimate the prediction accuracy of trained models, and the EAQ approach was originally designed for estimating the predictive uncertainty of individual samples. We apply all three compared models to estimate prediction accuracy of datasets by using a linear model. The parameters of the linear model are either computed by using multiple labeled datasets or one labeled dataset. The experimental results show that when compared with the RTQ approach, the proposed approach has much lower estimation errors for some datasets. When compared with the EAQ approach, the proposed approach is more robust for datasets with large distribution shifts. Finally, we show an additional benefit of the proposed approach. In case that the estimated accuracy is unsatisfactory, we may re-train the target model with a new training set, which contains the original training samples plus new training samples with manual labeling from the unlabeled dataset. The experimental results confirm that it is more effective to select (and label) new samples from those with low confidence values than those randomly selected. Overall, the proposed approach is a promising approach for estimating prediction accuracy on unlabeled datasets.