Small Amount Of Training Data Research Articles

Cross domain meta-network for sketch face recognition

Because of the large modal difference between sketch image and optical image, and the problem that traditional deep learning methods are easy to overfit in the case of a small amount of training data, the Cross Domain Meta-Network for sketch face recognition method is proposed. This method first designs a meta-learning training strategy to solve the small sample problem, and then proposes entropy average loss and cross domain adaptive loss to reduce the modal difference between the sketch domain and the optical domain. The experimental results on UoM-SGFS and PRIP-VSGC sketch face data sets show that this method and other sketch face recognition methods.

Classification of community opinion on the use of the Transjakarta bus based on twitter social network using naïve bayes method

At this time, social networking is becoming a very popular communication tool among Indonesians ranging from children to adults. One of them is Twitter, which has quite a lot of users. This social network is used by users to write their opinions on a matter. One of the topics discussed in the Twitter social network is the Trans-Jakarta bus. In this study, the opinions of Trans-Jakarta bus users listed on Twitter are only limited to positive and negative opinions. An application designed to predict positive or negative opinions using the Naïve Bayes algorithm. The Naïve Bayes algorithm was chosen because it only requires a small amount of training data to estimate averages and variable variations. The developed application can make it easier for decision-makers towards Trans-Jakarta to find out public opinion on Trans-Jakarta bus services. The resulting system accuracy is 73% because the amount of training data used is only 62.5% of the total data of 50 data. While the test data that has been used are only 30 data. The proportions of the training and testing data affect the level of accuracy, so if you want a higher level of accuracy you can reproduce the training data used.

Adversarial Attacks Against Network Intrusion Detection in IoT Systems

Deep learning (DL) has gained popularity in network intrusion detection, due to its strong capability of recognizing subtle differences between normal and malicious network activities. Although a variety of methods have been designed to leverage DL models for security protection, whether these systems are vulnerable to adversarial examples (AEs) is unknown. In this article, we design a novel adversarial attack against DL-based network intrusion detection systems (NIDSs) in the Internet-of-Things environment, with only black-box accesses to the DL model in such NIDS. We introduce two techniques: 1) model extraction is adopted to replicate the black-box model with a small amount of training data and 2) a saliency map is then used to disclose the impact of each packet attribute on the detection results, and the most critical features. This enables us to efficiently generate AEs using conventional methods. With these tehniques, we successfully compromise one state-of-the-art NIDS, Kitsune: the adversary only needs to modify less than 0.005% of bytes in the malicious packets to achieve an average 94.31% attack success rate.

Classification Algorithm for Person Identification and Gesture Recognition Based on Hand Gestures with Small Training Sets.

Classification algorithms require training data initially labelled by classes to build a model and then to be able to classify the new data. The amount and diversity of training data affect the classification quality and usually the larger the training set, the better the accuracy of classification. In many applications only small amounts of training data are available. This article presents a new time series classification algorithm for problems with small training sets. The algorithm was tested on hand gesture recordings in tasks of person identification and gesture recognition. The algorithm provides significantly better classification accuracy than other machine learning algorithms. For 22 different hand gestures performed by 10 people and the training set size equal to 5 gesture execution records per class, the error rate for the newly proposed algorithm is from 37% to 75% lower than for the other compared algorithms. When the training set consists of only one sample per class the new algorithm reaches from 45% to 95% lower error rate. Conducted experiments indicate that the algorithm outperforms state-of-the-art methods in terms of classification accuracy in the problem of person identification and gesture recognition.

Regularisation of neural networks by enforcing Lipschitz continuity

We investigate the effect of explicitly enforcing the Lipschitz continuity of neural networks with respect to their inputs. To this end, we provide a simple technique for computing an upper bound to the Lipschitz constant—for multiple p-norms—of a feed forward neural network composed of commonly used layer types. Our technique is then used to formulate training a neural network with a bounded Lipschitz constant as a constrained optimisation problem that can be solved using projected stochastic gradient methods. Our evaluation study shows that the performance of the resulting models exceeds that of models trained with other common regularisers. We also provide evidence that the hyperparameters are intuitive to tune, demonstrate how the choice of norm for computing the Lipschitz constant impacts the resulting model, and show that the performance gains provided by our method are particularly noticeable when only a small amount of training data is available.

Deterioration level estimation via neural network maximizing category-based ordinally supervised multi-view canonical correlation

A deterioration level estimation method via neural network maximizing category-based ordinally supervised multi-view canonical correlation is presented in this paper. This paper focuses on real world data such as industrial applications and has two contributions. First, a novel neural network handling multi-modal features transforms original features into features effectively representing deterioration levels in transmission towers, which are one of the infrastructures, with consideration of only correlation maximization. It can be realized by setting projection matrices maximizing correlations between multiple features into weights of hidden layers. That is, since the proposed network has only a few hidden layers, it can be trained from a small amount of training data. Second, since there exist diverse characteristics and an ordinal scale in deterioration levels, the proposed method newly derives category-based ordinally supervised multi-view canonical correlation analysis (Co-sMVCCA). Co-sMVCCA enables estimation of effective projection considering both within-class divergence and the ordinal scale between classes. Experimental results showed that the proposed method realizes accurate deterioration level estimation.

BiLuNetICP: A Deep Neural Network for Object Semantic Segmentation and 6D Pose Recognition

The ability of understanding a scene and predicting the pose of objects has attracted significant interests in recent years. Specifically, it is used with visual sensors to provide the information for a robotic manipulator to interact with the target. Thus, 6D pose estimation and object recognition from point clouds or RGB-D images are important tasks for visual servoing. In this article, we propose a learning based approach to perform 6D pose estimation for robotic manipulation using a BiLuNetICP pipeline. It consists of a multi-path convolutional neural network (CNN) for semantic segmentation on RGB images. The network extracts the object mask and uses it to merge with the depth information to perform 6D pose estimation by the Iterative Closest Point (ICP) algorithm. We collected our own dataset for training and evaluate with Intersection over Union (IoU). The proposed method is able to provide better results compared with Unet++ using a small amount of training data. For the robotic grasping application, we test and evaluate our approach using a HIWIN 6-axis robot with Asus Xtion Live 3D camera and our structured-light depth camera. The experimental results demonstrate its efficiency in computation and the high success rate in grasping.

InstaCovNet-19: A deep learning classification model for the detection of COVID-19 patients using Chest X-ray

Recently, the whole world became infected by the newly discovered coronavirus (COVID-19). SARS-CoV-2, or widely known as COVID-19, has proved to be a hazardous virus severely affecting the health of people. It causes respiratory illness, especially in people who already suffer from other diseases. Limited availability of test kits as well as symptoms similar to other diseases such as pneumonia has made this disease deadly, claiming the lives of millions of people. Artificial intelligence models are found to be very successful in the diagnosis of various diseases in the biomedical field In this paper, an integrated stacked deep convolution network InstaCovNet-19 is proposed. The proposed model makes use of various pre-trained models such as ResNet101, Xception, InceptionV3, MobileNet, and NASNet to compensate for a relatively small amount of training data. The proposed model detects COVID-19 and pneumonia by identifying the abnormalities caused by such diseases in Chest X-ray images of the person infected. The proposed model achieves an accuracy of 99.08% on 3 class (COVID-19, Pneumonia, Normal) classification while achieving an accuracy of 99.53% on 2 class (COVID, NON-COVID) classification. The proposed model achieves an average recall, F1 score, and precision of 99%, 99%, and 99%, respectively on ternary classification, while achieving a 100% precision and a recall of 99% on the binary class., while achieving a 100% precision and a recall of 99% on the COVID class. InstaCovNet-19’s ability to detect COVID-19 without any human intervention at an economical cost with high accuracy can benefit humankind greatly in this age of Quarantine.

Synthesis of computer simulation and machine learning for achieving the best material properties of filled rubber

Molecular dynamics (MD) simulation is used to analyze the mechanical properties of polymerized and nanoscale filled rubber. Unfortunately, the computation time for a simulation can require several months’ computing power, because the interactions of thousands of filler particles must be calculated. To alleviate this problem, we introduce a surrogate convolutional neural network model to achieve faster and more accurate predictions. The major difficulty when employing machine-learning-based surrogate models is the shortage of training data, contributing to the huge simulation costs. To derive a highly accurate surrogate model using only a small amount of training data, we increase the number of training instances by dividing the large-scale simulation results into 3D images of middle-scale filler morphologies and corresponding regional stresses. The images include fringe regions to reflect the influence of the filler constituents outside the core regions. The resultant surrogate model provides higher prediction accuracy than that trained only by images of the entire region. Afterwards, we extract the fillers that dominate the mechanical properties using the surrogate model and we confirm their validity using MD.

SAR Target Recognition via Meta-Learning and Amortized Variational Inference.

The challenge of small data has emerged in synthetic aperture radar automatic target recognition (SAR-ATR) problems. Most SAR-ATR methods are data-driven and require a lot of training data that are expensive to collect. To address this challenge, we propose a recognition model that incorporates meta-learning and amortized variational inference (AVI). Specifically, the model consists of global parameters and task-specific parameters. The global parameters, trained by meta-learning, construct a common feature extractor shared between all recognition tasks. The task-specific parameters, modeled by probability distributions, can adapt to new tasks with a small amount of training data. To reduce the computation and storage cost, the task-specific parameters are inferred by AVI implemented with set-to-set functions. Extensive experiments were conducted on a real SAR dataset to evaluate the effectiveness of the model. The results of the proposed approach compared with those of the latest SAR-ATR methods show the superior performance of our model, especially on recognition tasks with limited data.

Medical image segmentation using customized U-Net with adaptive activation functions

Since medical imaging is a fundamental step in clinical diagnosis and treatment, medical image processing is an attractive field for researchers. Among the different applications of medical image processing, this paper focuses on the segmentation task using a customized deep convolutional neural network (CNN). The proposed network is developed based on the idea of improving a deep network performance and speeding up its learning process while using less parameters. Using the famous U-Net architecture which has proven its effectiveness in the segmentation field, the customization is done here by applying adaptive activation functions. In the proposed network, the U-Net complexity is reduced about 200 times by alleviating some parameters to accelerate the learning process. However, the important modification is the use of adaptive activation functions where each convolution layer learns its own data-adaptive activation function as a linear combination of 16 well-known basic functions. This modification successfully compensates the accuracy drop caused by parameter alleviation and also makes the model capable to be tuned with small amount of training data. Conducting several experiments on five famous retinal image datasets, namely DRIVE, STARE, CHASE, HRF, and ARIA, the proposed customized U-Net achieved 96%, 97%, 96%, 97%, and 95% accuracy in segmenting blood vessels, respectively. The proposed network also showed 98% accuracy on ISIC skin lesion dataset for segmenting the lesion area. The obtained results obviously confirm the high performance of the proposed customized network compared to the previous successful researches in handling the medical segmentation task. They also light the hope that many famous deep networks can benefit from these types of customization to become efficient compact models with the ability to handle lack of sufficient data.

MJCN: Multi-objective Jaya Convolutional Network for handwritten optical character recognition

In recent years, the non-handcrafted feature extraction methods have gained increasing popularity for solving pattern classification tasks due to their inherent ability to extract robust features and handle outliers. However, the design of such features demands a large set of training data. Meta-heuristic optimization schemes can facilitate feature learning even with a small amount of training data. This paper presents a new feature learning mechanism called multi-objective Jaya convolutional network (MJCN) that attempts to learn meaningful features directly from the images. The proposed scheme, unlike the convolutional neural networks, comprises a convolution layer, a multiplication layer, an activation layer and an optimizer known as multi-objective Jaya optimizer (MJO). The convolution layer searches meaningful patterns in an image through the local neighborhood connections and the multiplication layer projects the convolutional response to a more compact feature space. The weights used in these layers are initialized randomly and MJO is then introduced to optimize the weights. The main objective of MJO is to maximize the inter-class distance and minimize the intra-class variance. The feature vectors are finally derived using the optimized weights. The derived features are finally fed to a set of standard classifiers for recognition of characters. The performance of the proposed model is evaluated on various benchmark datasets, namely, NITR Odia handwritten character, ISI Kolkata Odia numeral, ISI Kolkata Bangla numeral, and MNIST as well as a newly developed dataset NITR Bangla numeral. The experimental results show that the proposed scheme outperforms other state-of-the-art approaches in terms of recognition accuracy.

Spatial pseudo-labeling for semi-supervised facies classification

For the Quantitative classification of facies is crucial to link seismic data with its corresponding lithology for the evaluation of reservoir properties. During the past decade, seismic volumes have increased to the degree that it is perplexing for the geoscientists and seismic interpreters to examine every single seismic section to carry out facies classification. This has triggered and inspired the need and development of machine learning approaches to efficiently predict seismic facies. However, classified data or labeled data (i.e., well log data) are limited by various restraints and needs huge amount of time and cost to obtain, whereas unlabeled data (seismic data) are in abundance and relatively easy to obtain. Geoscientists are tasked with interpreting large amounts of unlabeled data using very little labeled data. In this paper, we adopt a semi-supervised machine learning approach utilizing pseudo-labeling technique to predict seismic facies and to account for the scarcity of training data by taking advantage of unlabeled data. In each iteration, a small amount of unlabeled data is labeled in spatial manner and the most confident labels (labels that are within 95% confidence interval) are added back to the training data set, starting from labeling the unlabeled data adjacent to the well and spatially moving away in small steps (few traces in each step). The labels obtained from the unlabeled data near the well (pseudo-labels) are added to the labeled data to retrain the deep learning model (classifier) in each iteration. This addition of spatial pseudo-labels adds invariance and diversity to the classifier which in turn helps classifier capitalize on known input variance, which might act as a helpful prior knowledge for next prediction. In other words, the proposed workflow accounts for spatial change in lithology that occurs while moving away from the well. Using our proposed workflow, we observed that, the classification accuracy of a machine learning model trained on a small amount of training data can be enhanced considerably. Pseudo-labels also helped improve lateral continuity of facies. Furthermore, the proposed method outperformed conventional methods, achieving a test accuracy of 96.7% and logarithmic loss as low as 0.15 compared to 91% and 0.6, respectively for supervised baseline. Whereas, for training and validation; the training accuracy, validation accuracy and loss were 98.9%, 99.09% and 0.029 respectively.

A robust algorithm for explaining unreliable machine learning survival models using the Kolmogorov–Smirnov bounds

A new robust algorithm based on the explanation method SurvLIME called SurvLIME-KS is proposed for explaining machine learning survival models. The algorithm is developed to ensure robustness to cases of a small amount of training data or outliers of survival data. The first idea behind SurvLIME-KS is to apply the Cox proportional hazards model to approximate the black-box survival model at the local area around a test example due to the linear relationship of covariates in the model. The second idea is to incorporate the well-known Kolmogorov–Smirnov bounds for constructing sets of predicted cumulative hazard functions. As a result, the robust maximin strategy is used, which aims to minimize the average distance between cumulative hazard functions of the explained black-box model and of the approximating Cox model, and to maximize the distance over all cumulative hazard functions in the interval produced by the Kolmogorov–Smirnov bounds. The maximin optimization problem is reduced to the quadratic program. Various numerical experiments with synthetic and real datasets demonstrate the SurvLIME-KS efficiency.

LEARNING WITH REAL-WORLD AND ARTIFICIAL DATA FOR IMPROVED VEHICLE DETECTION IN AERIAL IMAGERY

Abstract. Detecting objects in aerial images is an important task in different environmental and infrastructure-related applications. Deep learning object detectors like RetinaNet offer decent detection performance; however, they require a large amount of annotated training data. It is well known that the collection of annotated data is a time consuming and tedious task, which often cannot be performed sufficiently well for remote sensing tasks since the required data must cover a wide variety of scenes and objects. In this paper, we analyze the performance of such a network given a limited amount of training data and address the research question of whether artificially generated training data can be used to overcome the challenge of real-world data sets with a small amount of training data. For our experiments, we use the ISPRS 2D Semantic Labeling Contest Potsdam data set for vehicle detection, where we derive object-bounding boxes of vehicles suitable for our task. We generate artificial data based on vehicle blueprints and show that networks trained only on generated data may have a lower performance, but are still able to detect most of the vehicles found in the real data set. Moreover, we show that adding generated data to real-world data sets with a limited amount of training data, the performance can be increased significantly, and in some cases, almost reach baseline performance levels.

A Transferable and Effective Method for Monitoring Continuous Cover Forestry at the Individual Tree Level Using UAVs

Transformation to Continuous Cover Forestry (CCF) is a long and difficult process in which frequent management interventions rapidly alter forest structure and dynamics with long lasting impacts. Therefore, a critical component of transformation is the acquisition of up-to-date forest inventory data to direct future management decisions. Recently, the use of single tree detection methods derived from unmanned aerial vehicle (UAV) has been identified as being a cost effective method for inventorying forests. However, the rapidly changing structure of forest stands in transformation amplifies the difficultly in transferability of current individual tree detection (ITD) methods. This study presents a novel ITD Bayesian parameter optimisation approach that uses quantile regression and external biophysical tree data sets to provide a transferable and low cost ITD approach to monitoring stands in transformation. We applied this novel method to 5 stands in a variety of transformation stages in the UK and to a independent test study site in California, USA, to assess the accuracy and transferability of this method. Requiring small amounts of training data (15 reference trees) this approach had a mean test accuracy (F-score = 0.88) and provided mean tree diameter estimates (RMSE = 5.6 cm) with differences that were not significance to the ground data (p < 0.05). We conclude that this method can be used to monitor forests stands in transformation and thus can also be applied to a wide range of forest structures with limited manual parameterisation between sites.

Learning to See in Extremely Low-Light Environments with Small Data

Recent advances in deep learning have shown exciting promise in various artificial intelligence vision tasks, such as image classification, image noise reduction, object detection, semantic segmentation, and more. The restoration of the image captured in an extremely dark environment is one of the subtasks in computer vision. Some of the latest progress in this field depends on sophisticated algorithms and massive image pairs taken in low-light and normal-light conditions. However, it is difficult to capture pictures of the same size and the same location under two different light level environments. We propose a method named NL2LL to collect the underexposure images and the corresponding normal exposure images by adjusting camera settings in the “normal” level of light during the daytime. The normal light of the daytime provides better conditions for taking high-quality image pairs quickly and accurately. Additionally, we describe the regularized denoising autoencoder is effective for restoring a low-light image. Due to high-quality training data, the proposed restoration algorithm achieves superior results for images taken in an extremely low-light environment (about 100× underexposure). Our algorithm surpasses most contrasted methods solely relying on a small amount of training data, 20 image pairs. The experiment also shows the model adapts to different brightness environments.

Anomaly Detection Neural Network with Dual Auto-Encoders GAN and Its Industrial Inspection Applications.

Recently, researchers have been studying methods to introduce deep learning into automated optical inspection (AOI) systems to reduce labor costs. However, the integration of deep learning in the industry may encounter major challenges such as sample imbalance (defective products that only account for a small proportion). Therefore, in this study, an anomaly detection neural network, dual auto-encoder generative adversarial network (DAGAN), was developed to solve the problem of sample imbalance. With skip-connection and dual auto-encoder architecture, the proposed method exhibited excellent image reconstruction ability and training stability. Three datasets, namely public industrial detection training set, MVTec AD, with mobile phone screen glass and wood defect detection datasets, were used to verify the inspection ability of DAGAN. In addition, training with a limited amount of data was proposed to verify its detection ability. The results demonstrated that the areas under the curve (AUCs) of DAGAN were better than previous generative adversarial network-based anomaly detection models in 13 out of 17 categories in these datasets, especially in categories with high variability or noise. The maximum AUC improvement was 0.250 (toothbrush). Moreover, the proposed method exhibited better detection ability than the U-Net auto-encoder, which indicates the function of discriminator in this application. Furthermore, the proposed method had a high level of AUCs when using only a small amount of training data. DAGAN can significantly reduce the time and cost of collecting and labeling data when it is applied to industrial detection.

Auto-Regressive Time Delayed jump neural network for blood glucose levels forecasting

Diabetes mellitus is a widespread chronic disease and is one of the main causes of death worldwide. In order to improve the quality of life of people with diabetes and reduce the occurrence of complications, it is fundamental to prevent glycemic levels from exceeding the physiologic range. With this purpose, many works in recent years have been developed to forecast future glycemic trends using machine learning algorithms that exploit the reading of continuous glucose monitoring sensors, which gather glycemic data from diabetic patients 24 h a day. However, their application is limited in practice by the fact that they usually require a large amount of training data and other heterogeneous features gathered from patients. For this reason, in this work we present a novel neural network capable of predicting future glycemic levels using only the past glucose values as input while needing a small amount of training data. The model is a jump neural network with the addition of feedback connections from the output to the hidden layer, and time delays for each of the input-to-hidden, output-to-hidden and input-to-output connections. Experiments were conducted on a private and a public dataset. We evaluated performance in terms of RMSE and of adverse event detection. The proposed model outperforms other methods suited for time series forecasting, as well as models for blood glucose level prediction present in the literature.

A serialized classification method for pulmonary nodules based on lightweight cascaded convolutional neural network‐long short‐term memory

AbstractComputer Assisted Diagnosis (CAD) is an effective method to detect lung cancer from computed tomography (CT) scans. The development of artificial neural network makes CAD more accurate in detecting pathological changes. Due to the complexity of the lung environment, the existing neural network training still requires large datasets, excessive time, and memory space. To meet the challenge, we analysis 3D volumes as serialized 2D slices and present a new neural network structure lightweight convolutional neural network (CNN)‐long short‐term memory (LSTM) for lung nodule classification. Our network contains two main components: (a) optimized lightweight CNN layers with tiny parameter space for extracting visual features of serialized 2D images, and (b) LSTM network for learning relevant information among 2D images. In all experiments, we compared the training results of several models and our model achieved an accuracy of 91.78% for lung nodule classification with an AUC of 93%. We used fewer samples and memory space to train the model, and we achieved faster convergence. Finally, we analyzed and discussed the feasibility of migrating this framework to mobile devices. The framework can also be applied to cope with the small amount of training data and the development of mobile health device in future.