Reduced Training Data Research Articles

Reduction of Training Data from Large Datasets Using Encoder and Decoder Algorithm Without Much Compromise of Accuracy

Reduction of Training Data from Large Datasets Using Encoder and Decoder Algorithm Without Much Compromise of Accuracy

Collapsed Building Detection Using 3D Point Clouds and Deep Learning

Collapsed buildings should be detected with the highest priority during earthquake emergency response, due to the associated fatality rates. Although deep learning-based damage detection using vertical aerial images can achieve high performance, as depth information cannot be obtained, it is difficult to detect collapsed buildings when their roofs are not heavily damaged. Airborne LiDAR can efficiently obtain the 3D geometries of buildings (in the form of point clouds) and thus has greater potential to detect various collapsed buildings. However, there have been few previous studies on deep learning-based damage detection using point cloud data, due to a lack of large-scale datasets. Therefore, in this paper, we aim to develop a dataset tailored to point cloud-based building damage detection, in order to investigate the potential of point cloud data in collapsed building detection. Two types of building data are created: building roof and building patch, which contains the building and its surroundings. Comprehensive experiments are conducted under various data availability scenarios (pre–post-building patch, post-building roof, and post-building patch) with varying reference data. The pre–post scenario tries to detect damage using pre-event and post-event data, whereas post-building patch and roof only use post-event data. Damage detection is implemented using both basic and modern 3D point cloud-based deep learning algorithms. To adapt a single-input network, which can only accept one building’s data for a prediction, to the pre–post (double-input) scenario, a general extension framework is proposed. Moreover, a simple visual explanation method is proposed, in order to conduct sensitivity analyses for validating the reliability of model decisions under the post-only scenario. Finally, the generalization ability of the proposed approach is tested using buildings with different architectural styles acquired by a distinct sensor. The results show that point cloud-based methods can achieve high accuracy and are robust under training data reduction. The sensitivity analysis reveals that the trained models are able to locate roof deformations precisely, but have difficulty recognizing global damage, such as that relating to the roof inclination. Additionally, it is revealed that the model decisions are overly dependent on debris-like objects when surroundings information is available, which leads to misclassifications. By training on the developed dataset, the model can achieve moderate accuracy on another dataset with different architectural styles without additional training.

Using enhanced data co-registration to update Spanish National Forest Inventories (NFI) and to reduce training data under LiDAR-assisted inference

ABSTRACT The estimation of forest attributes at the local scale as well as in wall-to-wall approaches benefits from the integration of remote sensing data such as airborne laser scanning (ALS). A poor level of spatial co-registration between ALS data and ground National Forest Inventories (NFI) data has traditionally restricted the application of area-based (ABA) methods. Improving the reliability of the spatial co-registration in NFI can enhance the statistical inference. We evaluated the improvement of ABA models in six forest ecosystems in Spain when upgrading NFI plot positions using commercial-grade global navigation satellite systems (GNSS). The new ABA models based on more accurate NFI positioning reduced the root mean squared error (RMSE) for mean stand volume and basal area by 9.3% and 9.4%, respectively, and 2.1% for mean tree height compared to ABA models based on previous NFI positioning records. The model error and its variability showed a major decrement for the upgraded ABA models when iteratively fitting and using cross-validation. The variability of RMSE estimates decreased at a faster rate compared to ABA models based on less accurate NFI positions. Hence, we derived the minimum sampling intensity needed to control the variability of the RMSE. The results showed that upgraded ABA models required fewer training sample plots, although results were dependent on the assessed forest attribute and the forest ecotype. The observed marginal benefit of improving data co-registration can increase the operational efficiency of NFI designs. With this work, we provide further insights for the measuring of NFI samples considering both ALS-assisted inference and the effect of forest structure.

Predicting On-axis Rotorcraft Dynamic Responses Using Machine Learning Techniques

Physical-law-based models are widely utilized in the aerospace industry. One such use is to provide flight dynamics models for use in flight simulators. For human-in-the-loop use, such simulators must run in real-time. Owing to the complex physics of rotorcraft flight, to meet this real-time requirement, simplifications to the underlying physics sometimes have to be applied to the model, leading to errors in the model's predictions of the real vehicle's response. This study investigated whether a machine-learning technique could be employed to provide rotorcraft dynamic response predictions. Machine learning was facilitated using a Gaussian process (GP) nonlinear autoregressive model, which predicted the on-axis pitch rate, roll rate, yaw rate, and heave responses of a Bo105 rotorcraft. A variational sparse GP model was then developed to reduce the computational cost of implementing the approach on large datasets. It was found that both of the GP models were able to provide accurate on-axis response predictions, particularly when the model input contained all four control inceptors and one lagged on-axis response term. The predictions made showed improvement compared to a corresponding physics-based model. The reduction of training data to one-third (rotational axes) or one-half (heave axis) resulted in only minor degradation of the sparse GP model predictions.

Few-Shot Learning approach for plant disease classification using images taken in the field

Prompt plant disease detection is critical to prevent plagues and to mitigate their effects on crops. The most accurate automatic algorithms for plant disease identification using plant field images are based on deep learning. These methods require the acquisition and annotation of large image datasets, which is frequently technically or economically unfeasible. This study introduces Few-Shot Learning (FSL) algorithms for plant leaf classification using deep learning with small datasets.For the study 54,303 labeled images from the PlantVillage dataset were used, comprising 38 plant leaf and/or disease types (classes). The data was split into a source (32 classes) and a target (6 classes) domain. The Inception V3 network was fine-tuned in the source domain to learn general plant leaf characteristics. This knowledge was transferred to the target domain to learn new leaf types from few images. FSL using Siamese networks and Triplet loss was used and compared to classical fine-tuning transfer learning. The source and target domain sets were split into a training set (80%) to develop the methods and a test set (20%) to obtain the results. Algorithm performance was evaluated using the total accuracy, and the precision and recall per class. For the FSL experiments the algorithms were trained with different numbers of images per class and the experiments were repeated 20 times to statistically characterize the results.The accuracy in the source domain was 91.4% (32 classes), with a median precision/recall per class of 93.8%/92.6%. The accuracy in the target domain was 94.0% (6 classes) learning from all the training data, and the median accuracy (90% confidence interval) learning from 1 image per class was 55.5 (46.0–61.7)%. Median accuracies of 80.0 (76.4–86.5)% and 90.0 (86.1–94.2)% were reached for 15 and 80 images per class, yielding a reduction of 89.1% (80 images/class) in the training dataset with only a 4-point loss in accuracy. The FSL method outperformed the classical fine tuning transfer learning which had accuracies of 18.0 (16.0–24.0)% and 72.0 (68.0–77.3)% for 1 and 80 images per class, respectively.It is possible to learn new plant leaf and disease types with very small datasets using deep learning Siamese networks with Triplet loss, achieving almost a 90% reduction in training data needs and outperforming classical learning techniques for small training sets.

Local rotation invariance in 3D CNNs.

Locally Rotation Invariant (LRI) image analysis was shown to be fundamental in many applications and in particular in medical imaging where local structures of tissues occur at arbitrary rotations. LRI constituted the cornerstone of several breakthroughs in texture analysis, including Local Binary Patterns (LBP), Maximum Response 8 (MR8) and steerable filterbanks. Whereas globally rotation invariant Convolutional Neural Networks (CNN) were recently proposed, LRI was very little investigated in the context of deep learning. LRI designs allow learning filters accounting for all orientations, which enables a drastic reduction of trainable parameters and training data when compared to standard 3D CNNs. In this paper, we propose and compare several methods to obtain LRI CNNs with directional sensitivity. Two methods use orientation channels (responses to rotated kernels), either by explicitly rotating the kernels or using steerable filters. These orientation channels constitute a locally rotation equivariant representation of the data. Local pooling across orientations yields LRI image analysis. Steerable filters are used to achieve a fine and efficient sampling of 3D rotations as well as a reduction of trainable parameters and operations, thanks to a parametric representations involving solid Spherical Harmonics (SH),which are products of SH with associated learned radial profiles. Finally, we investigate a third strategy to obtain LRI based on rotational invariants calculated from responses to a learned set of solid SHs. The proposed methods are evaluated and compared to standard CNNs on 3D datasets including synthetic textured volumes composed of rotated patterns, and pulmonary nodule classification in CT. The results show the importance of LRI image analysis while resulting in a drastic reduction of trainable parameters, outperforming standard 3D CNNs trained with rotational data augmentation.

Training of head and neck segmentation networks with shape prior on small datasets.

Cancer in the head and neck area is commonly treated with radiotherapy. A key step for low-risk treatment is the accurate delineation of organs at risk in the planning imagery. The success of deep learning in image segmentation led to automated algorithms achieving human expert performance on certain datasets. However, such algorithms require large datasets for training and fail to segment previously unseen pathologies, where human experts still succeed. As pathologies are rare and large datasets costly to generate, we investigate the effect of: reduced training data, batch sizes and incorporation of prior knowledge. The small data problem is studied by training a full-volume segmentation network with the reduced amount of data from the MICCAI 2015 head and neck segmentation challenge. To improve the segmentation, we evaluate the batch size as a hyper-parameter and first study and then incorporate a stacked autoencoder as shape prior into the training process. We found that using half of the training data (12 images of 25) results in an accuracy drop of only 3% for the segmentation of organs at risk. Also, the batch size turns out to be relevant for the quality of the segmentation when trained with less than half of the data. By applying PCA on the autoencoder's latent space we achieve a compact and accurate shape model, which is used as a regularizer and significantly improves the segmentation results. Small training data of up to 12 training images is enough to train accurate head and neck segmentation models. By using a shape prior for regularization, the performance of the segmentation can be improved significantly on the full dataset. When training on fewer than 12 images, the batch size is relevant and models have to be trained much longer until convergence.

Reduced-DOF Three-Dimensional STAP via Subarray Synthesis for Nonsidelooking Planar Array Airborne Radar

Compared with conventional two-dimensional space-time adaptive processing (2D-STAP) methods, the elevation-azimuth-Doppler three-dimensional space-time adaptive processing (3D-STAP) method has the advantage of suppressing nonstationary clutter. Thereby, it is suitable for nonsidelooking airborne radar (non-SLAR) applications. However, its huge training data requirements and computational load are often beyond radar's ability in practical clutter environments. In this correspondence, we develop a simple but efficient reduced-degree-of-freedom (DOF) 3D-STAP method that significantly reduces the required training data and the computational complexity while maintaining the suboptimal clutter suppression performance. The proposed method transforms the planar array data into linear array data in azimuth and in elevation, respectively, thereby beamforming an equivalent cross-shape array prior to STAP. In consequence, only a few spatial DOFs, including azimuth and elevation dimension, are used for STAP directly resulting in the potential advantage for nonstationary clutter suppression and drastically reducing training data requirements and computational load. Furthermore, the clutter rank estimation rules of the planar array and the transformed cross-shape array are derived, and the required elevation DOFs of the proposed method are further discussed in detail. Simulations for clutter suppression of non-SLAR show that the proposed STAP method outperforms the state-of-the-art 3D-STAP method in terms of convergence and computational complexity.

Soil Organic Carbon Across Mexico and the Conterminous United States (1991–2010)

AbstractSoil organic carbon (SOC) information is fundamental for improving global carbon cycle modeling efforts, but discrepancies exist from country‐to‐global scales. We predicted the spatial distribution of SOC stocks (topsoil; 0–30 cm) and quantified modeling uncertainty across Mexico and the conterminous United States (CONUS). We used a multisource SOC dataset (>10 000 pedons, between 1991 and 2010) coupled with a simulated annealing regression framework that accounts for variable selection. Our model explained ~50% of SOC spatial variability (across 250‐m grids). We analyzed model variance, and the residual variance of six conventional pedotransfer functions for estimating bulk density to calculate SOC stocks. Two independent datasets confirmed that the SOC stock for both countries represents between 46 and 47 Pg with a total modeling variance of ±12 Pg. We report a residual variance of 10.4 ±5.1 Pg of SOC stocks calculated from six pedotransfer functions for soil bulk density. When reducing training data to define decades with relatively higher density of observations (1991–2000 and 2001–2010, respectively), model variance for predicted SOC stocks ranged between 41 and 55 Pg. We found nearly 42% of SOC across Mexico in forests and 24% in croplands, whereas 31% was found in forests and 28% in croplands across CONUS. Grasslands and shrublands stored 29 and 35% of SOC across Mexico and CONUS, respectively. We predicted SOC stocks >30% below recent global estimates that do not account for uncertainty and are based on legacy data. Our results provide insights for interpretation of estimates based on SOC legacy data and benchmarks for improving regional‐to‐global monitoring efforts.

Using Synthetic Data to Enhance the Accuracy of Fingerprint-Based Localization: A Deep Learning Approach

Human-centered data collection is typically costly and implicates issues of privacy. Various solutions have been proposed in the literature to reduce this cost, such as crowdsourced data collection, or the use of semi-supervised algorithms. However, semi-supervised algorithms require a source of unlabeled data, and crowd-sourcing methods require numbers of active participants. An alternative passive data collection modality is fingerprint-based localization. Such methods use received signal strength (RSS) or channel state information (CSI) in wireless sensor networks to localize users in indoor/outdoor environments. In this paper, we introduce a novel approach to reduce training data collection costs in fingerprint-based localization by using synthetic data. Generative adversarial networks (GANs) are used to learn the distribution of a limited sample of collected data and, following this, to produce synthetic data that can be used to augment the real collected data in order to increase overall positioning accuracy. Experimental results on a benchmark dataset show that by applying the proposed method and using a combination of 10% collected data and 90% synthetic data, we can obtain essentially similar positioning accuracy to that which would be obtained by using the full set of collected data. This means that by employing GAN-generated synthetic data, we can use 90% less real data, thereby reduce data-collection costs while achieving acceptable accuracy.

Speeding up training of automated bird recognizers by data reduction of audio features.

Automated acoustic recognition of birds is considered an important technology in support of biodiversity monitoring and biodiversity conservation activities. These activities require processing large amounts of soundscape recordings. Typically, recordings are transformed to a number of acoustic features, and a machine learning method is used to build models and recognize the sound events of interest. The main problem is the scalability of data processing, either for developing models or for processing recordings made over long time periods. In those cases, the processing time and resources required might become prohibitive for the average user. To address this problem, we evaluated the applicability of three data reduction methods. These methods were applied to a series of acoustic feature vectors as an additional postprocessing step, which aims to reduce the computational demand during training. The experimental results obtained using Mel-frequency cepstral coefficients (MFCCs) and hidden Markov models (HMMs) support the finding that a reduction in training data by a factor of 10 does not significantly affect the recognition performance.

User-adaptive models for activity and emotion recognition using deep transfer learning and data augmentation

Building predictive models for human-interactive systems is a challenging task. Every individual has unique characteristics and behaviors. A generic human–machine system will not perform equally well for each user given the between-user differences. Alternatively, a system built specifically for each particular user will perform closer to the optimum. However, such a system would require more training data for every specific user, thus hindering its applicability for real-world scenarios. Collecting training data can be time consuming and expensive. For example, in clinical applications it can take weeks or months until enough data is collected to start training machine learning models. End users expect to start receiving quality feedback from a given system as soon as possible without having to rely on time consuming calibration and training procedures. In this work, we build and test user-adaptive models (UAM) which are predictive models that adapt to each users’ characteristics and behaviors with reduced training data. Our UAM are trained using deep transfer learning and data augmentation and were tested on two public datasets. The first one is an activity recognition dataset from accelerometer data. The second one is an emotion recognition dataset from speech recordings. Our results show that the UAM have a significant increase in recognition performance with reduced training data with respect to a general model. Furthermore, we show that individual characteristics such as gender can influence the models’ performance.

Persymmetric subspace adaptive detection and performance analysis

This study deals with the problem of detecting a subspace signal in coloured Gaussian noise, where the subspace signal belongs to a known subspace, but with unknown coordinates. The authors exploit the persymmetric structure of the covariance matrix by a unitary transform and then devise a persymmetric subspace detector based on two-step design procedure. By exploiting the persymmetric structure of the covariance matrix, the proposed detector can reduce training data requirements. Additionally, approximate expressions for the probabilities of false alarm and detection of the proposed detector are derived. Numerical results demonstrate that the proposed detector can offer significantly enhanced detection performance in comparison with the conventional counterparts when the amount of training data is limited.

Selection of optimum frequency bands for detection of epileptiform patterns.

The significant research effort in the domain of epilepsy has been directed toward the development of an automated seizure detection system. In their usage of the electrophysiological recordings, most of the proposals thus far have followed the conventional practise of employing all frequency bands following signal decomposition as input features for a classifier. Although seemingly powerful, this approach may prove counterproductive since some frequency bins may not carry relevant information about seizure episodes and may, instead, add noise to the classification process thus degrading performance. A key thesis of the work described here is that the selection of frequency subsets may enhance seizure classification rates. Additionally, the authors explore whether a conservative selection of frequency bins can reduce the amount of training data needed for achieving good classification performance. They have found compelling evidence that using spectral components with <25 Hz frequency in scalp electroencephalograms can yield state-of-the-art classification accuracy while reducing training data requirements to just a tenth of those employed by current approaches.

Extremely Reduced Data Sets Indicate Optimal Stimulation Parameters for Classification in Brain-Computer Interfaces.

The time between the onset of subsequent auditory or visual stimuli - also known as stimulus onset asynchrony (SOA) - determines many of the event-related potential characteristics of the resulting evoked brain signals. Specifically, the SOA value influences the performance of an individual subject in brain-computer interface (BCI) applications like spellers. In the past, subject-specific optimization of the SOA was rarely considered in BCI studies. Our research strives to reduce the time requirements of individual BCI stimulus parameter optimization. This work contributes to this goal in two ways. First, we show that even the classification performance on extremely reduced training data subsets reveals the influence of SOA. Second, we show, that these noisy estimates are sufficient to make decisions for individual choices of the SOA that transfer to good classification performance on large training data sets. Thus our work contributes to individually tailored SOA selection procedures for BCI users.

Pengenalan Karakter Tulisan Tangan Dengan K-Support Vector Nearest Neighbor

Handwritten characters are difficult to be recognized by machine because people had various own writing style. This research recognizes handwritten character pattern of numbers and alphabet using K-Nearest Neighbour (KNN) algorithm. Handwritten recognition process is worked by preprocessing handwritten image, segmentation to obtain separate single characters, feature extraction, and classification. Features extraction is done by utilizing Zone method that will be used for classification by splitting this features data to training data and testing data. Training data from extracted features reduced by K-Support Vector Nearest Neighbor (K-SVNN) and for recognizing handwritten pattern from testing data, we used K-Nearest Neighbor (KNN). Testing result shows that reducing training data using K-SVNN able to improve handwritten character recognition accuracy.

Reduction of Training Data Using Parallel Hyperplane for Support Vector Machine

ABSTRACTSupport Vector Machine (SVM) is an efficient machine learning technique applicable to various classification problems due to its robustness. However, its time complexity grows dramatically as the number of training data increases, which makes SVM impractical for large-scale datasets. In this paper, a novel Parallel Hyperplane (PH) scheme is introduced which efficiently omits redundant training data with SVM. In the proposed scheme the PHs are recursively formed while the clusters of data points outside the PHs are removed at each repetition. Computer simulation reveals that the proposed scheme greatly reduces the training time compared to the existing clustering-based reduction scheme and SMO scheme, while allowing the accuracy of classification as high as no data reduction scheme.

PRACTICAL CONSIDERATIONS OF GREEDY COMPRESSED SAMPLING METHODS APPLICATION FOR OFDM CHANNEL ESTIMATION

Context. Traditionally a large number of pilot carriers are utilized to acquire channel state information in OFDM based systems. A larger number of pilot carriers gives better channel state estimation but leads to lower spectrum efficiency of the system. Objective. Primary objective of this paper is to look at the practical aspects of the application of novel CS-based channel estimation technique, that can achieve estimation quality on reduced training data, in context of real pilot aided OFDM systems. Method. A novel technique CS enables representation of sparse signals using fewer samples as compared to its original size. Exploiting the sparse nature of channel impulse response of multipath channels, we apply the CS technique for channel estimation in pilot aided OFDM system based on ISDB-T standard. Results. In this paper, we consider two the most popular CS-based recovery algorithms – OMP and CoSaMP. The MSE performance metrics are given for both CS-based channel estimation algorithms. Simulations results demonstrate that CoSaMP provides more stable results while requires more pilot carriers than OMP to achieve good estimation quality. Both algorithms require a priori knowledge of channel sparsity level but CoSaMP is much more sensitive to a correctness of this information. Conclusions. The compressed sampling approach shows the impressive capability of channel impulse response recovery from a significantly smaller amount of pilot carriers than traditional linear methods require. However, the need of sparsity knowledge by the most popular CS recovery methods seriously limits the applicability of these algorithms in real OFDM receivers. Nevertheless, CSbased channel estimation is a promising technique which worth further investigation to overcome this limitation.

Supervised band selection in hyperspectral images using single-layer neural networks

ABSTRACTHyperspectral images provide fine details of the scene under analysis in terms of spectral information. This is due to the presence of contiguous bands that make possible to distinguish different objects even when they have similar colour and shape. However, neighbouring bands are highly correlated, and, besides, the high dimensionality of hyperspectral images brings a heavy burden on processing and also may cause the Hughes phenomenon. It is therefore advisable to make a band selection pre-processing prior to the classification task. Thus, this paper proposes a new supervised filter-based approach for band selection based on neural networks. For each class of the data set, a binary single-layer neural network classifier performs a classification between that class and the remainder of the data. After that, the bands related to the biggest and smallest weights are selected, so, the band selection process is class-oriented. This process iterates until the previously defined number of bands is achieved. A comparison with three state-of-the-art band selection approaches shows that the proposed method yields the best results in 43.33% of the cases even with greatly reduced training data size, whereas the competitors have achieved between 13.33% and 23.33% on the Botswana, KSC and Indian Pines datasets.

Batch Mode Query by Committee for Motor Imagery-Based BCI.

Although brain-computer interface (BCI) has potential application in the rehabilitation of neural disease and performance improvement of the human in the loop system, it is restricted in the laboratory environment. One of the hindrances behind this restriction is the requirement of a long training data collection session for the user prior to operation of the system at each time. Several approaches have been proposed including the reduction of training data maintaining the robust performance. One of them is active learning (AL) which asks for labeling the training samples and it has the potential to reach robust performance using reduced informative training set. In this paper, one of the AL methods, query by committee (QBC), is applied by forming the committee in heterogeneous and homogeneous feature space. In heterogeneous feature space, three state-of-the-art feature extraction methods are coupled with linear discriminant analysis classifier. For homogeneous feature space, random K -fold sampling is applied after extracting the features using a single method to form the committee of K -members. The joint accuracy by QBC-heterogeneous has obtained the baselines using maximum 35% of the whole training set. It also shows a significant difference at the 5% significance level from QBC-homogeneous selection as well as other contemporary AL methods and random selection method. Thus, QBC-heterogeneous has reduced the labeling effort and the training data collection effort significantly more than that of random labeling process. It infers that QBC is a potential candidate for abridging overall calibration time of BCI systems.