Large Number Of Training Samples Research Articles

Crop Classification Based on GDSSM-CNN Using Multi-Temporal RADARSAT-2 SAR with Limited Labeled Data

Crop classification is an important part of crop management and yield estimation. In recent years, neural networks have made great progress in synthetic aperture radar (SAR) crop classification. However, the insufficient number of labeled samples limits the classification performance of neural networks. In order to solve this problem, a new crop classification method combining geodesic distance spectral similarity measurement and a one-dimensional convolutional neural network (GDSSM-CNN) is proposed in this study. The method consisted of: (1) the geodesic distance spectral similarity method (GDSSM) for obtaining similarity and (2) the one-dimensional convolutional neural network model for crop classification. Thereinto, a large number of training data are extracted by GDSSM and the generalized volume scattering model which is based on radar vegetation index (GRVI), and then classified by 1D-CNN. In order to prove the effectiveness of the GDSSM-CNN method, the GDSSM method and 1D-CNN method are compared in the case of a limited sample. In terms of evaluation and verification of methods, the GDSSM-CNN method has the highest accuracy, with an accuracy rate of 91.2%, which is 19.94% and 23.91% higher than the GDSSM method and the 1D-CNN method, respectively. In general, the GDSSM-CNN method uses a small number of ground measurement samples, and it uses the rich polarity information in multi-temporal fully polarized SAR data to obtain a large number of training samples, which can quickly improve the accuracy of classification in a short time, which has more new inspiration for crop classification.

Diagnosis of anomalies based on hybrid features extraction in thyroid images.

Diagnosing benign and malignant glands in thyroid ultrasound images is considered a challenging issue. Recently, deep learning techniques have significantly resulted in extracting features from medical images and classifying them. Convolutional neural networks ignore the hierarchical structure of entities within images and do not pay attention to spatial information as well as the need for a large number of training samples. Capsule networks consist of different hierarchical capsules equivalent to the same layers in the convolutional neural networks. We propose a feature extraction method for ultrasound images based on the capsule network. Then, we combine those deep features with conventional features such as Histogram of Oriented Gradients and Local Binary Pattern together to form a hybrid feature space. We increase the accuracy percentage of a support vector machine (SVM) by balancing and reducing the data dimensions of samples. Since the SVM provides different training kernels according to the sample distribution method, the extracted textural features were categorized using each of these kernels to obtain the result. The parameters of classification evaluation using the researcher-made model have outperformed the other methods in this field. Experimental results showed that the combination of HOG, LBP, and CapsNet methods outperformed the others, with 83.95% accuracy in the SVM with a linear kernel.

A Method for Detection of Corn Kernel Mildew Based on Co-Clustering Algorithm with Hyperspectral Image Technology.

Hyperspectral imaging can simultaneously acquire spectral and spatial information of the samples and is, therefore, widely applied in the non-destructive detection of grain quality. Supervised learning is the mainstream method of hyperspectral imaging for pixel-level detection of mildew in corn kernels, which requires a large number of training samples to establish the prediction or classification models. This paper presents an unsupervised redundant co-clustering algorithm (FCM-SC) based on multi-center fuzzy c-means (FCM) clustering and spectral clustering (SC), which can effectively detect non-uniformly distributed mildew in corn kernels. This algorithm first carries out fuzzy c-means clustering of sample features, extracts redundant cluster centers, merges the cluster centers by spectral clustering, and finally finds the category of corresponding cluster centers for each sample. It effectively solves the problems of the poor ability of the traditional fuzzy c-means clustering algorithm to classify the data with complex structure distribution and the complex calculation of the traditional spectral clustering algorithm. The experimental results demonstrated that the proposed algorithm could describe the complex structure of mildew distribution in corn kernels and exhibits higher stability, better anti-interference ability, generalization ability, and accuracy than the supervised classification model.

ViT-HGR: Vision Transformer-based Hand Gesture Recognition from High Density Surface EMG Signals.

Recently, there has been a surge of significant interest on application of Deep Learning (DL) models to autonomously perform hand gesture recognition using surface Electromyogram (sEMG) signals. Many of the existing DL models are, however, designed to be applied on sparse sEMG signals. Furthermore, due to the complex structure of these models, typically, we are faced with memory constraint issues, require large training times and a large number of training samples, and; there is the need to resort to data augmentation and/or transfer learning. In this paper, for the first time (to the best of our knowledge), we investigate and design a Vision Transformer (ViT) based architecture to perform hand gesture recognition from High Density (HD-sEMG) signals. Intuitively speaking, we capitalize on the recent breakthrough role of the transformer architecture in tackling different com-plex problems together with its potential for employing more input parallelization via its attention mechanism. The proposed Vision Transformer-based Hand Gesture Recognition (ViT-HGR) framework can overcome the aforementioned training time problems and can accurately classify a large number of hand gestures from scratch without any need for data augmentation and/or transfer learning. The efficiency of the proposed ViT-HGR framework is evaluated using a recently-released HD-sEMG dataset consisting of 65 isometric hand gestures. Our experiments with 64-sample (31.25 ms) window size yield average test accuracy of 84.62 ± 3.07%, where only 78,210 learnable parameters are utilized in the model. The compact structure of the proposed ViT-based ViT-HGR framework (i.e., having significantly reduced number of trainable parameters) shows great potentials for its practical application for prosthetic control.

A probe-feature for specific emitter identification using axiom-based grad-CAM

Specific emitter identification (SEI) is the process of discriminating different emitters based on the radio fingerprints extracted from the received signals. Without complex parameter adjustment, radio fingerprints can be efficiently extracted by end-to-end deep learning models. However, deep learning models usually rely on a large number of training samples, which can not be provided in the SEI tasks due to data confidentiality. Besides, the mechanism of deep learning models still lacks clear interpretation, reducing the user’s trust in the extracted radio fingerprints for the SEI. In this paper, we propose an interpretable SEI feature, based on the convolutional neural network (CNN), Probe-Feature, resorting to axiom-based Class Activation Mapping (XGrad-CAM). In general, the XGrad-CAM is firstly used as a probe into the CNN to discern important regions of the original features contributive to the current classification. Subsequently, those regions are fused into a discriminative feature with vivid extra-class differences. Experimental results demonstrate that the Probe-Feature can significantly improve the CNN’s classification accuracy compared to the original features as inputs, especially when only scanty training samples are provided. In addition, Probe-Feature also sheds some light on interpreting CNN’s classification with the SEI signals by two axioms, sensitivity, and conservation, used in XGrad-CAM.

An Efficient Knowledge-Based Artificial Neural Network for the Design of Circularly Polarized 3-D-Printed Lens Antenna

An efficient knowledge-based artificial neural network (KBANN) is proposed, and it is used for the design of circularly polarized (CP) lens antenna in this article. In this KBANN, forward neural network (FNN) and inverse neural network (INN) are included. In this model, INN is the major component to predict the antenna structure parameters. As multiple performance indices are required, INN requires a large number of training samples to deduce complex mapping relationship. To solve this problem, FNN is introduced to provide prior knowledge for INN. FNN generates a huge training dataset for INN training, and then the trained INN can directly output the geometric parameters by feeding the target electromagnetic responses as input. This article solves the problem of multiple performance indices in antenna design, and a CP lens antenna with wideband, good axial ratio, and high gain is designed and fabricated to verify the effectiveness of the KBANN model.

Efficient Online Data-Driven Enhanced-XGBoost Method for Antenna Optimization

The tremendous progress in artificial intelligence promotes the wide application of machine learning (ML) technology in the field of electronic science. Recently ML-based antenna optimization provides a distinct candidate and attracts considerable attention. However, the large number of training samples generated through time-consuming EM simulations becomes a significant challenge. In this article, an efficient online data-driven enhanced-XGBoost (E-XGBoost) method for antenna optimization is proposed, which is mainly composed of two parts, i.e., an input variable filter module (IVFM) and an antenna optimization module (AOM). Specifically, IVFM serves as a variable sensitivity analyzer, which is accomplished by E-XGBoost to efficiently reduce the dimension of design variable and hence save the training samples. Next, the design variables obtained by IVFM are fed into AOM to find the near-optimal solution. In AOM, an online learning strategy is proposed to train a local E-XGBoost model to evaluate the population in the metaheuristic optimization algorithm (MOA). Compared to the global ML model that can mimic the entire design space, this local E-XGBoost model can further cut down the training samples. To verify the performance of the proposed method, several different antenna examples, i.e., U-slot patch antenna, Fabry–Perot resonant antenna, and dual-polarized cross dipole antenna and 5G MIMO antenna array, are simulated. Numerical results support the proposed method in terms of its superior performance and potential advantage of saving computational overhead.

Few-Shot Image Classification: Current Status and Research Trends

Conventional image classification methods usually require a large number of training samples for the training model. However, in practical scenarios, the amount of available sample data is often insufficient, which easily leads to overfitting in network construction. Few-shot learning provides an effective solution to this problem and has been a hot research topic. This paper provides an intensive survey on the state-of-the-art techniques in image classification based on few-shot learning. According to the different deep learning mechanisms, the existing algorithms are divided into four categories: transfer learning based, meta-learning based, data augmentation based, and multimodal based methods. Transfer learning based methods transfer useful prior knowledge from the source domain to the target domain. Meta-learning based methods employ past prior knowledge to guide the learning of new tasks. Data augmentation based methods expand the amount of sample data with auxiliary information. Multimodal based methods use the information of the auxiliary modal to facilitate the implementation of image classification tasks. This paper also summarizes the few-shot image datasets available in the literature, and experimental results tested by some representative algorithms are provided to compare their performance and analyze their pros and cons. In addition, the application of existing research outcomes on few-shot image classification in different practical fields are discussed. Finally, a few future research directions are identified.

Deep Learning-Based Dynamic Computation Task Offloading for Mobile Edge Computing Networks.

This paper investigates the computation offloading problem in mobile edge computing (MEC) networks with dynamic weighted tasks. We aim to minimize the system utility of the MEC network by jointly optimizing the offloading decision and bandwidth allocation problems. The optimization of joint offloading decisions and bandwidth allocation is formulated as a mixed-integer programming (MIP) problem. In general, the problem can be efficiently generated by deep learning-based algorithms for offloading decisions and then solved by using traditional optimization methods. However, these methods are weakly adaptive to new environments and require a large number of training samples to retrain the deep learning model once the environment changes. To overcome this weakness, in this paper, we propose a deep supervised learning-based computational offloading (DSLO) algorithm for dynamic computational tasks in MEC networks. We further introduce batch normalization to speed up the model convergence process and improve the robustness of the model. Numerical results show that DSLO only requires a few training samples and can quickly adapt to new MEC scenarios. Specifically, it can achieve normalized system utility by using only four training samples per MEC scenario. Therefore, DSLO enables the fast deployment of computation offloading algorithms in future MEC networks.

Person Identification With Low Training Sample Based on Micro-Doppler Signatures Separation

Due to the ability to handle low-light environments, poor weather conditions and privacy protection, person identification based on radar micro-Doppler (m-D) signatures has emerged as a research interest. Moreover, as it is always difficult to construct large-scale radar data sets, we propose an approach to recognize a person’s identity with limited training sample. In which, the fine m-D signal processing technology is first used to obtain high-quality m-D data spectrograms. A three-layer convolutional principal component analysis network with a dimension optimization architecture (CPCAN-3) is then designed to learn the highly discriminative features and address the identification problem. The model has few network parameters, is easy to train, and has low dependence on a large number of training samples. Different daily activities data are captured in an indoor environment to evaluate the performance of proposed method, and the state-of-the-art algorithms are chosen for comparison. The experimental results show that the proposed scheme performs better performance than the others on small data sets. Especially when the motion of “running” is adopted to identify persons, the model achieves 98% accuracy on the identification of ten people.

A Component Content Measurement Method Modified Using Indirect Hard Modeling for Polymer Blends Based on Raman Spectroscopy.

Raman spectroscopy has been proven to be useful for the component content measurement of polymer blends. However, the soft modeling methods commonly used in quantitative analysis of Raman spectroscopy require a large number of training samples, resulting in a waste of materials and time. This work adopted a modified indirect hard modeling (IHM) method to measure the component content of polymer blends based on Raman spectroscopy. The Raman spectra of polypropylene (PP)/polystyrene (PS) blends with different component content were collected and resolved into the sum of multiple Voigt peak functions. For a large number of peak parameters, the two-dimensional correlation spectroscopy was used to screen out the characteristic Voigt peaks highly correlated with component content to reduce the parameter dimensions and build the parameterized spectral models. The spectral model of the blend was expressed as the weighted sum of the pure component spectral models, during which the parameters of the pure component models were adjusted within a range. According to the relationship between the weight and content of the pure component, a linear regression model for component content prediction was established. The coefficient of determination (R2)/root mean squared error of the IHM component content prediction model was 0.9931/0.4367 wt%. Besides, two popular soft modeling methods, partial least squares and artificial neural network, were compared with the IHM method, which showed that the IHM model had higher prediction accuracy with fewer training samples.

Dropout neuronal unit with tunable probability based on NbOx stochastic memristor for efficient suppression of overfitting

Recent years have witnessed dramatic upsurge of artificial intelligence and neural networks in various application areas. The classification accuracy of neural networks is closely related to the number of training samples, the complexity of the neural network structure, and the number of training epochs. To achieve high performance of the neural network, a large number of training samples are usually required. However, since the number of training samples available can be limited in many real scenes, significant overfitting issue commonly arises and seriously deteriorates the training performance of neural networks. Here, we show a dropout neuronal unit based on NbOx volatile memristor, which can enable dropout function in the neural network constructed. The proposed neural network with such dropout neuron is able to suppress the overfitting problem effectively in the training process, and the role of dropout probability is investigated based on MNIST and FashionMNIST datasets. The results clearly imply the high potential of the dropout neuronal unit in building highly efficient neuromorphic computing systems.

Image Visual Attention Mechanism-based Global and Local Semantic Information Fusion for Multi-modal English Machine Translation

<p>Machine translation is a hot research topic at present. Traditional machine translation methods are not effective because they require a large number of training samples. Image visual semantic information can improve the effect of the text machine translation model. Most of the existing works fuse the whole image visual semantic information into the translation model, but the image may contain different semantic objects. These different local semantic objects have different effects on the words prediction of the decoder. Therefore, this paper proposes a multi-modal machine translation model based on the image visual attention mechanism via global and local semantic information fusion. The global semantic information in the image and the local semantic information are fused into the text attention weight as the image attention. Thus, the alignment information between the hidden state of the decoder and the text of the source language is further enhanced. Experimental results on the English-German translation pair and the Indonesian-Chinese translation pair on the Multi30K dataset show that the proposed model has a better performance than the state-of-the-art multi-modal machine translation models, the BLEU values of English-German translation results and Indonesian-Chinese translation results exceed 43% and 29%, which proves the effectiveness of the proposed model.</p> <p> </p>

Deep Learning-Based 3D Shape Feature Extraction on Flash Animation Style

Flash animation, as a kind of digital learning resource, is an important media for delivering information content, and more importantly, it is an important online learning resource with text, graphics, images, audio, video, interaction, dynamic effects, etc. Flash animation, with its powerful multimedia interaction and presentation capabilities, is widely used in distance education, high-quality course websites, Q&A platforms, etc. With the continuous development of deep learning, the 3D shape feature extraction method combined with deep learning has become a hot research topic. In this paper, we combine deep learning with traditional 3D shape feature extraction methods, so that we can not only break the bottleneck of nondeep learning methods but also improve the accuracy of 3D shape data classification and retrieval tasks, especially in the case of non-rigid 3D shapes. The scheme in this paper not only does not require a large number of training samples but also its feature extraction for flash animation is accurate. Experiments show that the success rate of accurate feature extraction of this paper’s scheme is higher than that of the state-of-the-art methods.

Testing the Ability of Convolutional Neural Networks to Learn Radiomic Features

Background and objectiveRadiomics and deep learning have emerged as two distinct approaches to medical image analysis. However, their relative expressive power remains largely unknown. Theoretically, hand-crafted radiomic features represent a mere subset of features that neural networks can approximate, thus making deep learning a more powerful approach. On the other hand, automated learning of hand-crafted features may require a prohibitively large number of training samples. Here we directly test the ability of convolutional neural networks (CNNs) to learn and predict the intensity, shape, and texture properties of tumors as defined by standardized radiomic features. MethodsConventional 2D and 3D CNN architectures with an increasing number of convolutional layers were trained to predict the values of 16 standardized radiomic features from real and synthetic PET images of tumors, and tested. In addition, several ImageNet-pretrained advanced networks were tested. A total of 4000 images were used for training, 500 for validation, and 500 for testing. ResultsFeatures quantifying size and intensity were predicted with high accuracy, while shape irregularity and heterogeneity features had very high prediction errors and generalized poorly. For example, mean normalized prediction error of tumor diameter with a 5-layer CNN was 4.23 ± 0.25, while the error for tumor sphericity was 15.64 ± 0.93. We additionally found that learning shape features required an order of magnitude more samples compared to intensity and size features. ConclusionsOur findings imply that CNNs trained to perform various image-based clinical tasks may generally under-utilize the shape and texture information that is more easily captured by radiomics. We speculate that to improve the CNN performance, shape and texture features can be computed explicitly and added as auxiliary variables to the networks, or supplied as synthetic inputs.

Meta-learning with implicit gradients in a few-shot setting for medical image segmentation

Widely used traditional supervised deep learning methods require a large number of training samples but often fail to generalize on unseen datasets. Therefore, a more general application of any trained model is quite limited for medical imaging for clinical practice. Using separately trained models for each unique lesion category or a unique patient population will require sufficiently large curated datasets, which is not practical to use in a real-world clinical set-up. Few-shot learning approaches can not only minimize the need for an enormous number of reliable ground truth labels that are labour-intensive and expensive, but can also be used to model on a dataset coming from a new population. To this end, we propose to exploit an optimization-based implicit model agnostic meta-learning (iMAML) algorithm under few-shot settings for medical image segmentation. Our approach can leverage the learned weights from diverse but small training samples to perform analysis on unseen datasets with high accuracy. We show that, unlike classical few-shot learning approaches, our method improves generalization capability. To our knowledge, this is the first work that exploits iMAML for medical image segmentation and explores the strength of the model on scenarios such as meta-training on unique and mixed instances of lesion datasets. Our quantitative results on publicly available skin and polyp datasets show that the proposed method outperforms the naive supervised baseline model and two recent few-shot segmentation approaches by large margins. In addition, our iMAML approach shows an improvement of 2%–4% in dice score compared to its counterpart MAML for most experiments.

Lithium-ion battery state-of-charge estimation for small target sample sets using the improved GRU-based transfer learning

Accurate estimation of the state-of-charge (SOC) of lithium-ion batteries is a key technique for automotive battery management systems to overcome the non-linearity and complications of practical applications. The data-driven approach for estimating SOC requires a large number of training samples and costly input. To this end, an improved gated recurrent unit (GRU)-based transfer learning SOC estimation is proposed for small target sample sets. To ensure the completeness and consistency of data features, Lagrangian interpolations and standard normalization are used for analyzing the open-source battery datasets. The source domain GRU model is pre-trained to obtain rich battery characteristics with the preprocessed datasets; the GRU hidden unit structure can be enhanced, and it is advantageously used in conjunction with transfer learning. Moreover, weight parameters of the source domain are transferred to the GRU model of target batteries. The experimental results show that the proposed improved GRU-based transfer learning can use small target samples to achieve fast and accurate SOC estimations by ordinary computing hardware. In particular, the RMSEs are 1.115%, 1.867%, and 1.141% under dynamic conditions, 32 °C-FUDS, 36 °C-US06, and 50 °C-UDDS, respectively. The proposed method demonstrates the potential of SOC estimation using small target samples-based big data techniques in practice.

A transfer learning object detection model for defects detection in X-ray images of spacecraft composite structures

Usually, defect detection in X-ray images of composite materials is mainly performed by humans, which is time consuming and inefficient. Some deep learning methods have been proposed, but they are only used to identify a certain type of defect or need a large number of training samples. Besides, the size of void and inclusion defects in X-ray images of spacecraft composite structures (SCS) is very small, which is difficult for common object detection methods, especially when the number of training samples is small. Thus, we propose a transfer learning object detection model for detection of both of these defects. Based on domain adaptive Faster R-CNN (DA Faster), the feature pyramid network (FPN) is added into the feature extraction sub-module for multi-scale features adaptation, and small anchor strategies and ROI Align are utilized, which are helpful for localization of small-size defects. Besides, big NMS threshold and conditional domain adaptation (CDAN) is adopted in the box classification branch for more accurate features alignment. The experiment results demonstrate that strong ability of the proposed model in detecting small-size void and inclusion defects of SCS.

Multiview Spatial–Spectral Active Learning for Hyperspectral Image Classification

Supervised classification algorithms on the intricate ground object information of hyperspectral images (HSIs) require a large number of training samples that are annotated manually for model learning. To reduce the labeling cost and improve training sample effectiveness, a multiview spatial–spectral active learning (MVSS-AL) model is proposed in this study. First, a committee model composed of collaborative representation classification is introduced to form a leave-one-class-out (LOCO) multiview strategy, which explores more effective information in the limited training data. Second, the sample query strategy is designed from the perspective of classification confidence (CC) and training contribution (TC). The most inconsistent high-quality samples are screened by making full use of iterative prediction information and spatial–spectral features contained in hyperspectral imagery. Finally, the spatial–spectral LOCO active learning (AL) model obtains target samples through two-layer screening in each iteration and utilizes a support vector machine to obtain the final classification results. The proposed method is tested on three real-world hyperspectral datasets, and the comparison with several novel methods shows that the proposed method is better in the classification performance of restricted sample training.

Unsupervised Domain Adaptation for Low-Dose Computed Tomography Denoising

Deep neural networks have shown great improvements in low-dose computed tomography (CT) denoising. Early deep learning-based low-dose CT denoising algorithms were primarily based on supervised learning. However, supervised learning requires a large number of training samples, which is impractical in real-world scenarios. To address this problem, we propose a novel unsupervised domain adaptation approach for low-dose CT denoising. This proposed framework adapts the network pretrained with paired low- and normal-dose phantom images (source domain) to denoise unlabeled low-dose human CT images (target domain). Our framework modifies the action of the domain classifier, enabling the denoising network to be adapted to the target domain. Furthermore, we introduce a new backpropagation method, which we call domain-independent weighted backpropagation. By combining these techniques, we demonstrate that the denoising network can be properly trained without using clinical clean CT images. The experimental results showed that our method exhibited better performance in terms of both objective and perceptual image qualities when compared with current unsupervised denoising algorithms. Our proposed domain adaptation represents a first-use case in the context of CT denoising problems, with the possibility of extension to other image restoration tasks.