Shortage Of Training Data Research Articles

An Efficient Segmentation and Classification of Brain Tumor Detection using Deep Learning

In medical diagnostics, the identification of brain tumors is a crucial endeavour that frequently necessitates the careful review of intricate imaging data. With the development of deep learning methods, especially convolutional neural networks (CNNs), automated brain tumor detection has become a viable way to help radiologists diagnose patients accurately and quickly. This study provides an extensive overview of current developments in deep learning approaches for brain tumor identification. The suggested techniques usually start with preprocessing medical images, like CT or MRI scans, and then use CNN architectures to extract features. Different CNN architectures, such as U-Net and conventional CNNs, have been used to extract discriminative characteristics from brain pictures. The performance of brain tumor detection systems has also been improved using transfer learning techniques, which make use of pre-trained models on sizable datasets. These results are especially encouraging when there is a shortage of training data. Additionally, research has been done on using ensemble learning approaches to strengthen the generality and durability of brain tumor detection models. By combining several base models into one prediction, these methods improve overall performance and lower the likelihood of overfitting. Furthermore, the incorporation of sophisticated regularisation methods such batch normalisation and dropout has enhanced the deep learning models' capacity for generalisation in brain tumor identification tasks. Moreover, the use of deep learning models in practical clinical environments demands the resolution of issues of uncertainty estimation and model interpretability

Satellite Imagery Analysis for Crop Type Segmentation Using U-Net Architecture

Analyzing satellite images plays a critical role in surveying and monitoring agricultural areas, enabling various applications such as precision agriculture, land use planning, and yield estimation. One of the crucial steps in these applications is accurate crop-type segmentation from satellite imagery. This task becomes challenging in the scenarios of smallholder farms due to their irregular field shapes, frequent cloud cover, small plot sizes, and a severe shortage of training data, which make it difficult to apply conventional machine learning methods effectively. In this research work, a technique for segmenting the crop types from the difficult scenario of smallholder farms based on fully convolutional encoder-decoder semantic segmentation architecture, U-Net, has been proposed and its performance has been compared with the traditional machine learning techniques. To evaluate the proposed approach, experimental assessments were conducted on the Kenya satellite imagery crop dataset. The proposed technique achieved an accuracy, precision, recall, and F1 score of 95.3%, 80.2%, 68.1%, and 73.6%, respectively. The results demonstrate that the U-Net model surpasses conventional image classification methods in accurately segmenting different crop types.

Improving Human Activity Recognition With Wearable Sensors Through BEE: Leveraging Early Exit and Gradient Boosting.

Early-exiting has recently provided an ideal solution for accelerating activity inference by attaching internal classifiers to deep neural networks. It allows easy activity samples to be predicted at shallower layers, without executing deeper layers, hence leading to notable adaptiveness in terms of accuracy-speed trade-off under varying resource demands. However, prior most works typically optimize all the classifiers equally on all types of activity data. As a result, deeper classifiers will only see hard samples during test phase, which renders the model suboptimal due to the training-test data distribution mismatch. Such issue has been rarely explored in the context of activity recognition. In this paper, to close the gap, we propose to organize all these classifiers as a dynamic-depth network and jointly optimize them in a similar gradient-boosting manner. Specifically, a gradient-rescaling is employed to bound the gradients of parameters at different depths, that makes such training procedure more stable. Particularly, we perform a prediction reweighting to emphasize current deep classifier while weakening the ensemble of its previous classifiers, so as to relieve the shortage of training data at deeper classifiers. Comprehensive experiments on multiple HAR benchmarks including UCI-HAR, PAMAP2, UniMiB-SHAR, and USC-HAD verify that it is state-of-the-art in accuracy and speed. A real implementation is measured on an ARM-based mobile device.

Attentional ensemble model for accurate discharge and water level prediction with training data enhancement

Discharge is water flow from a higher to a lower elevation resulting from precipitation or surface runoff. It is an essential resource for human society, including drinking water and irrigation. Forecasting discharge and water level are crucial in establishing plans to secure water sources and predict floods. Deep learning has recently emerged as a potential solution for forecasting discharge and water levels. However, it faces three challenges: training data shortage, noise existence, and underestimation tendency. This research offers a novel deep learning-based method addressing the abovementioned shortcomings. To overcome data scarcity and improve prediction accuracy, we leverage the strengths of the one-dimensional Convolutional Neural Network (1D-CNN), Long short-term memory (LSTM), and Ensemble learning technique. Specifically, 1D-CNN uses a kernel moving in one direction and executing the convolution operations to extract the correlations between the features. LSTM helps to capture temporal relationships inside the data, while ensemble learning exploits multiple learning models for better predictive performance. Besides, Singular-Spectrum Analysis (SSA) technique is applied to decompose the input data into several components and eliminate noise-like ones while retaining essential parts. Besides, the attention mechanism is leveraged to assign higher weights for more essential features, further enhancing the accuracy. Finally, we utilize the Linear Exponential (LINEX) loss to prevent under- and overestimation. The experiments show that our proposal outperforms existing approaches regarding all metrics. Notably, our method improves the Nash–Sutcliffe model Efficiency coefficient (NSE) (the most critical metric) by up to 40% and 34% concerning one-step-ahead and multistep-ahead, respectively, compared to existing approaches.

Hybrid DNN training using both synthetic and real construction images to overcome training data shortage

Hybrid DNN training using both synthetic and real construction images to overcome training data shortage

Physics-informed hierarchical echo state network for predicting the dynamics of chaotic systems

Echo state network (ESN), a type of special recurrent neural network, has gained attention for its simplicity and low computational cost, making it commonly used for data-driven prediction of complex dynamical systems. However, in cases of insufficient or poor-quality data, data-driven approaches can suffer from low prediction accuracy caused by overfitting. To address this problem, a physics-informed hierarchical echo state network (Pi-HESN) is proposed for predicting the dynamics of chaotic systems. Firstly, the Pi-HESN can capture the latent evolutionary patterns hidden in the dynamical systems by processing data layer by layer in stacked reservoirs. Secondly, the Pi-HESN integrates data and physical laws in a unified way, incorporating prior physical knowledge into the objective function to ensure basic physical principles are respected. The combination of data-based and knowledge-based approaches in Pi-HESN improves model generalization, alleviates the shortage of training data, and ensures physical consistency of results. Experiments on four classical chaotic systems illustrate that the proposed Pi-HESN outperforms the original ESN and existing hierarchical ESN-based models in accuracy and predictability horizon.

Fault2SeisGAN: A method for the expansion of fault datasets based on generative adversarial networks

The development of supervised deep learning technology in seismology and related fields has been restricted due to the lack of training sets. A large amount of unlabeled data is recorded in seismic exploration, and their application to network training is difficult, e.g., fault identification. To solve this problem, herein, we propose an end-to-end training data set generative adversarial network Fault2SeisGAN. This network can expand limited labeled datasets to improve the performance of other neural networks. In the proposed method, the Seis-Loss is used to constrain horizon and amplitude information, Fault-Loss is used to constrain fault location information, and the Wasserstein distance is added to stabilize the network training to generate seismic amplitude data with fault location labels. A new fault identification network model was trained with a combination of expansion and original data, and the model was tested using actual seismic data. The results show that the use of the expanded dataset generated in this study improves the performance of the deep neural network with respect to seismic data prediction. Our method solves the shortage of training data set problem caused by the application of deep learning technology in seismology to a certain extent, improves the performance of neural networks, and promotes the development of deep learning technology in seismology.

Combating medical noisy labels by disentangled distribution learning and consistency regularization

Machine learning, particularly the deep convolutional neural network (CNN), has significantly benefited computer-aided diagnosis technology to improve disease identification performance. Nevertheless, the shortage of training data with high-quality labels hinders training an effective and robust model. Medical image annotation is time-consuming and usually requires domain experts’ knowledge. Thus, most medical image datasets contain label noise including uncertainty and inconsistency. This paper proposes a hybrid hard-soft label learning mechanism and consistency regularization from two perspectives to enhance a single model’s disease detection capability instead of struggling with model ensembling. Specifically, a disentangled distribution learning integrates multiple reference models’ predictions and disentangles them into a majority confident label vector and a description degree score vector for co-training the single-target model, mitigating the negative influence caused by label noise. Furthermore, inter- and intra-instance consistency regularization aim to improve the target model’s robustness to give consistent predictions on images with similar lesion appearances. We conducted substantial experiments on the public chest X-ray and fundus image datasets as well as a computer vision benchmark, Clothing 1M, showing that our model surpasses the state-of-the-art methods with consistent improvements. The mean AUC on the CheXpert dataset is increased by 2.44%. The Q.W. Kappa on the KaggleDR+ and FGADR datasets, is increased by 5.65% and 4.66%, respectively. The classification accuracy on the Clothing 1M dataset is increased by 4.47%.

Accurate discharge and water level forecasting using ensemble learning with genetic algorithm and singular spectrum analysis-based denoising

Forecasting discharge (Q) and water level (H) are essential factors in hydrological research and flood prediction. In recent years, deep learning has emerged as a viable technique for capturing the non-linear relationship of historical data to generate highly accurate prediction results. Despite the success in various domains, applying deep learning in Q and H prediction is hampered by three critical issues: a shortage of training data, the occurrence of noise in the collected data, and the difficulty in adjusting the model’s hyper-parameters. This work proposes a novel deep learning-based Q–H prediction model that overcomes all the shortcomings encountered by existing approaches. Specifically, to address data scarcity and increase prediction accuracy, we design an ensemble learning architecture that takes advantage of multiple deep learning techniques. Furthermore, we leverage the Singular-Spectrum Analysis (SSA) to remove noise and outliers from the original data. Besides, we exploit the Genetic Algorithm (GA) to propose a novel mechanism that can automatically determine the prediction model’s optimal hyper-parameters. We conducted extensive experiments on two datasets collected from Vietnam’s Red and Dakbla rivers. The results show that our proposed solution outperforms current techniques across a wide range of metrics, including NSE, MSE, MAE, and MAPE. Specifically, by exploiting the ensemble learning technique, we can improve the NSE by at least 2%. Moreover, with the aid of the SSA-based data preprocessing technique, the NSE is further enhanced by more than 5%. Finally, thanks to GA-based optimization, our proposed model increases the NSE by at least 6% and up to 40% in the best case.

Development of a neural network model for spatial data analysis

Objectives. The paper aimed to develop and validate a neural network model for spatial data analysis. The advantage of the proposed model is the presence of a large number of degrees of freedom allowing its flexible configuration depending on the specific problem. This development is part of the knowledge base of a deep machine learning model repository including a dynamic visualization subsystem based on adaptive web interfaces allowing interactive direct editing of the architecture and topology of neural network models.Methods. The presented solution to the problem of improving the accuracy of spatial data analysis and classification is based on a geosystem approach for analyzing the genetic homogeneity of territorial-adjacent entities of different scales and hierarchies. The publicly available EuroSAT dataset used for initial validation of the proposed methodology is based on Sentinel-2 satellite imagery for training and testing machine learning models aimed at classifying land use/land cover systems. The ontological model of the repository including the developed model is decomposed into domains of deep machine learning models, project tasks and data, thus providing a comprehensive definition of the formalizing area of knowledge. Each stored neural network model is mapped to a set of specific tasks and datasets. Results. Model validation for the EuroSAT dataset algorithmically extended in terms of the geosystem approach allows classification accuracy to be improved under training data shortage within 9% while maintaining the accuracy of ResNet50 and GoogleNet deep learning models.Conclusions. The implemention of the developed model into the repository enhances the knowledge base of models for spatial data analysis as well as allowing the selection of efficient models for solving problems in the digital economy.

Fuzzy Clustered Federated Learning Algorithm for Solar Power Generation Forecasting

Federated learning (FL) is a promising technique to construct a solar power generation forecasting model based on data collected from local generators. However, a set of local generators (i.e., cluster) for FL should be carefully defined to construct a high-accuracy forecasting model. Herein, we propose a fuzzy clustered FL algorithm (FCFLA) where each local generator can be included in more than one cluster. In FCFLA, a local generator has its own membership degree representing its sense of belonging to a specific cluster. Based on this membership degree, FCFLA can generate the high-accuracy forecasting model by catching different characteristics of the data of local generators while addressing the training data shortage problem. Evaluation results demonstrate that FCFLA has the fastest convergence time in achieving the desired accuracy.

Using Domain Adaptation for Incremental SVM Classification of Drift Data

A common assumption in machine learning is that training data is complete, and the data distribution is fixed. However, in many practical applications, this assumption does not hold. Incremental learning was proposed to compensate for this problem. Common approaches include retraining models and incremental learning to compensate for the shortage of training data. Retraining models is time-consuming and computationally expensive, while incremental learning can save time and computational costs. However, the concept drift may affect the performance. Two crucial issues should be considered to address concept drift in incremental learning: gaining new knowledge without forgetting previously acquired knowledge and forgetting obsolete information without corrupting valid information. This paper proposes an incremental support vector machine learning approach with domain adaptation, considering both crucial issues. Firstly, a small amount of new data is used to fine-tune the previous model to generate a model that is sensitive to the new data but retains the previous data information by transferring parameters. Secondly, an ensemble and model selection mechanism based on Bayesian theory is proposed to keep the valid information. The computational experiments indicate that the performance of the proposed model improved as new data was acquired. In addition, the influence of the degree of data drift on the algorithm is also explored. A gain in performance on four out of five industrial datasets and four synthetic datasets has been demonstrated over the support vector machine and incremental support vector machine algorithms.

ULN: An efficient face recognition method for person wearing a mask.

Although the face recognition has advanced by leaps and bounds in recent years, recognizing faces with large occlusion, e.g., masks, is still a challenging problem. In the context of the COVID-19 outbreak, wearing masks becomes mandatory, which fails numerous face attendance and surveillance systems. Therefore, a robust face recognition algorithm that can deal with facial masks is urgently needed. To build a mask-robust face recognition algorithm, we first generate numerous facial images with masks based on public face datasets, which obviously alleviates the problem of the training data shortage. Second, we propose a novel network architecture called Upper-Lower Network (ULN) to recognize the faces with masks efficiently. The upper branch of ULN with the mask-free images as input is pretrained that provides supervisory information for the training of the lower branch. Considering that the occlusion areas of masks usually appear in the lower parts of faces, we further divide the high-order semantic features into upper and lower parts. The designed loss function force the learned features of the lower branch similar to those of the upper branch with the same mask-free image inputs, but only the upper part of features similar to the mask counterparts. Extensive experiments demonstrate that the proposed method is effective for recognizing persons with masks and outperforms other state-of-the-art face recognition methods.

End-to-end Argument Mining with Cross-corpora Multi-task Learning

Abstract Mining an argument structure from text is an important step for tasks such as argument search and summarization. While studies on argument(ation) mining have proposed promising neural network models, they usually suffer from a shortage of training data. To address this issue, we expand the training data with various auxiliary argument mining corpora and propose an end-to-end cross-corpus training method called Multi-Task Argument Mining (MT-AM). To evaluate our approach, we conducted experiments for the main argument mining tasks on several well-established argument mining corpora. The results demonstrate that MT-AM generally outperformed the models trained on a single corpus. Also, the smaller the target corpus was, the better the MT-AM performed. Our extensive analyses suggest that the improvement of MT-AM depends on several factors of transferability among auxiliary and target corpora.

Intrusion detection system combined enhanced random forest with SMOTE algorithm

Network security is subject to malicious attacks from multiple sources, and intrusion detection systems play a key role in maintaining network security. During the training of intrusion detection models, the detection results generally have relatively large false detection rates due to the shortage of training data caused by data imbalance. To address the existing sample imbalance problem, this paper proposes a network intrusion detection algorithm based on the enhanced random forest and synthetic minority oversampling technique (SMOTE) algorithm. First, the method used a hybrid algorithm combining the K-means clustering algorithm with the SMOTE sampling algorithm to increase the number of minor samples and thus achieved a balanced dataset, by which the sample features of minor samples could be learned more effectively. Second, preliminary prediction results were obtained by using enhanced random forest, and then the similarity matrix of network attacks was used to correct the prediction results of voting processing by analyzing the type of network attacks. In this paper, the performance was tested using the NSL-KDD dataset with a classification accuracy of 99.72% on the training set and 78.47% on the test set. Compared with other related papers, our method has some improvement in the classification accuracy of detection.

Conditional Generative Adversarial Networks with Adversarial Attack and Defense for Generative Data Augmentation

Developing deep neural network (DNN) models for computer vision applications for construction is challenging due to the shortage of training data. To address this issue, we proposed a novel data augmentation method that integrates a conditional generative adversarial networks (GANs) framework with a target classifier. The integrated architecture enables adversarial attack and defense during end-to-end training, thereby making it possible to generate effective images for the target classifier’s training. We trained and tested two image classification DNNs with and without data augmentation, where we confirmed the effectiveness of the proposed method: with the data augmentation, the classification accuracy improved by 4.2 percentage points, from 71.24% to 75.46%, with qualitatively improved feature extraction more focused on the target object. Given that the application areas of our method are open-ended, the result is noteworthy. The proposed method can help construction researchers offset the data insufficiency, which will contribute to having more accurate and scalable DNN-powered vision models in construction applications.

Improving robustness of automatic cardiac function quantification from cine magnetic resonance imaging using synthetic image data

Although having been the subject of intense research over the years, cardiac function quantification from MRI is still not a fully automatic process in the clinical practice. This is partly due to the shortage of training data covering all relevant cardiovascular disease phenotypes. We propose to synthetically generate short axis CINE MRI using a generative adversarial model to expand the available data sets that consist of predominantly healthy subjects to include more cases with reduced ejection fraction. We introduce a deep learning convolutional neural network (CNN) to predict the end-diastolic volume, end-systolic volume, and implicitly the ejection fraction from cardiac MRI without explicit segmentation. The left ventricle volume predictions were compared to the ground truth values, showing superior accuracy compared to state-of-the-art segmentation methods. We show that using synthetic data generated for pre-training a CNN significantly improves the prediction compared to only using the limited amount of available data, when the training set is imbalanced.

A Novel Multistage Transfer Learning for Ultrasound Breast Cancer Image Classification.

Breast cancer diagnosis is one of the many areas that has taken advantage of artificial intelligence to achieve better performance, despite the fact that the availability of a large medical image dataset remains a challenge. Transfer learning (TL) is a phenomenon that enables deep learning algorithms to overcome the issue of shortage of training data in constructing an efficient model by transferring knowledge from a given source task to a target task. However, in most cases, ImageNet (natural images) pre-trained models that do not include medical images, are utilized for transfer learning to medical images. Considering the utilization of microscopic cancer cell line images that can be acquired in large amount, we argue that learning from both natural and medical datasets improves performance in ultrasound breast cancer image classification. The proposed multistage transfer learning (MSTL) algorithm was implemented using three pre-trained models: EfficientNetB2, InceptionV3, and ResNet50 with three optimizers: Adam, Adagrad, and stochastic gradient de-scent (SGD). Dataset sizes of 20,400 cancer cell images, 200 ultrasound images from Mendeley and 400 ultrasound images from the MT-Small-Dataset were used. ResNet50-Adagrad-based MSTL achieved a test accuracy of 99 ± 0.612% on the Mendeley dataset and 98.7 ± 1.1% on the MT-Small-Dataset, averaging over 5-fold cross validation. A p-value of 0.01191 was achieved when comparing MSTL against ImageNet based TL for the Mendeley dataset. The result is a significant improvement in the performance of artificial intelligence methods for ultrasound breast cancer classification compared to state-of-the-art methods and could remarkably improve the early diagnosis of breast cancer in young women.

Heterogeneous Clutter Suppression via Affine Transformation on Riemannian Manifold of HPD Matrices

Due to a serious shortage of training data, the performance of adaptive clutter suppression suffers remarkable degradation in heterogeneous environments. To address this problem, a novel clutter suppression method via affine transformation on manifolds is proposed. First, training samples in heterogeneous environments are characterized on an established manifold in which the distribution properties are analyzed. Then, a clutter classification scheme is proposed, whereby the KL divergence decision rule is derived to identify the training data as either homogenous or heterogeneous samples. Afterward, based on the distribution properties of samples and the clutter classification scheme, an affine transformation on manifolds is proposed for sample augmentation by transporting heterogeneous samples into the region of homogeneous samples. Finally, the clutter in the area of interest is suppressed on the manifold, which combines the transformed samples with the homogeneous samples, such that superior performance is obtained. Experiments on both simulated and real data validate the superiority of the proposed method in highly heterogeneous environments.

Facial depth descend: A generation paradigm for facial depth map

Over the past decades, three-dimensional(3D) face recognition has developed rapidly due to its intrinsic invariance to pose and illumination changes. Yet despite this, deep learning is rarely used in 3D face recognition due to lack of training dat a. Noting that the majority of the recognition processes are based on facial depth map instead of real 3D data, we focus on the generation of facial depth map to surmount the shortage of training data. In this paper, we present a novel paradigm, Facial Depth Descend, which generates facial depth map from existing 3D face data. The key to this generation paradigm is to recombine facial components from existing faces and thereafter generate brand new faces based on them. We then propose a learning framework RPC to generate recognition-friendly faces. First, it extracts three facial components (eyes, nose and mouth) from the 3D data of real faces. Then, with these components as input, a Relative Location Estimator (RLE) is used to predict the relative location among them so that they can be composed in a reasonable manner. Finally, the method feeds a Pix2Pix network with these composed facial components to extrapolate areas surrounding them and output a full facial depth map. We here enforce an identity preserving loss on the generation network to make the facial depth map more discriminative and favorable for recognition tasks. Since we can choose different identities’ different components as input, RPC is theoretically able to generate a vast number of novel 2.5D faces. More specifically, RPC can generate an enlarged face dataset as large as N3 identities, where N is the number of identities in the original one. Experiments show that, adding the generated depth maps to the training dataset can help improve the recognition rate in 2.5D face recognition.