Train Deep Convolutional Neural Networks Research Articles

Object detection and multi‐object tracking based on optimized deep convolutional neural network and unscented Kalman filtering

SummaryThe tracking of multiple objects has gained immense interest in several applications that ranges from video surveillance and robotics. The goal of tracking multiple objects is to discover correspondences and perform matching among various objects in frames. However, occlusion is a major issue in several classical techniques, which fail to track multiple moving objects. Here, the extraction of frames is done to obtain information about an action. Thereafter, the segmentation of objects is done using the entropy weighting K‐means (EWKM) algorithm to generate optimum segments from the frame. After that, the detection of the object is done using a deep convolutional neural network (DCNN) to obtain objects from segments. Proposed Jaya‐political optimizer (Jaya‐PO) is used to train DCNN, which is devised by collaborating political optimizer (PO) and Jaya optimization algorithm. After detecting the object, the tracking of multiple objects is done using the proposed Henry gas solubility optimized unscented Kalman filtering (HGSO‐based UKF), wherein the UKF is modified with HGSO. The proposed Jaya‐PO‐based DCNN and HGSO‐based UKF outperformed other methods with high accuracy of 92%, high multiple object tracking precision (MOTP) of 87.6%, high sensitivity of 91.7%, and high specificity of 92%.

Transductive Semisupervised Deep Hashing.

Deep hashing methods have shown their superiority to traditional ones. However, they usually require a large amount of labeled training data for achieving high retrieval accuracies. We propose a novel transductive semisupervised deep hashing (TSSDH) method which is effective to train deep convolutional neural network (DCNN) models with both labeled and unlabeled training samples. TSSDH method consists of the following four main ingredients. First, we extend the traditional transductive learning (TL) principle to make it applicable to DCNN-based deep hashing. Second, we introduce confidence levels for unlabeled samples to reduce adverse effects from uncertain samples. Third, we employ a Gaussian likelihood loss for hash code learning to sufficiently penalize large Hamming distances for similar sample pairs. Fourth, we design the large-margin feature (LMF) regularization to make the learned features satisfy that the distances of similar sample pairs are minimized and the distances of dissimilar sample pairs are larger than a predefined margin. Comprehensive experiments show that the TSSDH method can produce superior image retrieval accuracies compared to the representative semisupervised deep hashing methods under the same number of labeled training samples.

Deep Learning Algorithm in Biomedical Engineering in Intelligent Automatic Processing and Analysis of Sports Images

In order to improve the detection and identification ability of sports injury ultrasound medicine, a segmentation method of sports injury ultrasound medical image based on local features is proposed, and the research on the sports injury ultrasound medical detection and identification ability is carried out. Methods of the sports injury ultrasound medical image segmentation model are established; the sports injury ultrasound medical image information is enhanced by using the sports skeletal muscle block matching technology; the image features are extracted; and the characteristics of sports injury ultrasound medical images are analyzed by CT bright spot feature transmission. In detail, combined with the deep convolutional neural network training method, the extracted sports injury points are automatically detected for sports injury ultrasound medical images, and the sports injury ultrasound medical image segmentation is realized. The simulation results show that this method has high accuracy for sports injury ultrasound medical image segmentation, the error value can be controlled below 0.103, and finally, the effect of zero error is achieved. It is confirmed that the method proposed in this paper has high resolution and accuracy for sports injury point detection and has strong practical application ability.

N-1 static security assessment method for power grids with high penetration rate of renewable energy generation

A new N-1 static security assessment method based on deep convolutional neural network (DCNN) for the power grid with a high penetration rate of renewable energy generation is proposed in this paper. First, to generate realistic wind and solar scenarios for better training of DCNN, we built a dual discriminator generative adversarial network (D2GAN). Second, a DCNN for calculating the N-1 static security index of the power grid was built. The inputs are operating conditions constructed by the generated wind and solar power scenarios and the line outages, and the output is the corresponding N-1 static security index. Finally, the numerical experimental results based on the historical wind and solar power shows that the scenarios generated by D2GAN fit the real scenarios better than the scenarios generated by the commonly used Monte Carlo (MC) method. And the numerical results of applying the proposed approach to IEEE 14-bus, 30-bus and 118-bus test system demonstrate its effectiveness for N-1 contingency analysis. In terms of accuracy and computation speed, the comparison with traditional method based on load flow analysis, the ANN, and the DCNN trained with the scenarios generated by MC indicate that the proposed model is reliable and more efficient.

A Data-Adaptive Loss Function for Incomplete Data and Incremental Learning in Semantic Image Segmentation.

In the last years, deep learning has dramatically improved the performances in a variety of medical image analysis applications. Among different types of deep learning models, convolutional neural networks have been among the most successful and they have been used in many applications in medical imaging. Training deep convolutional neural networks often requires large amounts of image data to generalize well to new unseen images. It is often time-consuming and expensive to collect large amounts of data in the medical image domain due to expensive imaging systems, and the need for experts to manually make ground truth annotations. A potential problem arises if new structures are added when a decision support system is already deployed and in use. Since the field of radiation therapy is constantly developing, the new structures would also have to be covered by the decision support system. In the present work, we propose a novel loss function to solve multiple problems: imbalanced datasets, partially-labeled data, and incremental learning. The proposed loss function adapts to the available data in order to utilize all available data, even when some have missing annotations. We demonstrate that the proposed loss function also works well in an incremental learning setting, where an existing model is easily adapted to semi-automatically incorporate delineations of new organs when they appear. Experiments on a large in-house dataset show that the proposed method performs on par with baseline models, while greatly reducing the training time and eliminating the hassle of maintaining multiple models in practice.

An improved categorical cross entropy for remote sensing image classification based on noisy labels

In recent years, volunteered geographic information (VGI) is widely used to train deep convolutional neural networks (DCNNs) for high-resolution remote sensing image classification. However, noisy labels are often included in the training samples generated by VGI, which inevitably affects the performance of DCNNs. To solve this problem, the negative effect of noisy labels on remote sensing image classification by DCNNs is analyzed. Then, an improved categorical cross-entropy (ICCE) is proposed to address the issue of noisy labels. The ICCE improves the robustness of DCNN to noisy labels by revisiting the sample weighting scheme so that much attention is paid to the clean samples instead of the noisy samples. Besides, the error bound of ICCE is derived with strict mathematical proof under different types of noisy labels, which ensures the advantages of ICCE from theory. Additionally, extensive experiments are conducted on three remote sensing image datasets with simulated and real noisy labels to quantitatively evaluate the performance of ICCE.

Performance evaluation of China's photovoltaic poverty alleviation project using machine learning and satellite images

Photovoltaic poverty alleviation project (PPAP) is one of China's essential targeted poverty alleviation projects. This study proposes a machine learning model and uses satellite images to evaluate the performance of PPAP in China. The trained deep convolutional neural network (DCNN) with transfer learning was first used to identify the scale of photovoltaic (PV) power stations. Then the PV power capacity for poverty alleviation and carbon emission mitigation were estimated. The results identified 38 large-scale centralized and approximately 5,063,293 m2PV power stations built in Jinzhai County, Anhui province, China, by November 2020. The main findings are as follows. (1) The power generation and carbon mitigation of PPAP in Jinzhai County is about 1.8×103 MWh and 1.389 Mt per year. (2) The PPAP in Jinzhai County can recover the total costs and get benefits within three years, at least after completion. (3) Dynamic subsidy policy is needed to prevent over-scale or excessive government investment in PPAP. (4) The utilization of PPAP needs to be strengthened to transform the current “blood transfusion type” of poverty alleviation that relies more on government subsidies into more sustainable “hematopoietic style” poverty alleviation. This study is of significance to more accurately and comprehensively evaluate the performance of PPAP and give better utilize the role of renewable energy in promoting energy conservation, carbon emission reduction, and economic development.

Estimating boreal forest ground cover vegetation composition from nadir photographs using deep convolutional neural networks

Ground cover and surface vegetation information are key inputs to wildfire propagation models and are important indicators of ecosystem health. Often these variables are approximated using visual estimation by trained professionals but the results are prone to bias and error. This study analyzed the viability of using nadir or downward photos from smartphones (iPhone 7) to provide quantitative ground cover and biomass loading estimates. Good correlations were found between field measured values and pixel counts from manually segmented photos delineating a pre-defined set of 10 discrete cover types. Although promising, segmenting photos manually was labor intensive and therefore costly. We explored the viability of using a trained deep convolutional neural network (DCNN) to perform image segmentation automatically. The DCNN was able to segment nadir images with 95% accuracy when compared with manually delineated photos. To validate the flexibility and robustness of the automated image segmentation algorithm, we applied it to an independent dataset of nadir photographs captured at a different study site with similar surface vegetation characteristics to the training site with promising results.

Comparison of radiologist versus natural language processing-based image annotations for deep learning system for tuberculosis screening on chest radiographs

Although natural language processing (NLP) can rapidly extract disease labels from radiology reports to create datasets for deep learning models, this may be less accurate than having radiologists manually review the images. In this study, we compared agreement between natural language processing (NLP) and radiologist-curated labels for possible tuberculosis (TB) on chest radiographs (CXR) and evaluated the performance of deep convolutional neural networks (DCNN) trained to identify TB using the preceding two sets of labels. We collected 10,951 CXRs from the NIH ChestX-ray14 dataset and labeled them as positive or negative for possible TB based on two methods: 1) NLP-derived disease labels and 2) radiologist-review of images. These images were used to train DCNNs on varying dataset sizes for possible TB and tested on an external dataset of 800 CXRs. Area under the ROC curve (AUC) was used to evaluate DCNNs. There was poor agreement between NLP and radiologist-curated labels for potential TB (Kappa coefficient 0.34). DCNNs trained using radiologist-curated labels had higher performance than the algorithm trained using the NLP-labels, regardless of the number of images used for training. The best-performing DCNN had an AUC of 0.88, which was trained on 10,951 images using the radiologist-annotated sets. DCNNs trained on CXRs labeled by a radiologist consistently outperformed those trained on the same CXRs labeled by NLP, highlighting the benefit of radiologists' determining groundtruth for machine learning dataset curation.

Training Deep Convolutional Spiking Neural Networks With Spike Probabilistic Global Pooling.

Recent work on spiking neural networks (SNNs) has focused on achieving deep architectures. They commonly use backpropagation (BP) to train SNNs directly, which allows SNNs to go deeper and achieve higher performance. However, the BP training procedure is computing intensive and complicated by many trainable parameters. Inspired by global pooling in convolutional neural networks (CNNs), we present the spike probabilistic global pooling (SPGP) method based on a probability function for training deep convolutional SNNs. It aims to remove the difficulty of too many trainable parameters brought by multiple layers in the training process, which can reduce the risk of overfitting and get better performance for deep SNNs (DSNNs). We use the discrete leaky-integrate-fire model and the spatiotemporal BP algorithm for training DSNNs directly. As a result, our model trained with the SPGP method achieves competitive performance compared to the existing DSNNs on image and neuromorphic data sets while minimizing the number of trainable parameters. In addition, the proposed SPGP method shows its effectiveness in performance improvement, convergence, and generalization ability.

Dataset complexity assessment based on cumulative maximum scaled area under Laplacian spectrum

Dataset complexity assessment aims to predict classification performance on a dataset with complexity calculation before training a classifier, which can also be used for classifier selection and dataset reduction. The training process of deep convolutional neural networks (DCNNs) is iterative and time-consuming because of hyperparameter uncertainty and the domain shift introduced by different datasets. Hence, it is meaningful to predict classification performance by assessing the complexity of datasets effectively before training DCNN models. This paper proposes a novel method called cumulative maximum scaled Area Under Laplacian Spectrum (cmsAULS), which can achieve state-of-the-art complexity assessment performance on six datasets.

Multi-classification deep CNN model for diagnosing COVID-19 using iterative neighborhood component analysis and iterative ReliefF feature selection techniques with X-ray images

BackgroundThe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) disease seriously affected worldwide health. It remains an important worldwide concern as the number of patients infected with this virus and the death rate is increasing rapidly. Early diagnosis is very important to hinder the spread of the coronavirus. Therefore, this article is intended to facilitate radiologists automatically determine COVID-19 early on X-ray images. Iterative Neighborhood Component Analysis (INCA) and Iterative ReliefF (IRF) feature selection methods are applied to increase the accuracy of the performance criteria of trained deep Convolutional Neural Networks (CNN). Materials and methodsThe COVID-19 dataset consists of a total of 15153 X-ray images for 4961 patient cases. The work includes thirteen different deep CNN model architectures. Normalized data of lung X-ray image for each deep CNN mesh model are analyzed to classify disease status in the category of Normal, Viral Pneumonia and COVID-19. The performance criteria are improved by applying the INCA and IRF feature selection methods to the trained CNN in order to improve the analysis, forecasting results, make a faster and more accurate decision. ResultsThirteen different deep CNN experiments and evaluations are successfully performed based on 80-20% of lung X-ray images for training and testing, respectively. The highest predictive values are seen in the analysis using INCA feature selection in the VGG16 network. The means of performance criteria obtained using the accuracy, sensitivity, F-score, precision, MCC, dice, Jaccard, and specificity are 99.14%, 97.98%, 99.58%, 98.80%, 97.81%, 98.83%, 97.68%, and 99.56%, respectively. This proposed study is indicated the useful application of deep CNN models to classify COVID-19 in X-ray images.

Semantic Segmentation Using Pixel-Wise Adaptive Label Smoothing via Self-Knowledge Distillation for Limited Labeling Data.

To achieve high performance, most deep convolutional neural networks (DCNNs) require a significant amount of training data with ground truth labels. However, creating ground-truth labels for semantic segmentation requires more time, human effort, and cost compared with other tasks such as classification and object detection, because the ground-truth label of every pixel in an image is required. Hence, it is practically demanding to train DCNNs using a limited amount of training data for semantic segmentation. Generally, training DCNNs using a limited amount of data is problematic as it easily results in a decrease in the accuracy of the networks because of overfitting to the training data. Here, we propose a new regularization method called pixel-wise adaptive label smoothing (PALS) via self-knowledge distillation to stably train semantic segmentation networks in a practical situation, in which only a limited amount of training data is available. To mitigate the problem caused by limited training data, our method fully utilizes the internal statistics of pixels within an input image. Consequently, the proposed method generates a pixel-wise aggregated probability distribution using a similarity matrix that encodes the affinities between all pairs of pixels. To further increase the accuracy, we add one-hot encoded distributions with ground-truth labels to these aggregated distributions, and obtain our final soft labels. We demonstrate the effectiveness of our method for the Cityscapes dataset and the Pascal VOC2012 dataset using limited amounts of training data, such as 10%, 30%, 50%, and 100%. Based on various quantitative and qualitative comparisons, our method demonstrates more accurate results compared with previous methods. Specifically, for the Cityscapes test set, our method achieved mIoU improvements of 0.076%, 1.848%, 1.137%, and 1.063% for 10%, 30%, 50%, and 100% training data, respectively, compared with the method of the cross-entropy loss using one-hot encoding with ground truth labels.

Personalized Smart Clothing Design Based on Multimodal Visual Data Detection.

In the traditional clothing customization system, only the designer participates in the clothing design, and the style is single. In the face of numerous styles, the user just repeatedly arranges and combines the styles, but does not realize the user's innovative design. In this paper, we propose a novel multitask deep convolutional neural network training method for task-by-task transfer learning, and learn deep image features for image retrieval tasks on noisy user click data. Image retrieval model based on image-text multimodal correlation features: this paper uses image-text multimodal correlation features to calculate the correlation between query keywords and images, and calculates the correlation between images and images. In this paper, the method of automatic generation of clothing style is researched, and parameterized coding is designed for it. Taking the typical style of suits as the initial population, through the human-computer interaction interface, the scoring value is assigned to the fitness value to carry out the evolution process. The binary string of the suit style generated by the genetic algorithm is decoded through the decoding algorithm and rules, decoded into a visual style diagram of the style of the suit style, and the style diagram of the suit style is automatically drawn. From the perspective of clothing design, this paper summarizes the general design methods of outdoor sports smart clothing, follows the integration of people-clothing-environment, and proposes a wearer-centered design concept to deeply explore the rationality of outdoor sports smart clothing design methods. This paper further solves the key problems in the design process, and takes the fashion of clothing as the core principle to design the structure of clothing modeling. The fabric selection of suits is based on the principle of clothing comfort. The key is to realize the outdoor sports monitoring function of suits through sensing technology. The design uses Arduino as an electronic prototype platform, so as to detect the heart rate of the human body during exercise and the microclimate temperature under the clothes. This kind of suit with monitoring function is ultimately a combination of sensing device and clothing. It not only has monitoring function, but also has the aesthetic concept of clothing design and conforms to the performance of human body structure, which will provide reference and reference for the design of outdoor sports smart clothing.

Deep learning for proximal soil sensor development towards smart irrigation

Excessive agricultural water consumption threatens safe access of billions of people to potable water. Smart irrigation systems offer more efficient water use in irrigated agriculture. Determining irrigation requirements of various soil texture classes in production fields requires more sophisticated sensing technologies such as deep learning. This study has proposed a proximal sensing system by means of using a color camera towards smart irrigation based on computer vision and deep learning to identify water requirements of three soil texture classes under different illumination conditions. An imaging station was composed to reduce the workload in obtaining training images required for training deep convolutional neural network models. Five deep learning architectures were employed to identify texture-water classes: AlexNet, GoogleNet, ResNet, VGG16, and SqueezeNet. Those models were experimented with and investigated to determine the best models in terms of detection performance and speed. By using cross-validation rules, approximately 12,214 images were studied individually for the purpose of training and testing. The AlexNet model outperformed the other deep learning models with an F1-score of 0.9973 in identifying twelve soil texture-water classes. GoogleNet and ResNet displayed the fastest detection speeds with an average processing time of 16.92 ms. The findings obtained from this study have indicated that deep learning bears a great potential in determining irrigation requirements of production fields under varying conditions

Use of deep learning in the MRI diagnosis of Chiari malformation type I

PurposeTo train deep learning convolutional neural network (CNN) models for classification of clinically significant Chiari malformation type I (CM1) on MRI to assist clinicians in diagnosis and decision making.MethodsA retrospective MRI dataset of patients diagnosed with CM1 and healthy individuals with normal brain MRIs from the period January 2010 to May 2020 was used to train ResNet50 and VGG19 CNN models to automatically classify images as CM1 or normal. A total of 101 patients diagnosed with CM1 requiring surgery and 111 patients with normal brain MRIs were included (median age 30 with an interquartile range of 23–43; 81 women with CM1). Isotropic volume transformation, image cropping, skull stripping, and data augmentation were employed to optimize model accuracy. K-fold cross validation was used to calculate sensitivity, specificity, and the area under receiver operating characteristic curve (AUC) for model evaluation.ResultsThe VGG19 model with data augmentation achieved a sensitivity of 97.1% and a specificity of 97.4% with an AUC of 0.99. The ResNet50 model achieved a sensitivity of 94.0% and a specificity of 94.4% with an AUC of 0.98.ConclusionsVGG19 and ResNet50 CNN models can be trained to automatically detect clinically significant CM1 on MRI with a high sensitivity and specificity. These models have the potential to be developed into clinical support tools in diagnosing CM1.

Efficient-Grad: Efficient Training Deep Convolutional Neural Networks on Edge Devices with Grad ient Optimizations

With the prospering of mobile devices, the distributed learning approach, enabling model training with decentralized data, has attracted great interest from researchers. However, the lack of training capability for edge devices significantly limits the energy efficiency of distributed learning in real life. This article describes Efficient-Grad, an algorithm-hardware co-design approach for training deep convolutional neural networks, which improves both throughput and energy saving during model training, with negligible validation accuracy loss. The key to Efficient-Grad is its exploitation of two observations. Firstly, the sparsity has potential for not only activation and weight, but gradients and the asymmetry residing in the gradients for the conventional back propagation (BP). Secondly, a dedicated hardware architecture for sparsity utilization and efficient data movement can be optimized to support the Efficient-Grad algorithm in a scalable manner. To the best of our knowledge, Efficient-Grad is the first approach that successfully adopts a feedback-alignment (FA)-based gradient optimization scheme for deep convolutional neural network training, which leads to its superiority in terms of energy efficiency. We present case studies to demonstrate that the Efficient-Grad design outperforms the prior arts by 3.72x in terms of energy efficiency.

Data augmentation by a CycleGAN-based extra-supervised model for nondestructive testing

The deep learning method is widely used in computer vision tasks with large-scale annotated datasets. However, obtaining such datasets in most directions of the vision based nondestructive testing (NDT) field is very challenging. Data augmentation is proved as an efficient way of dealing with the lack of large-scale annotated datasets. In this paper, we propose a CycleGAN-based extra-supervised (CycleGAN-ES) model to generate synthetic NDT images, where the ES is used to ensure that the bidirectional mapping is learned for corresponding labels and defects. Furthermore, we show the effectiveness of using the synthesized images to train deep convolutional neural networks (DCNNs) for defect recognition. In the experiments, we extract a number of x-ray welding images with both defect and no defects from the published GDXray dataset, and CycleGAN-ES is used to generate the synthetic defect images based on a small number of extracted defect images and manually drawn labels that are used as a content guide. For quality verification of the synthesized defect images, we use a high-performance classifier pretrained using a big dataset to recognize the synthetic defects and show the comparability of the performances of classifiers trained using synthetic defects and real defects, respectively. To present the effectiveness of using the synthesized defects as an augmentation method, we train and evaluate the performances of DCNN for defect recognition with or without the synthesized defects.

Study of the Few-Shot Learning for ECG Classification Based on the PTB-XL Dataset.

The electrocardiogram (ECG) is considered a fundamental of cardiology. The ECG consists of P, QRS, and T waves. Information provided from the signal based on the intervals and amplitudes of these waves is associated with various heart diseases. The first step in isolating the features of an ECG begins with the accurate detection of the R-peaks in the QRS complex. The database was based on the PTB-XL database, and the signals from Lead I–XII were analyzed. This research focuses on determining the Few-Shot Learning (FSL) applicability for ECG signal proximity-based classification. The study was conducted by training Deep Convolutional Neural Networks to recognize 2, 5, and 20 different heart disease classes. The results of the FSL network were compared with the evaluation score of the neural network performing softmax-based classification. The neural network proposed for this task interprets a set of QRS complexes extracted from ECG signals. The FSL network proved to have higher accuracy in classifying healthy/sick patients ranging from 93.2% to 89.2% than the softmax-based classification network, which achieved 90.5–89.2% accuracy. The proposed network also achieved better results in classifying five different disease classes than softmax-based counterparts with an accuracy of 80.2–77.9% as opposed to 77.1% to 75.1%. In addition, the method of R-peaks labeling and QRS complexes extraction has been implemented. This procedure converts a 12-lead signal into a set of R waves by using the detection algorithms and the k-mean algorithm.

Real-Time Hand Gesture Recognition Using Fine-Tuned Convolutional Neural Network.

Hand gesture recognition is one of the most effective modes of interaction between humans and computers due to being highly flexible and user-friendly. A real-time hand gesture recognition system should aim to develop a user-independent interface with high recognition performance. Nowadays, convolutional neural networks (CNNs) show high recognition rates in image classification problems. Due to the unavailability of large labeled image samples in static hand gesture images, it is a challenging task to train deep CNN networks such as AlexNet, VGG-16 and ResNet from scratch. Therefore, inspired by CNN performance, an end-to-end fine-tuning method of a pre-trained CNN model with score-level fusion technique is proposed here to recognize hand gestures in a dataset with a low number of gesture images. The effectiveness of the proposed technique is evaluated using leave-one-subject-out cross-validation (LOO CV) and regular CV tests on two benchmark datasets. A real-time American sign language (ASL) recognition system is developed and tested using the proposed technique.