Data Enhancement Methods Research Articles

Atrous Pyramid GAN Segmentation Network for Fish Images with High Performance

With the development of computer science technology, theory and method of image segmentation are widely used in fish discrimination, which plays an important role in improving the efficiency of fisheries sorting and biodiversity studying. However, the existing methods of fish images segmentation are less accurate and inefficient, which is worthy of in-depth exploration. Therefore, this paper proposes an atrous pyramid GAN segmentation network aimed at increasing accuracy and efficiency. This paper introduces an atrous pyramid structure, and the GAN module is added before the CNN backbone in order to augment the dataset. The Atrous pyramid structure first fuses the input and output of the dilated convolutional layer with a small sampling rate and then feeds the fused features into the subsequent dilated convolutional layer with a large sampling rate to obtain dense multiscale contextual information. Thus, by capturing richer contextual information, this structure improves the accuracy of segmentation results. In addition to the aforementioned innovation, various data enhancement methods, such as MixUp, Mosaic, CutMix, and CutOut, are used in this paper to enhance the model’s robustness. This paper also improves the loss function and uses the label smoothing method to prevent model overfitting. The improvement is also tested by extensive ablation experiments. As a result, our model’s F1-score, GA, and MIoU were tested on the validation dataset, reaching 0.961, 0.981, and 0.973, respectively. This experimental result demonstrates that the proposed model outperforms all the other contrast models. Moreover, in order to accelerate the deployment of the encapsulated model on hardware, this paper optimizes the execution time of the matrix multiplication method on Hbird E203 based on Strassen’s algorithm to ensure the efficient operation of the model on this hardware platform.

A deep learning radiomics analysis for identifying sinus invasion in patients with meningioma before operation using tumor and peritumoral regions

BackgroundFor patients with meningioma, surgical procedures are different because of the status of sinus invasion. However, there is still no suitable technique to identify the status of sinus invasion in patients with meningiomas. We aimed to build a deep learning radiomics model to identify sinus invasion before surgery. MethodsA total of 1048 patients with meningiomas were retrospectively enrolled from two hospitals. T1 enhanced-weighted (T1c) and T2-weighted MRI data for each patient were collected. Tumors and their corresponding peritumors were analyzed. Four ResNet50 models were built with different types of regions of interest (ROIs) (tumor and peritumor) and different modal images (T1c and T2) to predict the status of sinus invasion. Several data enhancement methods were applied before ResNet50 model building. The final model was generated by combining four ResNet50 models. ResultsThe models with a combination of tumors and peritumors using multimodal images achieved the highest predictive performance (AUC = 0.884, ACC = 78.1%) in the independent test cohort. The Delong test proved that the model built with combination ROIs achieved significantly higher performance than the model built only with tumors. The net reclassification improvement and integrated discrimination improvement tests both proved that including peritumor ROIs in the tumor ROIs could significantly improve the prediction ability. ConclusionIn the current study, the deep learning model showed potential for identifying sinus invasion before surgery in patients with meningioma. Including peritumors could significantly improve predictive performance.

Research on remote sensing image extraction based on deep learning.

Remote sensing technology has the advantages of fast information acquisition, short cycle, and a wide detection range. It is frequently used in surface resource monitoring tasks. However, traditional remote sensing image segmentation technology cannot make full use of the rich spatial information of the image, the workload is too large, and the accuracy is not high enough. To address these problems, this study carried out atmospheric calibration, band combination, image fusion, and other data enhancement methods for Landsat 8 satellite remote sensing data to improve the data quality. In addition, deep learning is applied to remote-sensing image block segmentation. An asymmetric convolution-CBAM (AC-CBAM) module based on the convolutional block attention module is proposed. This optimization module of the integrated attention and sliding window prediction method is adopted to effectively improve the segmentation accuracy. In the experiment of test data, the mIoU, mAcc, and aAcc in this study reached 97.34%, 98.66%, and 98.67%, respectively, which is 1.44% higher than that of DNLNet (95.9%). The AC-CBAM module of this research provides a reference for deep learning to realize the automation of remote sensing land information extraction. The experimental code of our AC-CBAM module can be found at https://github.com/LinB203/remotesense.

Defect object detection algorithm for electroluminescence image defects of photovoltaic modules based on deep learning

AbstractVisual inspection of photovoltaic modules using electroluminescence (EL) images is a common method of quality inspection. Because human inspection requires a lot of time, object detection algorithm to replace human inspection is a popular research direction in recent years. To solve the problem of low accuracy and slow speed in EL image detection, we propose a YOLO‐based object detection algorithm YOLO‐PV, which achieves 94.55% of AP (average precision) on the photovoltaic module EL image data set, and the interference speed exceeds 35 fps. The improvement of speed and accuracy benefits from the targeted design of the network architecture according to the characteristics of EL image. First, we weaken the backbone's ability to extract deep‐level information so that it can focus on extracting the low‐level defect information. Second, the PAN network is used for feature fusion in the Neck part. But, only the single‐size feature map output is retained, which significantly reduces the amount of calculation. Also, we analyze the impact of data enhancement methods on model overfitting and performance. Finally, we give effective data enhancement methods. The results show that the object detection algorithm in this paper can meet the requirements for high‐precision and real‐time processing on the PV module production line.

FORMING THE REQUIREMENTS FOR EMOTION DETECTION METHODS

In the technological age, emotion detection has gradually become a research hotspot and one of the most important fields in human-computer interaction. Different proposed technologies of recent years on emotion detection have been summarized and compared. The relatively often used emotion types are presented, including Paul Ekman's, Parrot’s, Circumplex model and Plutchik’s Wheel. Speech, Text, Facial Expressions, and Electroencephalogram were discussed as sources of emotions. 2 main sub-branches of emotion detection are discussed: unimodal and multimodal, which use the above sources. Each of these sources has its own advantages and disadvantages, and depending on the problem, the use of one may be more effective than that of the other. As a result, several important problems are highlighted, such as real-time data processing without external servers and devices, resource management, model creation for non-popular languages, etc. For model training a variety of datasets are used, some authors have even combined several for higher accuracy. Data enhancement methods were used by all authors in order to reduce noise at the preprocessing stage. Although some sources claim that better results can be obtained with the unimodal method, the multimodal method, if properly processed, results in a higher accuracy system.

Improving the Transferability of Adversarial Examples With a Noise Data Enhancement Framework and Random Erasing.

Deep neural networks (DNNs) are proven vulnerable to attack against adversarial examples. Black-box transfer attacks pose a massive threat to AI applications without accessing target models. At present, the most effective black-box attack methods mainly adopt data enhancement methods, such as input transformation. Previous data enhancement frameworks only work on input transformations that satisfy accuracy or loss invariance. However, it does not work for other transformations that do not meet the above conditions, such as the transformation which will lose information. To solve this problem, we propose a new noise data enhancement framework (NDEF), which only transforms adversarial perturbation to avoid the above issues effectively. In addition, we introduce random erasing under this framework to prevent the over-fitting of adversarial examples. Experimental results show that the black-box attack success rate of our method Random Erasing Iterative Fast Gradient Sign Method (REI-FGSM) is 4.2% higher than DI-FGSM in six models on average and 6.6% higher than DI-FGSM in three defense models. REI-FGSM can combine with other methods to achieve excellent performance. The attack performance of SI-FGSM can be improved by 22.9% on average when combined with REI-FGSM. Besides, our combined version with DI-TI-MI-FGSM, i.e., DI-TI-MI-REI-FGSM can achieve an average attack success rate of 97.0% against three ensemble adversarial training models, which is greater than the current gradient iterative attack method. We also introduce Gaussian blur to prove the compatibility of our framework.

Segmentation of infected region in CT images of COVID-19 patients based on QC-HC U-net

Since the outbreak of COVID-19 in 2019, the rapid spread of the epidemic has brought huge challenges to medical institutions. If the pathological region in the COVID-19 CT image can be automatically segmented, it will help doctors quickly determine the patient’s infection, thereby speeding up the diagnosis process. To be able to automatically segment the infected area, we proposed a new network structure and named QC-HC U-Net. First, we combine residual connection and dense connection to form a new connection method and apply it to the encoder and the decoder. Second, we choose to add Hypercolumns in the decoder section. Compared with the benchmark 3D U-Net, the improved network can effectively avoid vanishing gradient while extracting more features. To improve the situation of insufficient data, resampling and data enhancement methods are selected in this paper to expand the datasets. We used 63 cases of MSD lung tumor data for training and testing, continuously verified to ensure the training effect of this model, and then selected 20 cases of public COVID-19 data for training and testing. Experimental results showed that in the segmentation of COVID-19, the specificity and sensitivity were 85.3% and 83.6%, respectively, and in the segmentation of MSD lung tumors, the specificity and sensitivity were 81.45% and 80.93%, respectively, without any fitting.

DBA_SSD: A Novel End-to-End Object Detection Algorithm Applied to Plant Disease Detection

In response to the difficulty of plant leaf disease detection and classification, this study proposes a novel plant leaf disease detection method called deep block attention SSD (DBA_SSD) for disease identification and disease degree classification of plant leaves. We propose three plant leaf detection methods, namely, squeeze-and-excitation SSD (Se_SSD), deep block SSD (DB_SSD), and DBA_SSD. Se_SSD fuses SSD feature extraction network and attention mechanism channel, DB_SSD improves VGG feature extraction network, and DBA_SSD fuses the improved VGG network and channel attention mechanism. To reduce the training time and accelerate the training process, the convolutional layers trained in the Image Net image dataset by the VGG model are migrated to this model, whereas the collected plant leaves disease image dataset is randomly divided into training set, validation set, and test set in the ratio of 8:1:1. We chose the PlantVillage dataset after careful consideration because it contains images related to the domain of interest. This dataset consists of images of 14 plants, including images of apples, tomatoes, strawberries, peppers, and potatoes, as well as the leaves of other plants. In addition, data enhancement methods, such as histogram equalization and horizontal flip were used to expand the image data. The performance of the three improved algorithms is compared and analyzed in the same environment and with the classical target detection algorithms YOLOv4, YOLOv3, Faster RCNN, and YOLOv4 tiny. Experiments show that DBA_SSD outperforms the two other improved algorithms, and its performance in comparative analysis is superior to other target detection algorithms.

Multi-modal Broad Learning System for Medical Image and Text-based Classification.

Automatic classification of medical images plays an essential role in computer-aided diagnosis. However, the medical images arise from the small number of available data and the improvement of existing data-enhancement methods are limited. In order to fulfil this demand, a Multi-Modal Broad Learning System (M2-BLS) is proposed, which has two subnetworks for simultaneous learning of both medical images and the corresponding radiology reports. M2-BLS provides two advantages: i) our M2-BLS has closed-form solution and avoids iterative training, once the image feature is available; ii) benefit from the simultaneous learning of both image and text data, our M2-BLS achieves high accuracy for medical classification. Experimental results on the publicly available datasets IU X-RAY and PEIR GROSS_895 show that our M2-BLS highly improves the classification performance, compared to SOTA deep models that learn single-type of data information only.

Synthetic region screening and adaptive feature fusion for constructing a flexible object detection database

A large amount of training data (including samples and labels) are required to achieve the task of object detection. However, the construction of databases generally has several problems. The samples are not balanced. In some scenarios, very few or even no samples can be collected; manual collection and labeling are expensive. Obtaining samples from public object detection data sets is simple, but the categories that can be obtained are limited and cannot meet the needs of other detection tasks. Therefore, we propose a method to automatically construct a flexible object detection database and complete automatic labeling. Our main advantage lies in the proposed synthetic region screening and adaptive feature fusion algorithm, which generates sample images and corresponding annotation files that are similar to the real collection. Our algorithm has a stronger and more realistic synthesis ability for three-dimensional objects than other data synthesis methods. Our method improves the system’s object detection accuracy in the following ways: (1) it synthesizes a wider range of images for certain categories with few samples in the data set to solve the problem of sample imbalance; (2) it adds new categories to the public data set to meet the needs of rapid deployment of special object detection tasks; and (3) it fuses different foregrounds and backgrounds to enrich the diversity of database samples. To effectively evaluate the proposed method, we conducted experiments separately, and the experimental results proved that our method is superior to other existing data synthesis and data enhancement methods. Combining the generated samples with the real data set, the object detection accuracy value increased from 9.47% to 58.92%. Extending the objects of the self-collected data set to the public data set and automatically generating high-quality annotations, the detection accuracy of extended objects reached more than 50%.

Surface Detection of Solid Wood Defects Based on SSD Improved with ResNet

Due to the lack of forest resources in China and the low detection efficiency of wood surface defects, the output of solid wood panels is not high. Therefore, this paper proposes a method for detecting surface defects of solid wood panels based on a Single Shot MultiBox Detector algorithm (SSD) to detect typical wood surface defects. The wood panel images are acquired by an independently designed image acquisition system. The SSD model included the first five layers of the VGG16 network, the SSD feature mapping layer, the feature detection layer, and the Non-Maximum Suppression (NMS) module. We used TensorFlow to train the network and further improved it on the basis of the SSD network structure. As the basic network part of the improved SSD model, the deep residual network (ResNet) replaced the VGG network part of the original SSD network to optimize the input features of the regression and classification tasks of the predicted bounding box. The solid wood panels selected in this paper are Chinese fir and pine. The defects include live knots, dead knots, decay, mildew, cracks, and pinholes. A total of more than 5000 samples were collected, and the data set was expanded to 100,000 through data enhancement methods. After using the improved SSD model, the average detection accuracy of the defects we obtained was 89.7%, and the average detection time was 90 ms. Both the detection accuracy and the detection speed were improved.

Rock thin sections identification based on improved squeeze-and-Excitation Networks model

Rock thin section recognition provides geological information, which is crucial in petroleum geology, exploration, and mining research as a kind of fundamental work. Although many machine learning methods solve this research, there are still problems of data hierarchy and model pertinence. We use the hierarchical classification method to divide the dataset into sedimentary, metamorphic, and igneous rock as first-level, and to subdivide a total of 105 s-level further from the three categories. We propose the MaSE-ResNeXt model based on the fundamental work in the SeNet that can enhance the feature connection between different channels. The MaSE-ResNeXt adopted hierarchical filter groups, bottleneck stacking, and other strategies to enhance the representational capability of the model which advances the solving ability in rock recognition. Six data enhancement methods and other techniques are used to improve the robustness and effectiveness. The accuracy in the test set was 90.89% and 81.97% for the first and second level, respectively, with the inference duration is only 0.0357s. This study also designs a degeneration experiment and model comparison to demonstrate the model's effectiveness. Future research can employ this model as the fundamental base of transfer learning in geology to save time in the training of study.

Research on the classification of lymphoma pathological images based on deep residual neural network.

BACKGROUND: Malignant lymphoma is a type of tumor that originated from the lymphohematopoietic system, with complex etiology, diverse pathological morphology, and classification. It takes a lot of time and energy for doctors to accurately determine the type of lymphoma by observing pathological images.OBJECTIVE: At present, an automatic classification technology is urgently needed to assist doctors in analyzing the type of lymphoma.METHODS: In this paper, by comparing the training results of the BP neural network and BP neural network optimized by genetic algorithm (GA-BP), adopts a deep residual neural network model (ResNet-50), with 374 lymphoma pathology images as the experimental data set. After preprocessing the dataset by image flipping, color transformation, and other data enhancement methods, the data set is input into the ResNet-50 network model, and finally classified by the softmax layer.RESULTS: The training results showed that the classification accuracy was 98.63%. By comparing the classification effect of GA-BP and BP neural network, the accuracy of the network model proposed in this paper is improved.CONCLUSIONS: The network model can provide an objective basis for doctors to diagnose lymphoma types.

Apple Disease Recognition Based on Small-scale Data Sets

HighlightsThe proposed data enhancement method can be used for small-scale data sets with rich sample image features.The accuracy of the new model reaches 98.5%, which is better than the traditional CNN method.Abstract: GoogLeNet offers far better performance in identifying apple disease compared to traditional methods. However, the complexity of GoogLeNet is relatively high. For small volumes of data, GoogLeNet does not achieve the same performance as it does with large-scale data. We propose a new apple disease identification model using GoogLeNet’s inception module. The model adopts a variety of methods to optimize its generalization ability. First, geometric transformation and image modification of data enhancement methods (including rotation, scaling, noise interference, random elimination, color space enhancement) and random probability and appropriate combination of strategies are used to amplify the data set. Second, we employ a deep convolution generative adversarial network (DCGAN) to enhance the richness of generated images by increasing the diversity of the noise distribution of the generator. Finally, we optimize the GoogLeNet model structure to reduce model complexity and model parameters, making it more suitable for identifying apple tree diseases. The experimental results show that our approach quickly detects and classifies apple diseases including rust, spotted leaf disease, and anthrax. It outperforms the original GoogLeNet in recognition accuracy and model size, with identification accuracy reaching 98.5%, making it a feasible method for apple disease classification. Keywords: Apple disease identification, Data enhancement, DCGAN, GoogLeNet.

Deep Learning Based Classification of Radar Spectral Maps

This paper discusses the use of deep convolutional neural networks for radar target classification. In this paper, three parts of the work are carried out: firstly, effective data enhancement methods are used to augment the dataset and address unbalanced datasets. Second, using deep learning techniques, we explore an effective framework for classifying and identifying targets based on radar spectral map data. By using data enhancement and the framework, we achieved an overall classification accuracy of 0.946. In the end, we researched the automatic annotation of image ROI (region of interest). By adjusting the model, we obtained a 93% accuracy in automatic labeling and classification of targets for both car and cyclist categories.

Target Recognition of SAR Image Based on CN-GAN and CNN in Complex Environment

In recent years, with the rapid development of deep learning, the research of radar image automatic target recognition (ATR) has made great progress. However, because of the complex environments and special imaging principles, Synthetic Aperture Radar (SAR) image still have the problems of sample scarcity and strong speckle noise, which affects the target recognition performance. To solve the above problems, we proposed a target recognition method of SAR image based on Constrained Naive Generative Adversarial Networks (CN-GAN) and Convolutional Neural Network (CNN). Combining Least Squares Generative Adversarial Networks (LSGAN) and Image-to-Image Translation (Pix2Pix), CN-GAN can overcome these problems of low Signal-to-Clutter-Noise Ratio (SCNR), model instability and the excessive freedom degree of the output, which are produced by conventional naive GAN. Besides, we adopted a shallow network structure design in CNN, which can effectively improve the generalization ability of the model and avoid the problem of model overfitting. The experimental results in this paper demonstrate that CN-GAN has achieved the data generation and data enhancement, the SCNR of generated data is higher than the origin data set and data sets gained by other forms of GANs, the recognition performance based on the extended data set is better than the origin data set, and the recognition rate of data set enhanced by CN-GAN is higher than that of other common data enhancement methods.

Development of Frequency-Mixed Point-Focusing Shear Horizontal Guided-Wave EMAT for Defect Inspection Using Deep Neural Network

We propose a new frequency-mixed point-focusing shear horizontal (SH) guided-wave electromagnetic acoustic transducer (EMAT) in this work to obtain the defect positions and plate thickness simultaneously and accurately. Compared with other guided-wave detection methods, it is not required to measure the plate thickness in advance because we can easily obtain it during the test. We use the variational mode decomposition method to decompose the received frequency-mixed defect signal into subsignals with different center frequencies and to remove the noise. Furthermore, we use the continuous wavelet transform to analyze these subsignals using the time-frequency method and to obtain the time-of-flight information of the guided wave under different frequencies and modes. Therefore, we can obtain accurate defect positions and plate thicknesses via the new transducer and signal processing methods while improving the signal intensities. In the identification of defect types, we first constructed a database set containing three types of defects of different sizes using data enhancement methods. Then, the dense network, convolutional neural network, recurrent neural network, and newly proposed deep GFresNet are studied to analyze the defect classification performance of each structure. The results show that the proposed GFresNet has very good defect identification accuracy, which is about 95% along any depth of the defects, and that it can automatically extract high-level information without sophisticated feature engineering.

Wavelet Feature Outdoor Fingerprint Localization Based on ResNet and Deep Convolution GAN

Due to the explosive development of location-based services (LBS), localization has attracted significant research attention over the past decade. Among the associated techniques, wireless fingerprint positioning has garnered much interest due to its compatibility with existing hardware. At present, with the widespread deployment of long-term evolution (LTE) networks and the uniqueness of wireless information fingerprints, fingerprint positioning based on LTE networks is the mainstream method for outdoor positioning. However, in order to improve its accuracy, this method needs to collect enough data at a large number of reference points, which is a labor-intensive task. In this paper, experimental data are collected at different reference points and then converted into wavelet feature maps. Then, a Deep Convolutional Generative Adversarial Network (DCGAN) is leveraged to generate a symmetric fingerprint database. Localization is then carried out by the proposed Deep Residual Network (Resnet), which is capable of learning reliable features from a fingerprint image database. To further increase the robustness of the positioning system, a variety of data enhancement methods are used. Finally, we experimentally demonstrate that the generated symmetric fingerprint database and proposed Resnet reduce the manpower required for fingerprint database collection and improve the accuracy of the outdoor positioning system.

Outdoor positioning based on deep learning and wireless network fingerprint technology

Localization has attracted considerable attention in recent years because of the growth in location-based services and computer-aided technology. In particular, wireless fingerprint-based positioning techniques have received considerable attention due to their satisfactory positioning performance with smartphones. In this work, a two-level hierarchical structure positioning system is proposed to achieve satisfactory positioning accuracy. First, long-time-evolution (LTE) mobile networks are used to provide positioning information, and the collected measurements of LTE signals are converted into grayscale fingerprint images to construct the fingerprint database. To overcome the instability of LTE signals, several data enhancement methods are leveraged to increase the diversity of the fingerprint database. Second, a modified deep residual network (MResNet) coarse localizer is used to learn reliable features from the fingerprint image database. Then, inspired by the ideas of feature fusion and transfer learning, a multilayer perceptron (MLP)-based fine localizer is used to further learn the features of the fingerprint images and achieve a better positioning performance. Our experimental results convincingly reveal that the proposed positioning system can achieve satisfactory positioning performance in a variety of outdoor environments.

Breast cancer pathological image classification based on deep learning.

The automatic classification of breast cancer pathological images has important clinical application value. However, to develop the classification algorithm using the artificially extracted image features faces several challenges including the requirement of professional domain knowledge to extract and compute highiquality image features, which are often time-consuming, laborious, and difficult. For overcoming these challenges, this study developed and applied an improved deep convolutional neural network model to perform automatic classification of breast cancer using pathological images. Specifically, in this study, data enhancement and migration learning methods are used to effectively avoid the overfitting problems with deep learning models when they are limited by training image sample size. Experimental results show that a 91% recognition rate or accuracy when applying this improved deep learning model to a publicly available dataset of BreaKHis. Comparing with other previously used models, the new model yields good robustness and generalization.