Deep Learning-based Convolutional Neural Network Research Articles

Breast lesions classifications of mammographic images using a deep convolutional neural network-based approach

Breast cancer is one of the worst illnesses, with a higher fatality rate among women globally. Breast cancer detection needs accurate mammography interpretation and analysis, which is challenging for radiologists owing to the intricate anatomy of the breast and low image quality. Advances in deep learning-based models have significantly improved breast lesions’ detection, localization, risk assessment, and categorization. This study proposes a novel deep learning-based convolutional neural network (ConvNet) that significantly reduces human error in diagnosing breast malignancy tissues. Our methodology is most effective in eliciting task-specific features, as feature learning is coupled with classification tasks to achieve higher performance in automatically classifying the suspicious regions in mammograms as benign and malignant. To evaluate the model’s validity, 322 raw mammogram images from Mammographic Image Analysis Society (MIAS) and 580 from Private datasets were obtained to extract in-depth features, the intensity of information, and the high likelihood of malignancy. Both datasets are magnificently improved through preprocessing, synthetic data augmentation, and transfer learning techniques to attain the distinctive combination of breast tumors. The experimental findings indicate that the proposed approach achieved remarkable training accuracy of 0.98, test accuracy of 0.97, high sensitivity of 0.99, and an AUC of 0.99 in classifying breast masses on mammograms. The developed model achieved promising performance that helps the clinician in the speedy computation of mammography, breast masses diagnosis, treatment planning, and follow-up of disease progression. Moreover, it has the immense potential over retrospective approaches in consistency feature extraction and precise lesions classification.

Deep learning-based convolutional neural network for intramodality brain MRI synthesis.

PurposeThe existence of multicontrast magnetic resonance (MR) images increases the level of clinical information available for the diagnosis and treatment of brain cancer patients. However, acquiring the complete set of multicontrast MR images is not always practically feasible. In this study, we developed a state‐of‐the‐art deep learning convolutional neural network (CNN) for image‐to‐image translation across three standards MRI contrasts for the brain.MethodsBRATS’2018 MRI dataset of 477 patients clinically diagnosed with glioma brain cancer was used in this study, with each patient having T1‐weighted (T1), T2‐weighted (T2), and FLAIR contrasts. It was randomly split into 64%, 16%, and 20% as training, validation, and test set, respectively. We developed a U‐Net model to learn the nonlinear mapping of a source image contrast to a target image contrast across three MRI contrasts. The model was trained and validated with 2D paired MR images using a mean‐squared error (MSE) cost function, Adam optimizer with 0.001 learning rate, and 120 epochs with a batch size of 32. The generated synthetic‐MR images were evaluated against the ground‐truth images by computing the MSE, mean absolute error (MAE), peak signal‐to‐noise ratio (PSNR), and structural similarity index (SSIM).ResultsThe generated synthetic‐MR images with our model were nearly indistinguishable from the real images on the testing dataset for all translations, except synthetic FLAIR images had slightly lower quality and exhibited loss of details. The range of average PSNR, MSE, MAE, and SSIM values over the six translations were 29.44–33.25 dB, 0.0005–0.0012, 0.0086–0.0149, and 0.932–0.946, respectively. Our results were as good as the best‐reported results by other deep learning models on BRATS datasets.ConclusionsOur U‐Net model exhibited that it can accurately perform image‐to‐image translation across brain MRI contrasts. It could hold great promise for clinical use for improved clinical decision‐making and better diagnosis of brain cancer patients due to the availability of multicontrast MRIs. This approach may be clinically relevant and setting a significant step to efficiently fill a gap of absent MR sequences without additional scanning.

Artificial Intelligence-Based Real-Time Pineapple Quality Classification Using Acoustic Spectroscopy

The pineapple is an essential fruit in Taiwan. Farmers separate pineapples into two types, according to the percentages of water in the pineapples. One is the “drum sound pineapple” and the other is the “meat sound pineapple”. As there is more water in the meat sound pineapple, the meat sound pineapple more easily rots and is more challenging to store than the drum sound pineapple. Thus, farmers need to filter out the meat sound pineapple, so that they can sell pineapples overseas. The classification, based on striking the pineapple fruit with rigid objects (e.g., plastic rulers) is most commonly used by farmers due to the negligibly low costs and availability. However, it is a time-consuming job, so we propose a method to automatically classify pineapples in this work. Using embedded onboard computing processors, servo, and an ultrasonic sensor, we built a hitting machine and combined it with a conveyor to automatically separate pineapples. To classify pineapples, we proposed a method related to acoustic spectrogram spectroscopy, which uses acoustic data to generate spectrograms. In the acoustic data collection step, we used the hitting machine mentioned before and collected many groups of data with different factors; some groups also included the noise in the farm. With these differences, we tested our deep learning-based convolutional neural network (CNN) performances. The best accuracy of the developed CNN model is 0.97 for data Group V. The proposed hitting machine and the CNN model can assist in the classification of pineapple fruits with high accuracy and time efficiency.

InSiNet: a deep convolutional approach to skin cancer detection and segmentation

Cancer is among the common causes of death around the world. Skin cancer is one of the most lethal types of cancer. Early diagnosis and treatment are vital in skin cancer. In addition to traditional methods, method such as deep learning is frequently used to diagnose and classify the disease. Expert experience plays a major role in diagnosing skin cancer. Therefore, for more reliable results in the diagnosis of skin lesions, deep learning algorithms can help in the correct diagnosis. In this study, we propose InSiNet, a deep learning-based convolutional neural network to detect benign and malignant lesions. The performance of the method is tested on International Skin Imaging Collaboration HAM10000 images (ISIC 2018), ISIC 2019, and ISIC 2020, under the same conditions. The computation time and accuracy comparison analysis was performed between the proposed algorithm and other machine learning techniques (GoogleNet, DenseNet-201, ResNet152V2, EfficientNetB0, RBF-support vector machine, logistic regression, and random forest). The results show that the developed InSiNet architecture outperforms the other methods achieving an accuracy of 94.59%, 91.89%, and 90.54% in ISIC 2018, 2019, and 2020 datasets, respectively. Since the deep learning algorithms eliminate the human factor during diagnosis, they can give reliable results in addition to traditional methods.

Fusion of SAR and optical images using pixel-based CNN

Sensors of different wavelengths in remote sensing field capture data. Each and every sensor has its own capabilities and limitations. Synthetic aperture radar (SAR) collects data that has a high spatial and radiometric resolution. The optical remote sensors capture images with good spectral information. Fused images from these sensors will have high information when implemented with a better algorithm resulting in the proper collection of data to predict weather forecasting, soil exploration, and crop classification. This work encompasses a fusion of optical and radar data of Sentinel series satellites using a deep learning-based convolutional neural network (CNN). The three-fold work of the image fusion approach is performed in CNN as layered architecture covering the image transform in the convolutional layer, followed by the activity level measurement in the max pooling layer. Finally, the decision-making is performed in the fully connected layer. The objective of the work is to show that the proposed deep learning-based CNN fusion approach overcomes some of the difficulties in the traditional image fusion approaches. To show the performance of the CNN-based image fusion, a good number of image quality assessment metrics are analyzed. The consequences demonstrate that the integration of spatial and spectral information is numerically evident in the output image and has high robustness. Finally, the objective assessment results outperform the state-of-the-art fusion methodologies.

Mobile Food Journalling Application with Convolutional Neural Network and Transfer Learning: A Case for Diabetes Management in Malaysia

Diabetes is an ever worsening problem in modern society, placing a heavy burden on healthcare systems. Due to the association between obesity and diabetes, food journaling mobile applications are an effective approach for managing and improving the outcome of diabetics. Due to the efficacy of nutritional tracking and management in managing diabetes, we implemented a deep learning-based Convolutional Neural Network food classification model to aid with food logging. The model is trained on a subset of the Food-101 and Malaysian Food 11 datasets, including web-scraped images, with a focus on food items found locally in Malaysia. In our experiments, we explore how fine-tuning of the image dataset improves the performance of the model.

Transferable Convolutional Neural Network for Weed Mapping With Multisensor Imagery

Automatic weed monitoring and classification are critical for effective site-specific weed management. With the increasing availability of different sensors, it is possible for weed management to be achieved by processing a wide range of images captured from various remote sensing platforms. A deep learning-based convolutional neural network (CNN) can learn the sophisticated spectral, spatial, and structural features to discriminate weed species. The challenge is to train a CNN architecture for each dataset with limited training samples. In this study, we develop a partial transferable CNN to cope with a new dataset with a different spatial resolution, a different number of bands, and variation in the signal-to-noise ratio. The goal is to make the training for each new dataset less demanding. We conducted a series of experiments on simulated image datasets from two sensors. This study reveals that the dropout layers between the convolutional layers have a significant impact for partial transferable CNN. Even-numbered subset layers from source CNN has a stronger impact on dealing with a task of different spatial resolution. For a different number of bands in source and target datasets, except for the first convolutional layer, the remaining layers are used for the analysis. Results show that network transfer is possible when the numbers of bands of the two datasets are not very different. For the variation in signal-to-noise ratio, it is found that the performance of transfer learning is acceptable when the noise level is not high. Based on these findings, experiments were conducted on two real datasets from two sensors, which includes all the variations. The comparison results using different state-of-the-art models show that partial CNN transfer with even-numbered layers provides better mapping accuracy for the target dataset with a limited number of training samples.

Semi-automatic quantitative analysis of the pelvic bony structures on apparent diffusion coefficient maps based on deep learning: establishment of reference ranges.

Apparent diffusion coefficient (ADC) maps provide quantitative information on both normal and abnormal tissues. However, it is difficult to distinguish between these tissues unless consistent and precise ADC values can be obtained from normal tissues. For this study we developed a deep learning-based convolutional neural network (CNN) for pelvic bony structure segmentation and established the reference ranges of ADC parameters for normal pelvic bony structures. We retrospectively enrolled 767 prostate cancer (PCa) patients for quantitative ADC analyses of normal pelvic bony structures. A subset of 288 patients who did not receive treatment for PCa (S1) were used to develop a CNN model for the segmentation of 8 pelvic bony structures (lumbar vertebra, sacrococcyx, ilium, acetabulum, femoral head, femoral neck, ischium, and pubis). The proposed CNN was used for the automated segmentation of these pelvic bony structures from a subset of 405 patients who did not receive treatment (S2) and 74 patients who received treatment [radiotherapy (S3) or endocrine therapy (S4)]. The 95% confidence interval (CI) was used to establish reference ranges for the ADC values from the normal pelvic bony structures of S1 and S2. The Dice scores (Sørensen-Dice coefficient) for the CNN segmentation of the 8 pelvic bones on the ADC maps ranged from 0.90±0.02 (ilium) to 0.95±0.03 (femoral head) in the S1 testing set. In the S2 data set, the Dice scores showed no significant difference among the different scanners (P>0.05), and no significant differences were found among the S2, S3, and S4 data sets. The correlation analysis revealed that the b value and field strength were significantly correlated with ADC values (all P<0.001), while age and treatment were not significant variables (all P>0.05). The ADC reference ranges (95% CI) were as follows: lumbar vertebra, 1.11 (0.90-1.54); sacrococcyx, 0.82 (0.61-1.15); ilium, 0.57 (0.45-0.62); acetabulum, 0.59 (0.40-0.69); femoral head, 0.46 (0.25-0.58); femoral neck, 0.43 (0.25-0.48); ischium, 0.45 (0.26-0.55); and pubis, 0.57 (0.45-0.65). This study preliminarily established reference ranges for the ADC values of normal pelvic bony structures. The image acquisition parameters had an influence on the ADC values.

Fusion of SAR and optical images using pixel-based CNN

Sensors of different wavelengths in remote sensing field capture data. Each and every sensor has its own capabilities and limitations. Synthetic aperture radar (SAR) collects data that has a high spatial and radiometric resolution. The optical remote sensors capture images with good spectral information. Fused images from these sensors will have high information when implemented with a better algorithm resulting in the proper collection of data to predict weather forecasting, soil exploration, and crop classification. This work encompasses a fusion of optical and radar data of Sentinel series satellites using a deep learning-based convolutional neural network (CNN). The three-fold work of the image fusion approach is performed in CNN as layered architecture covering the image transform in the convolutional layer, followed by the activity level measurement in the max pooling layer. Finally, the decision-making is performed in the fully connected layer. The objective of the work is to show that the proposed deep learning-based CNN fusion approach overcomes some of the difficulties in the traditional image fusion approaches. To show the performance of the CNN-based image fusion, a good number of image quality assessment metrics are analyzed. The consequences demonstrate that the integration of spatial and spectral information is numerically evident in the output image and has high robustness. Finally, the objective assessment results outperform the state-of-the-art fusion methodologies.

Convolutional Neural Network-Based Intelligent Decision-Making for Automated Vehicles

The decision-making module provides proper driving maneuvers for Automated vehicles (AVs), which is essential to the deployment of AVs. Current research works attempted to tackle this issue by optimizing some specific significant objectives or modeling separate typical maneuvers. However, highly dynamic and complicated traffic flows bring huge challenges to this module. This paper develops a convolutional neural network-based (CNN) deep learning framework to learn proper driving maneuvers from bulks of the collected traffic data. The proposed method firstly covers the dynamic traffic information through constructing virtual traffic images, namely dynamic motion images. Those images contain important motion relationships between the ego car and its neighbors. Then, the proposed CNN learning model is trained to capture inherent features from those motion images and to learn to make proper driving behaviors. We employ the learning framework to learn typical highway driving scenarios. Training and test results validate the effectiveness of our proposed CNN-based learning decision-making approach and suggest that deep-learning-based methods have dramatic potential to promote the intelligent driving capability of AVs and thus accelerate the launch of AVs.

Real‐world multimodal lifelog dataset for human behavior study

To understand the multilateral characteristics of human behavior and physiological markers related to physical, emotional, and environmental states, extensive lifelog data collection in a real-world environment is essential. Here, we propose a data collection method using multimodal mobile sensing and present a long-term dataset from 22 subjects and 616 days of experimental sessions. The dataset contains over 10 000 hours of data, including physiological, data such as photoplethysmography, electrodermal activity, and skin temperature in addition to the multivariate behavioral data. Furthermore, it consists of 10 372 user labels with emotional states and 590 days of sleep quality data. To demonstrate feasibility, human activity recognition was applied on the sensor data using a convolutional neural network-based deep learning model with 92.78% recognition accuracy. From the activity recognition result, we extracted the daily behavior pattern and discovered five representative models by applying spectral clustering. This demonstrates that the dataset contributed toward understanding human behavior using multimodal data accumulated throughout daily lives under natural conditions.

A Deep Convolutional Neural Network-Based Multi-Class Image Classification for Automatic Wafer Map Failure Recognition in Semiconductor Manufacturing

Wafer maps provide engineers with important information about the root causes of failures during the semiconductor manufacturing process. Through the efficient recognition of the wafer map failure pattern type, the semiconductor manufacturing process and its product performance can be improved, as well as reducing the product cost. Therefore, this paper proposes an accurate model for the automatic recognition of wafer map failure types using a deep learning-based convolutional neural network (DCNN). For this experiment, we use WM811K, which is an open-source real-time wafer map dataset containing wafer map images of nine failure classes. Our research contents can be briefly summarized as follows. First, we use random sampling to extract 500 images from each class of the original image dataset. Then we propose a deep convolutional neural network model to generate a multi-class classification model. Lastly, we evaluate the performance of the proposed prediction model and compare it with three other popular machine learning-based models—logistic regression, random forest, and gradient boosted decision trees—and several well-known deep learning models—VGGNet, ResNet, and EfficientNet. Consequently, the comprehensive analysis showed that the performance of the proposed DCNN model outperformed those of other popular machine learning and deep learning-based prediction models.

A Self-Activated CNN Approach for Multi-Class Chest-Related COVID-19 Detection

Chest diseases can be dangerous and deadly. They include many chest infections such as pneumonia, asthma, edema, and, lately, COVID-19. COVID-19 has many similar symptoms compared to pneumonia, such as breathing hardness and chest burden. However, it is a challenging task to differentiate COVID-19 from other chest diseases. Several related studies proposed a computer-aided COVID-19 detection system for the single-class COVID-19 detection, which may be misleading due to similar symptoms of other chest diseases. This paper proposes a framework for the detection of 15 types of chest diseases, including the COVID-19 disease, via a chest X-ray modality. Two-way classification is performed in proposed Framework. First, a deep learning-based convolutional neural network (CNN) architecture with a soft-max classifier is proposed. Second, transfer learning is applied using fully-connected layer of proposed CNN that extracted deep features. The deep features are fed to the classical Machine Learning (ML) classification methods. However, the proposed framework improves the accuracy for COVID-19 detection and increases the predictability rates for other chest diseases. The experimental results show that the proposed framework, when compared to other state-of-the-art models for diagnosing COVID-19 and other chest diseases, is more robust, and the results are promising.

Hybrid gated recurrent unit and convolutional neural network-based deep learning architecture-based visibility improvement scheme for improving fog-degraded images

Hybrid gated recurrent unit and convolutional neural network-based deep learning architecture-based visibility improvement scheme for improving fog-degraded images

Convolutional neural network-based deep transfer learning for fault detection of gas turbine combustion chambers

Gas turbine combustion chambers work in highly adverse environment and thus malfunction more easily compared to other components. Fault detection of gas turbine combustion chambers can significantly increase the safety and reliability. Current researches on data-driven fault detection require abundant historical data. However, for a new gas turbine that has just operated for a short time, the available historical data are quite few, and the available fault data are even fewer. To address this problem, this paper first proposes the concept of gas turbine group fault diagnosis. Deep transfer learning is introduced into the fault detection of gas turbine combustion chambers for the first time. Convolutional neural network is pretrained using the data from one data-rich gas turbine and the pretrained convolutional neural network is finetuned for fault detection of another data-poor gas turbine. Through deep transfer learning, fault knowledge is shared between one data-rich gas turbine and another data-poor gas turbine. Experiments in two actual gas turbines show the proposed method can obtain good detection performance. Further comparison experiments show that the proposed method can significantly improve the detection performance compared with using the data from data-poor gas turbine directly and mixing the data from data-rich gas turbine and data-poor gas turbine. Detailed visualization analysis is made to explain what common fault knowledge is shared. This can also explain why deep transfer learning and gas turbine group fault diagnosis are effective to some extent.

Accuracy Assessment in Convolutional Neural Network-Based Deep Learning Remote Sensing Studies—Part 2: Recommendations and Best Practices

Convolutional neural network (CNN)-based deep learning (DL) has a wide variety of applications in the geospatial and remote sensing (RS) sciences, and consequently has been a focus of many recent studies. However, a review of accuracy assessment methods used in recently published RS DL studies, focusing on scene classification, object detection, semantic segmentation, and instance segmentation, indicates that RS DL papers appear to follow an accuracy assessment approach that diverges from that of traditional RS studies. Papers reporting on RS DL studies have largely abandoned traditional RS accuracy assessment terminology; they rarely reported a complete confusion matrix; and sampling designs and analysis protocols generally did not provide a population-based confusion matrix, in which the table entries are estimates of the probabilities of occurrence of the mapped landscape. These issues indicate the need for the RS community to develop guidance on best practices for accuracy assessment for CNN-based DL thematic mapping and object detection. As a first step in that process, we explore key issues, including the observation that accuracy assessments should not be biased by the CNN-based training and inference processes that rely on image chips. Furthermore, accuracy assessments should be consistent with prior recommendations and standards in the field, should support the estimation of a population confusion matrix, and should allow for assessment of model generalization. This paper draws from our review of the RS DL literature and the rich record of traditional remote sensing accuracy assessment research while considering the unique nature of CNN-based deep learning to propose accuracy assessment best practices that use appropriate sampling methods, training and validation data partitioning, assessment metrics, and reporting standards.

Accuracy Assessment in Convolutional Neural Network-Based Deep Learning Remote Sensing Studies—Part 1: Literature Review

Convolutional neural network (CNN)-based deep learning (DL) is a powerful, recently developed image classification approach. With origins in the computer vision and image processing communities, the accuracy assessment methods developed for CNN-based DL use a wide range of metrics that may be unfamiliar to the remote sensing (RS) community. To explore the differences between traditional RS and DL RS methods, we surveyed a random selection of 100 papers from the RS DL literature. The results show that RS DL studies have largely abandoned traditional RS accuracy assessment terminology, though some of the accuracy measures typically used in DL papers, most notably precision and recall, have direct equivalents in traditional RS terminology. Some of the DL accuracy terms have multiple names, or are equivalent to another measure. In our sample, DL studies only rarely reported a complete confusion matrix, and when they did so, it was even more rare that the confusion matrix estimated population properties. On the other hand, some DL studies are increasingly paying attention to the role of class prevalence in designing accuracy assessment approaches. DL studies that evaluate the decision boundary threshold over a range of values tend to use the precision-recall (P-R) curve, the associated area under the curve (AUC) measures of average precision (AP) and mean average precision (mAP), rather than the traditional receiver operating characteristic (ROC) curve and its AUC. DL studies are also notable for testing the generalization of their models on entirely new datasets, including data from new areas, new acquisition times, or even new sensors.

A Deep Learning-Based Data Minimization Algorithm for Fast and Secure Transfer of Big Genomic Datasets

In the age of Big Genomics Data, institutions such as the National Human Genome Research Institute (NHGRI) are challenged in their efforts to share volumes of data between researchers, a process that has been plagued by unreliable transfers and slow speeds. These occur due to throughput bottlenecks of traditional transfer technologies. Two factors that affect the efficiency of data transmission are the channel bandwidth and the amount of data. Increasing the bandwidth is one way to transmit data efficiently, but might not always be possible due to resource limitations. Another way to maximize channel utilization is by decreasing the bits needed for transmission of a dataset. Traditionally, transmission of big genomic data between two geographical locations is done using general-purpose protocols, such as hypertext transfer protocol (HTTP) and file transfer protocol (FTP) secure. In this paper, we present a novel deep learning-based data minimization algorithm that 1) minimizes the datasets during transfer over the carrier channels; 2) protects the data from the man-in-the-middle (MITM) and other attacks by changing the binary representation (content-encoding) several times for the same dataset: we assign different codewords to the same character in different parts of the dataset. Our data minimization strategy exploits the alphabet limitation of DNA sequences and modifies the binary representation (codeword) of dataset characters using deep learning-based convolutional neural network (CNN) to ensure a minimum of code word uses to the high frequency characters at different time slots during the transfer time. This algorithm ensures transmission of big genomic DNA datasets with minimal bits and latency and yields an efficient and expedient process. Our tested heuristic model, simulation, and real implementation results indicate that the proposed data minimization algorithm is up to 99 times faster and more secure than the currently used content-encoding scheme used in HTTP of the HTTP content-encoding scheme and 96 times faster than FTP on tested datasets. The developed protocol in C# will be available to the wider genomics community and domain scientists.

A comprehensive swarming intelligent method for optimizing deep learning-based object detection by unmanned ground vehicles.

Unmanned ground vehicles (UGVs) are an important research application of artificial intelligence. In particular, the deep learning-based object detection method is widely used in UGV-based environmental perception. Good experimental results are achieved by the deep learning-based object detection method Faster region-based convolutional neural network (Faster R-CNN). However, the exploration space of the region proposal network (RPN) is restricted by its expression. In our paper, a boosted RPN (BRPN) with three improvements is developed to solve this problem. First, a novel enhanced pooling network is designed in this paper. Therefore, the BRPN can adapt to objects with different shapes. Second, the expression of BRPN loss function is improved to learn the negative samples. Furthermore, the grey wolf optimizer (GWO) is used to optimize the parameters of the improved BRPN loss function. Thereafter, the performance of the BRPN loss function is promoted. Third, a novel GA-SVM classifier is applied to strengthen the classification capacity. The PASCAL VOC 2007, VOC 2012 and KITTI datasets are used to test the BRPN. Consequently, excellent experimental results are obtained by our deep learning-based object detection method.

A CNN-based novel solution for determining the survival status of heart failure patients with clinical record data: numeric to image

The aim of this study is to effectively evaluate numerical data, which are frequently encountered in the medical field, with popular deep learning-based Convolutional Neural Network (CNN) models. Heart failure is a common disease worldwide and it is very important to identify patients with a high survival rate and whose condition will deteriorate. A heart failure dataset consisting of numerical values only, needs to be converted into image data for analysis using the advantages of CNN. For this, first all raw data are normalized, then each normalized feature is placed in a region in the grid image. Thus, images with different brightness regions are obtained according to the numerical value of each feature. After the data augmentation step, these images are trained with five different CNN models (GoogleNet, MobileNet v2, ResNet18, ResNet50 and ResNet101) and classified. The highest accuracy of 95.13 % is obtained with the ResNet18 model and this accuracy is superior to studies using previous numerical raw data. The success proves the applicability of the proposed method and shows that numerical data in different fields can be easily classified with CNN models.