Types Of Convolutional Neural Networks Research Articles

Towards human-level performance on automatic pose estimation of infant spontaneous movements.

Assessment of spontaneous movements can predict the long-term developmental disorders in high-risk infants. In order to develop algorithms for automated prediction of later disorders, highly precise localization of segments and joints by infant pose estimation is required. Four types of convolutional neural networks were trained and evaluated on a novel infant pose dataset, covering the large variation in 1424 videos from a clinical international community. The localization performance of the networks was evaluated as the deviation between the estimated keypoint positions and human expert annotations. The computational efficiency was also assessed to determine the feasibility of the neural networks in clinical practice. The best performing neural network had a similar localization error to the inter-rater spread of human expert annotations, while still operating efficiently. Overall, the results of our study show that pose estimation of infant spontaneous movements has a great potential to support research initiatives on early detection of developmental disorders in children with perinatal brain injuries by quantifying infant movements from video recordings with human-level performance.

Bidirection-Fusion-YOLOv3: An Improved Method for Insulator Defect Detection Using UAV Image

With advances in the continuous improvement and development of the power system, insulators have gradually become one of the most important components. At present, unmanned aerial vehicles(UAV) have been widely used to inspect insulators, and insulators in pictures are accurately and efficiently identified by convolutional neural networks and this method has been extensively applied. These existing methods have been widely used to identify insulators in pictures, with high accuracy and efficiency. However, they are based on Faster R-CNN and you only look once(YOLO) either require more identification time due to the complex network structure, or do not have sufficient accuracy for insulator defects. More identification time is required due to the complexity of the network structure, or there is not enough accuracy for insulator defects. Based on the YOLOv3 network, this paper proposes a new type of convolutional neural network(CNN) for target detection, which can improve enhance the efficiency while ensuring the detection speed. In addition, this paper applies the latest EIoU and loss functions to YOLOv3, which significantly improves the coincidence of the prediction frame and the annotation frame and accelerates the convergence speed. The experimental results show that the detection model proposed in this paper has an average precision(AP) of 0.94 for insulators and 0.89 for insulator defects, and its detection speed can reach 93.5ms/image. Finally, after experimental verification, the detection model proposed in this paper meets the requirements of power inspection and has good engineering application prospects.

No-Reference Image Quality Assessment with Convolutional Neural Networks and Decision Fusion

No-reference image quality assessment (NR-IQA) has always been a difficult research problem because digital images may suffer very diverse types of distortions and their contents are extremely various. Moreover, IQA is also a very hot topic in the research community since the number and role of digital images in everyday life is continuously growing. Recently, a huge amount of effort has been devoted to exploiting convolutional neural networks and other deep learning techniques for no-reference image quality assessment. Since deep learning relies on a massive amount of labeled data, utilizing pretrained networks has become very popular in the literature. In this study, we introduce a novel, deep learning-based NR-IQA architecture that relies on the decision fusion of multiple image quality scores coming from different types of convolutional neural networks. The main idea behind this scheme is that a diverse set of different types of networks is able to better characterize authentic image distortions than a single network. The experimental results show that our method can effectively estimate perceptual image quality on four large IQA benchmark databases containing either authentic or artificial distortions. These results are also confirmed in significance and cross database tests.

Real Time Object Detection Using YoloReal Time Object Detection Using Yolo

Abstract: Object detection is related to computer vision and involves identifying the kinds of objects that have been detected. It is challenging to detect and classify objects. Recent advances in deep learning have allowed it to detect objects more accurately. In the past, there were several methods or tools used: R-CNN, Fast-RCNN, Faster-RCNN, YOLO, SSD, etc. This research focuses on "You Only Look Once" (YOLO) as a type of Convolutional Neural Network. Results will be accurate and timely when tested. So, we analysed YOLOv3's work by using Yolo3-tiny to detect both image and video objects. Keywords: YOLO, Intersection over Union (IOU), Anchor box, Non-Max Suppression, YOLO application, limitation.

Application of a Convolutional Neural Network for the Detection of Sea Ice Leads

Despite accounting for a small fraction of the surface area in the Arctic, long and narrow sea ice fractures, known as “leads”, play a critical role in the energy flux between the ocean and atmosphere. As the volume of sea ice in the Arctic has declined over the past few decades, it is increasingly important to monitor the corresponding changes in sea ice leads. A novel approach has been developed using artificial intelligence (AI) to detect sea ice leads using satellite thermal infrared window data from the Moderate Resolution Imaging Spectroradiometer (MODIS) and the Visible Infrared Imaging Radiometer Suite (VIIRS). In this new approach, a particular type of convolutional neural network, a U-Net, replaces a series of conventional image processing tests from our legacy algorithm. Results show the new approach has a high detection accuracy with F1 Scores on the order of 0.7. Compared to the legacy algorithm, the new algorithm shows improvement, with more true positives, fewer false positives, fewer false negatives, and better agreement between satellite instruments.

Satellite Image Classification Using a Hierarchical Ensemble Learning and Correlation Coefficient-Based Gravitational Search Algorithm

Satellite image classification is widely used in various real-time applications, such as the military, geospatial surveys, surveillance and environmental monitoring. Therefore, the effective classification of satellite images is required to improve classification accuracy. In this paper, the combination of Hierarchical Framework and Ensemble Learning (HFEL) and optimal feature selection is proposed for the precise identification of satellite images. The HFEL uses three different types of Convolutional Neural Networks (CNN), namely AlexNet, LeNet-5 and a residual network (ResNet), to extract the appropriate features from images of the hierarchical framework. Additionally, the optimal features from the feature set are extracted using the Correlation Coefficient-Based Gravitational Search Algorithm (CCGSA). Further, the Multi Support Vector Machine (MSVM) is used to classify the satellite images by extracted features from the fully connected layers of the CNN and selected features of the CCGSA. Hence, the combination of HFEL and CCGSA is used to obtain the precise classification over different datasets such as the SAT-4, SAT-6 and Eurosat datasets. The performance of the proposed HFEL–CCGSA is analyzed in terms of accuracy, precision and recall. The experimental results show that the HFEL–CCGSA method provides effective classification over the satellite images. The classification accuracy of the HFEL–CCGSA method is 99.99%, which is high when compared to AlexNet, LeNet-5 and ResNet.

Deep-Learning-Based Segmentation of Fresh or Young Concrete Sections from Images of Construction Sites.

The objective of this study is to evaluate the feasibility of deep-learning-based segmentation of the area covered by fresh and young concrete in the images of construction sites. The RGB images of construction sites under various actual situations were used as an input into several types of convolutional neural network (CNN)–based segmentation models, which were trained using training image sets. Various ranges of threshold values were applied for the classification, and their accuracy and recall capacity were quantified. The trained models could segment the concrete area overall although they were not able to judge the difference between concrete of different ages as professionals can. By increasing the threshold values for the softmax classifier, the cases of incorrect prediction as concrete became almost zero, while some areas of concrete became segmented as not concrete.

Fast identification of fluorescent components in three-dimensional excitation-emission matrix fluorescence spectra via deep learning

Three-dimensional excitation-emission matrix (3D-EEM) fluorescence spectroscopy has been widely applied to detect the fluorescent components in samples from natural water bodies to wastewater treatment processes. Data interpretation methods such as parallel factor analysis (PARAFAC) are required to decompose the overlapped fluorescent signals in the 3D-EEM spectra. However, strict requirements of data and complicated procedures of the PARAFAC limit the online monitoring and analysis of samples. Here we develop a fast fluorescent identification network (FFI-Net) model based on the deep learning approach to fast predict the numbers and maps of fluorescent components by simply inputting a single 3D-EEM spectrum. Two types of convolutional neural networks (CNN) are trained to classify the numbers of fluorescent components with an accuracy of 0.956 and predict the maps of fluorescent components with the min mean absolute error of 8.9 × 10-4. We demonstrate that the accuracy of the FFI-Net model will be further improved when more 3D-EEM data are available as a training dataset. Meanwhile, a user-friendly interface is designed to facilitate practical applications. Our approach gives a robust way to overcome the shortage of the PARAFAC and provides a new platform for online analysis of the fluorescent components in water samples.

Using Convolutional Neural Networks for Detection and Morphometric Analysis of Carolina Bays from Publicly Available Digital Elevation Models

Carolina Bays are oriented and sandy-rimmed depressions that are ubiquitous throughout the Atlantic Coastal Plain (ACP). Their origin has been a highly debated topic since the 1800s and remains unsolved. Past population estimates of Carolina Bays have varied vastly, ranging between as few as 10,000 to as many as 500,000. With such a large uncertainty around the actual population size, mapping these enigmatic features is a problem that requires an automated detection scheme. Using publicly available LiDAR-derived digital elevation models (DEMs) of the ACP as training images, various types of convolutional neural networks (CNNs) were trained to detect Carolina bays. The detection results were assessed for accuracy and scalability, as well as analyzed for various morphologic, land-use and land cover, and hydrologic characteristics. Overall, the detector found over 23,000 Carolina Bays from southern New Jersey to northern Florida, with highest densities along interfluves. Carolina Bays in Delmarva were found to be smaller and shallower than Bays in the southeastern ACP. At least a third of Carolina Bays have been converted to agricultural lands and almost half of all Carolina Bays are forested. Few Carolina Bays are classified as open water basins, yet almost all of the detected Bays were within 2 km of a water body. In addition, field investigations based upon detection results were performed to describe the sedimentology of Carolina Bays. Sedimentological investigations showed that Bays typically have 1.5 m to 2.5 m thick sand rims that show a gradient in texture, with coarser sand at the bottom and finer sand and silt towards the top. Their basins were found to be 0.5 m to 2 m thick and showed a mix of clayey, silty, and sandy deposits. Last, the results compiled during this study were compared to similar depressional features (i.e., playa-lunette systems) to pinpoint any similarities in origin processes. Altogether, this study shows that CNNs are valuable tools for automated geomorphic feature detection and can lead to new insights when coupled with various forms of remotely sensed and field-based datasets.

LAE-GAN-Based Face Image Restoration for Low-Light Age Estimation

Age estimation is applicable in various fields, and among them, research on age estimation using human facial images, which are the easiest to acquire, is being actively conducted. Since the emergence of deep learning, studies on age estimation using various types of convolutional neural networks (CNN) have been conducted, and they have resulted in good performances, as clear images with high illumination were typically used in these studies. However, human facial images are typically captured in low-light environments. Age information can be lost in facial images captured in low-illumination environments, where noise and blur generated by the camera in the captured image reduce the age estimation performance. No study has yet been conducted on age estimation using facial images captured under low light. In order to overcome this problem, this study proposes a new generative adversarial network for low-light age estimation (LAE-GAN), which compensates for the brightness of human facial images captured in low-light environments, and a CNN-based age estimation method in which compensated images are input. When the experiment was conducted using the MORPH, AFAD, and FG-NET databases—which are open databases—the proposed method exhibited more accurate age estimation performance and brightness compensation in low-light images compared to state-of-the-art methods.

Label-free SARS-CoV-2 detection and classification using phase imaging with computational specificity

Efforts to mitigate the COVID-19 crisis revealed that fast, accurate, and scalable testing is crucial for curbing the current impact and that of future pandemics. We propose an optical method for directly imaging unlabeled viral particles and using deep learning for detection and classification. An ultrasensitive interferometric method was used to image four virus types with nanoscale optical path-length sensitivity. Pairing these data with fluorescence images for ground truth, we trained semantic segmentation models based on U-Net, a particular type of convolutional neural network. The trained network was applied to classify the viruses from the interferometric images only, containing simultaneously SARS-CoV-2, H1N1 (influenza-A virus), HAdV (adenovirus), and ZIKV (Zika virus). Remarkably, due to the nanoscale sensitivity in the input data, the neural network was able to identify SARS-CoV-2 vs. the other viruses with 96% accuracy. The inference time for each image is 60 ms, on a common graphic-processing unit. This approach of directly imaging unlabeled viral particles may provide an extremely fast test, of less than a minute per patient. As the imaging instrument operates on regular glass slides, we envision this method as potentially testing on patient breath condensates. The necessary high throughput can be achieved by translating concepts from digital pathology, where a microscope can scan hundreds of slides automatically.

Visualizing Multimodal Deep Learning for Lesion Prediction.

A U-Net is a type of convolutional neural network that has been shown to output impressive results in medical imaging segmentation tasks. Still, neural networks in general form a black box that is hard to interpret, especially by noncomputer scientists. This work provides a visual system that allows users to examine U-Nets that were trained to predict brain lesions caused by stroke using multimodal imaging. We provide several visualization views that allow users to load trained U-Nets, run them on different patient data, and examine the results while visually following the computation of the U-Net. With these visualizations, we can provide useful information for our medical collaborators showing how the training database can be improved and which features are best learned by the neural network.

Implementation of a Deep Learning Algorithm Based on Vertical Ground Reaction Force Time-Frequency Features for the Detection and Severity Classification of Parkinson's Disease.

Conventional approaches to diagnosing Parkinson’s disease (PD) and rating its severity level are based on medical specialists’ clinical assessment of symptoms, which are subjective and can be inaccurate. These techniques are not very reliable, particularly in the early stages of the disease. A novel detection and severity classification algorithm using deep learning approaches was developed in this research to classify the PD severity level based on vertical ground reaction force (vGRF) signals. Different variations in force patterns generated by the irregularity in vGRF signals due to the gait abnormalities of PD patients can indicate their severity. The main purpose of this research is to aid physicians in detecting early stages of PD, planning efficient treatment, and monitoring disease progression. The detection algorithm comprises preprocessing, feature transformation, and classification processes. In preprocessing, the vGRF signal is divided into 10, 15, and 30 s successive time windows. In the feature transformation process, the time domain vGRF signal in windows with varying time lengths is modified into a time–frequency spectrogram using a continuous wavelet transform (CWT). Then, principal component analysis (PCA) is used for feature enhancement. Finally, different types of convolutional neural networks (CNNs) are employed as deep learning classifiers for classification. The algorithm performance was evaluated using k-fold cross-validation (kfoldCV). The best average accuracy of the proposed detection algorithm in classifying the PD severity stage classification was 96.52% using ResNet-50 with vGRF data from the PhysioNet database. The proposed detection algorithm can effectively differentiate gait patterns based on time–frequency spectrograms of vGRF signals associated with different PD severity levels.

Detail-Oriented Capsule Network for classification of CT scan images performing the detection of COVID-19

COVID-19 is one of the biggest pandemics that the world is facing today, and every day, we are coming up with new challenges in this area. Still, much research is already going on to overcome this pandemic, and we also get succeeded to some extent. Diverse sources such as MRI, CT scanning, blood samples, X-ray image, and many more are available to detect COVID-19. Thus, it can be easily said that through image processing, the classification of COVID-19 can be done. In this study, the COVID-19 detection is done by classifying with the use of a type of convolutional neural network termed a detail-oriented capsule network. Chest CT scan imaging for the prediction of COVID-19 and non-COVID-19 are classified in the present paper using a Detailed Oriented capsule network (DOCN). Accuracy, specificity, and sensitivity are parameters used for model evaluation. The proposed model has achieved 98% accuracy, 81% sensitivity, and 98.4% specificity.

Application of convolutional neural networks for distal radio-ulnar fracture detection on plain radiographs in the emergency room.

ObjectiveRecent studies have suggested that deep-learning models can satisfactorily assist in fracture diagnosis. We aimed to evaluate the performance of two of such models in wrist fracture detection.MethodsWe collected image data of patients who visited with wrist trauma at the emergency department. A dataset extracted from January 2018 to May 2020 was split into training (90%) and test (10%) datasets, and two types of convolutional neural networks (i.e., DenseNet-161 and ResNet-152) were trained to detect wrist fractures. Gradient-weighted class activation mapping was used to highlight the regions of radiograph scans that contributed to the decision of the model. Performance of the convolutional neural network models was evaluated using the area under the receiver operating characteristic curve. ResultsFor model training, we used 4,551 radiographs from 798 patients and 4,443 radiographs from 1,481 patients with and without fractures, respectively. The remaining 10% (300 radiographs from 100 patients with fractures and 690 radiographs from 230 patients without fractures) was used as a test dataset. The sensitivity, specificity, positive predictive value, negative predictive value, and accuracy of DenseNet-161 and ResNet-152 in the test dataset were 90.3%, 90.3%, 80.3%, 95.6%, and 90.3% and 88.6%, 88.4%, 76.9%, 94.7%, and 88.5%, respectively. The area under the receiver operating characteristic curves of DenseNet-161 and ResNet-152 for wrist fracture detection were 0.962 and 0.947, respectively.ConclusionWe demonstrated that DenseNet-161 and ResNet-152 models could help detect wrist fractures in the emergency room with satisfactory performance.

Deep learning and Its Development

Deep learning is a promising branch of machine learning. It is an algorithm that uses artificial neural networks as the architecture to characterize and learn data. In recent years, many companies, for example, Google, Microsoft and Baidu, have become interested in the field of deep learning and have set up many large-scale projects, such as Google’s Deepmind project, including alphago, which has achieved success in Go and e-sports. This article analyzes and summarizes each current research direction and approach of deep learning, with prospection about the future research direction and development of deep learning expounded. An overview of the three basic models of deep learning is given, namely multilayer perceptrons and perceptrons, convolutional neural networks and recurrent neural networks. The benefits and superiority of the deep learning algorithm are illustrated and compared with the conventional methodology used in the common applications. Further research on emerging types of convolutional neural networks and recurrent neural networks are introduced. An overview of the three basic models of deep learning is given, namely multilayer perceptrons and perceptrons, convolutional neural networks and recurrent neural networks. The current application of deep learning in various fields is summarized, such as artificial intelligence, computer vision and natural language processing applications, and some open problems for future research are also analyzed. Finally, the significance and purpose of deep learning are discussed.

Semantic segmentation of human oocyte images using deep neural networks

BackgroundInfertility is a significant problem of humanity. In vitro fertilisation is one of the most effective and frequently applied ART methods. The effectiveness IVF depends on the assessment and selection of gametes and embryo with the highest developmental potential. The subjective nature of morphological assessment of oocytes and embryos is still one of the main reasons for seeking effective and objective methods for assessing quality in automatic manner. The most promising methods to automatic classification of oocytes and embryos are based on image analysis aided by machine learning techniques. The special attention is paid on deep neural networks that can be used as classifiers solving the problem of automatic assessment of the oocytes/embryos.MethodsThis paper deals with semantic segmentation of human oocyte images using deep neural networks in order to develop new version of the predefined neural networks. Deep semantic oocyte segmentation networks can be seen as medically oriented predefined networks understanding the content of the image. The research presented in the paper is focused on the performance comparison of different types of convolutional neural networks for semantic oocyte segmentation. In the case study, the merits and limitations of the selected deep neural networks are analysed.Results71 deep neural models were analysed. The best score was obtained for one of the variants of DeepLab-v3-ResNet-18 model, when the training accuracy (Acc) reached about 85% for training patterns and 79% for validation ones. The weighted intersection over union (wIoU) and global accuracy (gAcc) for test patterns were calculated, as well. The obtained values of these quality measures were 0,897 and 0.93, respectively.ConclusionThe obtained results prove that the proposed approach can be applied to create deep neural models for semantic oocyte segmentation with the high accuracy guaranteeing their usage as the predefined networks in other tasks.

Optimized convolutional neural network model for plant species identification from leaf images using computer vision

In recent works of computer science, especially in the fields of image processing and pattern recognition techniques with machine learning, considerable focus is given to plant taxonomy which enhances the abilities of people to recognize plant species. This paper presents a method that analyzes color images of leaves using a type of Convolutional Neural Network to recognize plant species. The proposed Neural Network consists of four convolutional layers followed by two Fully-Connected layers and a final soft-max layer to offer a feature representation for different plant species. Four max-pooling layers are performed over a 2 × 2 pixel window with stride 2. Results on five plant datasets viz. Leaf snap (52 plant species), UCI leaf (40 plant species), PlantVillage (38 plant species), Flavia (32 plant species) and Swedish (15 plant species) are tabulated that demonstrate the remarkable performance of the proposed deep neural network when compared to the state of art methods.

Convolutional Neural Network-Based Artificial Intelligence for Classification of Protein Localization Patterns

Identifying localization of proteins and their specific subpopulations associated with certain cellular compartments is crucial for understanding protein function and interactions with other macromolecules. Fluorescence microscopy is a powerful method to assess protein localizations, with increasing demand of automated high throughput analysis methods to supplement the technical advancements in high throughput imaging. Here, we study the applicability of deep neural network-based artificial intelligence in classification of protein localization in 13 cellular subcompartments. We use deep learning-based on convolutional neural network and fully convolutional network with similar architectures for the classification task, aiming at achieving accurate classification, but importantly, also comparison of the networks. Our results show that both types of convolutional neural networks perform well in protein localization classification tasks for major cellular organelles. Yet, in this study, the fully convolutional network outperforms the convolutional neural network in classification of images with multiple simultaneous protein localizations. We find that the fully convolutional network, using output visualizing the identified localizations, is a very useful tool for systematic protein localization assessment.

Cystoscopic image classification by an ensemble of VGG-nets

Over the last three decades, artificial intelligence has attracted lots of attentions in medical diagnosis tasks. However, few studies have been presented to assist urologists to diagnose bladder cancer in spite of its high prevalence worldwide. In this paper, a new computer aided diagnosis system is proposed to classify four types of cystoscopic images including malignant masses, benign masses, blood in urine, and normal. The proposed classifier is an ensemble of a well-known type of convolutional neural networks (CNNs) called VGG-Net. To combine the VGG-Nets, bootstrap aggregating approach is used. The proposed ensemble classifier was evaluated on a dataset of 720 images. Based on the experiments, the presented method achieved an accuracy of 63% which outperforms base VGG-Nets and other competing methods.