Modern Convolutional Neural Networks Architectures Research Articles

#176 System to detect vascular calcification through convolutional neural networks

Abstract Background and Aims Cardiovascular disease is the major cause of death in patients with chronic kidney disease (CKD). It is increasingly believed that the presence of vascular calcification (VC) increases the risk of mortality due to cardiovascular events. VC is very common in CKD, especially in hemodialysis patient. To improve the life expectancy of patients, it is important to detect VC in the early stages of the disease. There are several ways to discover VC such as computed tomography (CT) scans, echocardiography, and X-Ray among others. However, one of the best options is the usage of X-ray images because is an affordable and simpler process in comparison to using CT scans or echocardiography. However, the problem with X-ray images is the difficulty of finding a well-trained professional that can detect the VC after analyzing the images which leads to possible misleading diagnoses. The goal of this study is to develop a prototype that detects Vascular Calcification from X-ray images in an automated way using the power of Convolutional Neural Networks (CNN). Method To train a CNN to detect VC from lumbosacral spine X-ray images, a set of 700 DICOM files complemented with 236 JPEG images was available. After getting the images, they had to be anonymized, and data, such as name and age was removed from each image. Then, the images were divided into three datasets: Train, Test, and Validation. Given that the dataset was not balanced between the pictures with VC (Positive) and those with no VC (Negative), it had to be reduced (Table 1). The next step was implemented to clean the images and prepare them to feed the CNN. An example of preprocessing is that some X-ray images were of the entire body. For the analysis, the VC detection was assessed at the anterior and the posterior walls of the abdominal aorta adjacent to vertebrae L1-L4. In addition to that, the DICOM images were converted to JPEG. After having the dataset prepared, a CNN architecture had to be selected. To achieve it, a revision of modern CNN architectures was performed. Two items were considered when selecting these: the performance on the ImageNet Large Scale Visual Recognition Challenge (ILSVRC) and if there was an available pre-trained model. With that criteria, pre-trained EfficientNet-B0 and ResNet50 models were used to build the prototype due to the limited dataset. Results After having the dataset prepared, the architectures selected, and the pre-trained models, the Training, Validation and Test stages were executed. For Training, with EfficientNet-B0, the Accuracy was 86.9%, while for Validation, the Accuracy was 57.4% (Fig. 1). Using the RestNet50 model, the precision was 90.506% during Training, while for Validation, it was 54.8% (Fig. 2). Finally, during the Testing stage, ResNet50 had a better performance, because EfficientNetB0 detected 80% for the negative class and 60% for the positive class. After the CNN models were trained, an additional validation stage was performed to compare the model results with the analysis made by a radiologist and a nephrologist. This comparison determined that the model performed better with images having VC, while it was less accurate with images having no presence of VC. This contrast could have different reasons, one being possible false-positive images. Conclusion This study created a CNN model using pre-trained models on top of modern architectures such as EfficientNet-B0 and ResNet50. Tools were used to prepare the images as well as to train, validate and test the model. The Transfer Learning helped to build a model faster, giving high Accuracy rates with only a few images due to the limited dataset. However, the model could give better results if a wider and balanced set of images is available. Also, a better pre-processing stage can be executed to reduce false-positive images. For future work, this model can be used as the core of a system that not only accurately detects Vascular Calcification from X-Ray images, but also can determine the level of calcification using measures such as the Kauppila Index or Adragao.

SAR deep learning sea ice retrieval trained with airborne laser scanner measurements from the MOSAiC expedition

Abstract. Automated sea ice charting from synthetic aperture radar (SAR) has been researched for more than a decade, and we are still not close to unlocking the full potential of automated solutions in terms of resolution and accuracy. The central complications arise from ground truth data not being readily available in the polar regions. In this paper, we build a data set from 20 near-coincident x-band SAR acquisitions and as many airborne laser scanner (ALS) measurements from the Multidisciplinary drifting Observatory for the Study of Arctic Climate (MOSAiC), between October and May. This data set is then used to assess the accuracy and robustness of five machine-learning-based approaches by deriving classes from the freeboard, surface roughness (standard deviation at 0.5 m correlation length) and reflectance. It is shown that there is only a weak correlation between the radar backscatter and the sea ice topography. Accuracies between 44 % and 66 % and robustness between 71 % and 83 % give a realistic insight into the performance of modern convolutional neural network architectures across a range of ice conditions over 8 months. It also marks the first time algorithms have been trained entirely with labels from coincident measurements, allowing for a probabilistic class retrieval. The results show that segmentation models able to learn from the class distribution perform significantly better than pixel-wise classification approaches by nearly 20 % accuracy on average.

UAV Thermal Imaging for Unexploded Ordnance Detection by Using Deep Learning

A few promising solutions for thermal imaging Unexploded Ordnance (UXO) detection were proposed after the start of the military conflict in Ukraine in 2014. At the same time, most of the landmine clearance protocols and practices are based on old, 20th-century technologies. More than 60 countries worldwide are still affected by explosive remnants of war, and new areas are contaminated almost every day. To date, no automated solutions exist for surface UXO detection by using thermal imaging. One of the reasons is also that there are no publicly available data. This research bridges both gaps by introducing an automated UXO detection method, and by publishing thermal imaging data. During a project in Bosnia and Herzegovina in 2019, an organisation, Norwegian People’s Aid, collected data about unexploded ordnances and made them available for this research. Thermal images with a size of 720 × 480 pixels were collected by using an Unmanned Aerial Vehicle at a height of 3 m, thus achieving a very small Ground Sampling Distance (GSD). One of the goals of our research was also to verify if the explosive war remnants’ detection accuracy could be improved further by using Convolutional Neural Networks (CNN). We have experimented with various existing modern CNN architectures for object identification, whereat the YOLOv5 model was selected as the most promising for retraining. An eleven-class object detection problem was solved primarily in this study. Our data were annotated semi-manually. Five versions of the YOLOv5 model, fine-tuned with a grid-search, were trained end-to-end on randomly selected 640 training and 80 validation images from our dataset. The trained models were verified on the remaining 88 images from our dataset. Objects from each of the eleven classes were identified with more than 90% probability, whereat the Mean Average Precision (mAP) at a 0.5 threshold was 99.5%, and the mAP at thresholds from 0.5 to 0.95 was 87.0% up to 90.5%, depending on the model’s complexity. Our results are comparable to the state-of-the-art, whereat these object detection methods have been tested on other similar small datasets with thermal images. Our study is one of the few in the field of Automated UXO detection by using thermal images, and the first that solves the problem of identifying more than one class of objects. On the other hand, publicly available thermal images with a relatively small GSD will enable and stimulate the development of new detection algorithms, where our method and results can serve as a baseline. Only really accurate automatic UXO detection solutions will help to solve one of the least explored worldwide life-threatening problems.

Application of Neural Networks for Virtual and Augmented Reality

The article analyzes modern virtual reality and augmented reality algorithms and ways of their implementation using neural networks. As a result, a classification of current virtual reality tasks is presented, the advantages and disadvantages of algorithms are identified, and the use of convolutional neural networks is proposed. As part of the study, a qualitative analysis of modern convolutional neural network architectures was carried out and their individual disadvantages when used in virtual reality systems were shown. As a result of the study, the optimal ways of applying neural networks in various tasks of identification, generativity and support in augmented and virtual reality systems were established. The functional and structural description of convolutional neural networks, the optimal structure and parameters for initialization and training of a convolutional neural network suitable for solving virtual reality problems are presented.

Channel pruning based on convolutional neural network sensitivity

Pruning is a useful technique for decreasing the memory consumption and floating point operations (FLOPs) of deep convolutional neural network (CNN) models. Nevertheless, at modest pruning levels, current structured pruning approaches often lead to considerable declines in accuracy. Furthermore, existing approaches often treat pruning rates as super parameters, neglecting the sensitivity of different convolution layers. In this study, we propose a novel sensitivity-based method for channel pruning that utilizes second-order sensitivity as a criterion. The essential concept is to prune insensitive filters while retaining sensitive ones. We quantify the sensitivity of the filter using the sum of the sensitivities of all weights in the filter, rather than the magnitude-based metric frequently applied in the literature. Furthermore, a layer sensitivity approach based on the Hessian eigenvalues of each layer is introduced into the process of automatically choosing the most appropriate pruning rate for each layer. Experiments on a variety of modern CNN architectures demonstrate that we can considerably enhance the pruning rate while sacrificing a small amount of accuracy, resulting in a reduction of more than 60% in FLOPs on CIFAR-10. Notably, on ImageNet, pruning based on ResNet50 decreased the FLOPs by 56.3% while losing only 0.92% of accuracy.

Calibrated Convolution with Gaussian of Difference

Attention mechanisms are widely used for Convolutional Neural Networks (CNNs) when performing various visual tasks. Many methods introduce multi-scale information into attention mechanisms to improve their feature transformation performance; however, these methods do not take into account the potential importance of scale invariance. This paper proposes a novel type of convolution, called Calibrated Convolution with Gaussian of Difference (CCGD), that takes into account both the attention mechanisms and scale invariance. A simple yet effective scale-invariant attention module that operates within a single convolution is able to adaptively build powerful scale-invariant features to recalibrate the feature representation. Along with this, a CNN with a heterogeneously grouped structure is used, which enhances the multi-scale representation capability. CCGD can be flexibly deployed in modern CNN architectures without introducing extra parameters. During experimental tests on various datasets, the method increased the ResNet50-based classification accuracy from 76.40% to 77.87% on the ImageNet dataset, and the tests generally confirmed that CCGD can outperform other state-of-the-art attention methods.

Passive acoustic monitoring of animal populations with transfer learning

Progress in deep learning, more specifically in using convolutional neural networks (CNNs) for the creation of classification models, has been tremendous in recent years. Within bioacoustics research, there has been a large number of recent studies that use CNNs. Designing CNN architectures from scratch is non-trivial and requires knowledge of machine learning. Furthermore, hyper-parameter tuning associated with CNNs is extremely time consuming and requires expensive hardware. In this paper we assess whether it is possible to build good bioacoustic classifiers by adapting and re-using existing CNNs pre-trained on the ImageNet dataset – instead of designing them from scratch, a strategy known as transfer learning that has proved highly successful in other domains. This study is a first attempt to conduct a large-scale investigation on how transfer learning can be used for passive acoustic monitoring (PAM), to simplify the implementation of CNNs and the design decisions when creating them, and to remove time consuming hyper-parameter tuning phases. We compare 12 modern CNN architectures across 4 passive acoustic datasets that target calls of the Hainan gibbon Nomascus hainanus, the critically endangered black-and-white ruffed lemur Varecia variegata, the vulnerable Thyolo alethe Chamaetylas choloensis, and the Pin-tailed whydah Vidua macroura. We focus our work on data scarcity issues by training PAM binary classification models very small datasets, with as few as 25 verified examples. Our findings reveal that transfer learning can result in up to 82% F1 score while keeping CNN implementation details to a minimum, thus rendering this approach accessible, easier to design, and speeding up further vocalisation annotations to create PAM robust models.

Application of Machine Learning for Emotion Classification

In this paper we propose an implement a general convolutional neural network (CNN) building framework for designing real-time CNNs. We validate our models by creat- ing a real-time vision system which accomplishes the tasks of face detection, gender classification and emotion classification simultaneously in one blended step using our proposed CNN architecture. After presenting the details of the training pro- cedure setup we proceed to evaluate on standard benchmark sets. We report accuracies of 96% in the IMDB gender dataset and 66% in the FER-2013 emotion dataset. Along with this we also introduced the very recent real-time enabled guided back- propagation visualization technique. Guided back-propagation uncovers the dynamics of the weight changes and evaluates the learned features. We argue that the careful implementation of modern CNN architectures, the use of the current regu- larization methods and the visualization of previously hidden features are necessary in order to reduce the gap between slow performances and real-time architectures. Our system has been validated by its deployment on a Care-O-bot 3 robot used during RoboCup@Home competitions. All our code, demos and pre- trained architectures have been released under an open-source license in our public repository.

Artificial intelligence for melanoma diagnosis.

Convolutional neural networks (CNN) have shown unprecedented accuracy in digital image analysis, which can be harnessed for melanoma recognition through automated evaluation of clinical and dermatoscopic images. In experimental studies, modern CNN architectures perform single image analysis at the level of dermatologists and domain-experts, also for multiclass predictions including a multitude of possible diagnoses. This may not necessarily translate to good clinical performance, and reliable randomized controlled prospective clinical trials for modern CNNs are essentially missing. Weaknesses of CNNs are that limitations of available training image datasets propagate to limitations of CNN predictions, and they cannot provide a reliable estimate of uncertainty. Recent research focuses on human-computer collaboration, where gains in accuracy were measured even with imperfect CNNs. With missing academic and clinical agreement on equivocal melanocytic lesions, fully automating histologic assessment of them with CNNs appear problematic, and applications in the near future are probably limited to supporting, referencing or recommendation roles.

A few filters are enough: Convolutional neural network for P300 detection

Over the past decade convolutional neural networks (CNNs) have become the driving force of an ever-increasing set of applications, achieving state-of-the-art performance. Modern CNN architectures are often composed of many convolutional and some fully connected layers, and have thousands or millions of parameters. CNNs have shown to be effective in the detection of Event-Related Potentials from electroencephalogram (EEG) signals, notably the P300 component which is frequently employed in Brain-Computer Interfaces (BCIs). However, for this task, the increase in detection rates compared to approaches based on human-engineered features has not been as impressive as in other areas and might not justify such a large number of parameters. In this paper, we study the performance of existing CNN architectures with diverse complexities for single-trial within-subject and cross-subject P300 detection on four different datasets. We also proposed SepConv1D, a very simple CNN architecture consisting of a single depthwise separable 1D convolutional layer followed by a fully connected Sigmoid classification neuron. We found that with as few as four filters in its convolutional layer and an overall small number of parameters, SepConv1D obtained competitive performances in the four datasets. We believe these results may represent an important step towards building simpler, cheaper, faster, and more portable BCIs.

Artificial intelligence for melanoma diagnosis: a review of published studies until 2020

Convolutional neural networks (CNN) have shown unprecedented accuracy in digital image analysis, which can be harnessed for melanoma recognition through automated evaluation of clinical and dermatoscopic images. In experimental studies, modern CNN architectures perform single image analysis at the level of dermatologists and domain-experts, also for multi-class predictions including a multitude of possible diagnoses. This may not necessarily translate to good clinical performance, and reliable randomised controlled prospective clinical trials for modern CNNs are essentially missing. Weaknesses of CNNs are that limitations of available training image datasets propagate to limitations of CNN predictions, and they can not provide a reliable estimate of uncertainty. Recent research focuses on human-computer collaboration, where gains in accuracy were measured even with imperfect CNNs. With missing academic and clinical agreement on equivocal melanocytic lesions, fully automating histologic assessment of them with CNNs appear problematic, and applications in the near future are probably limited to supporting, referencing or recommendation roles.

Structured feature sparsity training for convolutional neural network compression

Convolutional neural networks (CNNs) with large model size and computing operations are difficult to be deployed on embedded systems, such as smartphones or AI cameras. In this paper, we propose a novel structured pruning method, termed the structured feature sparsity training (SFST), to speed up the inference process and reduce the memory usage of CNNs. Unlike other existing pruning methods, which require multiple iterations of pruning and retraining to ensure stable performance, SFST only needs to fine-tune the pretrained model with additional regularization on the less important features and then prune them, no multiple pruning and retraining needed. SFST can be deployed to a variety of modern CNN architectures including VGGNet, ResNet and MobileNetv2. Experimental results on CIFAR, SVHN, ImageNet and MSTAR benchmark dataset demonstrate the effectiveness of our scheme, which achieves superior performance over the state-of-the-art methods.

Hardware implementation of a convolutional neural network using calculations in the residue number system

Modern convolutional neural networks architectures are very resource intensive which limits the possibilities for their wide practical application. We propose a convolutional neural network architecture in which the neural network is divided into hardware and software parts to increase performance and reduce the cost of implementation resources. We also propose to use the residue number system in the hardware part to implement the convolutional layer of the neural network for resource costs reducing. A numerical method for quantizing the filters coefficients of a convolutional network layer is proposed to minimize the influence of quantization noise on the calculation result in the residue number system and determine the bit-width of the filters coefficients. This method is based on scaling the coefficients by a fixed number of bits and rounding up and down. The operations used make it possible to reduce resources in hardware implementation due to the simplifying of their execution. All calculations in the convolutional layer are performed on numbers in a fixed-point format. Software simulations using Matlab 2017b showed that convolutional neural network with a minimum number of layers can be quickly and successfully trained. Hardware implementation using the field-programmable gate array Kintex7 xc7k70tfbg484-2 showed that the use of residue number system in the convolutional layer of the neural network reduces the hardware costs by 32.6% compared with the traditional approach based on the two’s complement representation. The research results can be applied to create effective video surveillance systems, for recognizing handwriting, individuals, objects and terrain.

Densely Based Multi-Scale and Multi-Modal Fully Convolutional Networks for High-Resolution Remote-Sensing Image Semantic Segmentation

Automatic and accurate semantic segmentation from high-resolution remote-sensing images plays an important role in the field of aerial images analysis. The task of dense semantic segmentation requires that semantic labels be assigned to each pixel in the image. Recently, convolutional neural networks (CNNs) have proven to be powerful tools for image classification, and they have been adopted in the remote-sensing community. But many limitations still exist when modern CNN architectures are directly applied to remote-sensing images, such as gradient explosion when the depth of the network increases, over-fitting with limited labeled remote-sensing data, and special differences between remote-sensing images and natural images. In this paper, we present a novel architecture that combines the thought of dense connection and fully convolutional networks, referred as DFCN, to automatically provide fine-grained semantic segmentation maps. In addition, we improve DFCN with multi-scale filters to widen the network and to increase the richness and diversity of extracted information, making the network more powerful and expressive than the naive convolution layer. Furthermore, we investigate a multi-modal network that incorporates digital surface models (DSMs) into a DFCN structure, and then we propose dual-path densely convolutional networks where the encoder consists of two paths that, respectively, extract features from spectral data and DSMs data and then fuse them. Finally, through conducting comprehensive experimental evaluations on two remote sensing benchmark datasets, we test our proposed models and compare them with other deep networks. The results demonstrate the effectiveness of proposed approaches; they can achieve competitive performance compared with the current state-of-the-art methods.

Convolutional neural networks as aid in core lithofacies classification

Artificial intelligence methods have a very wide range of applications. From speech recognition to self-driving cars, the development of modern deep-learning architectures is helping researchers to achieve new levels of accuracy in different fields. Although deep convolutional neural networks (CNNs) (a kind of deep-learning technique) have reached or surpassed human-level performance in image recognition tasks, little has been done to transport this new image classification technology to geoscientific problems. We have developed what we believe to be the first use of CNNs to identify lithofacies in cores. We use highly accurate models (trained with millions of images) and transfer learning to classify images of cored carbonate rocks. We found that different modern CNN architectures can achieve high levels of lithologic image classification accuracy (approximately 90%) and can aid in the core description task. This core image classification technique has the potential to greatly standardize and accelerate the description process. We also provide the community with a new set of labeled data that can be used for further geologic/data science studies.

SatCNN: satellite image dataset classification using agile convolutional neural networks

ABSTRACTWith the launch of various remote-sensing satellites, more and more high-spatial resolution remote-sensing (HSR-RS) images are becoming available. Scene classification of such a huge volume of HSR-RS images is a big challenge for the efficiency of the feature learning and model training. The deep convolutional neural network (CNN), a typical deep learning model, is an efficient end-to-end deep hierarchical feature learning model that can capture the intrinsic features of input HSR-RS images. However, most published CNN architectures are borrowed from natural scene classification with thousands of training samples, and they are not designed for HSR-RS images. In this paper, we propose an agile CNN architecture, named as SatCNN, for HSR-RS image scene classification. Based on recent improvements to modern CNN architectures, we use more efficient convolutional layers with smaller kernels to build an effective CNN architecture. Experiments on SAT data sets confirmed that SatCNN can quickly and effectively learn robust features to handle the intra-class diversity even with small convolutional kernels, and the deeper convolutional layers allow spontaneous modelling of the relative spatial relationships. With the help of fast graphics processing unit acceleration, SatCNN can be trained within about 40 min, achieving overall accuracies of 99.65% and 99.54%, which is the state-of-the-art for SAT data sets.