Use Of Convolutional Neural Networks Research Articles

AUTOMATIC CLASSIFICATION OF PAINTINGS BY YEAR OF CREATION

Context. The problem of automatic verification of the legitimacy of the export of works of art is considered. Objective. A method is proposed for automatically determining the age of a painting from a digital photograph using a classification that is performed by an intelligent decision-making system. Method. It is proposed to use the attribute of picture year of creation as the main criterion for making a decision during the customs check of exports legitimacy. Instead of a long and expensive museum examination, photographing works of art in customs conditions and processing photos using a set of descriptors is used. The set of descriptors is proposed, include local binary patterns, their color modification, Haralik’s texture features, the first four moments, Tamura’s texturt features, SIFT descriptor. The data obtained as a result of descriptors action give the values of several dozen private attributes. They form data vectors, which are then concatenated into a generalized object description vector. In the feature space thus created, automatic classification by weighted k-nearest neighbors is performed. The proposed algorithm calculates the distance between objects in a multidimensional space of attribute values and assigns new objects to already formed classes. The criterion for creating classes is the age of the painting from the existing database. As a measure of the objects proximity, it is proposed to use the Euclid and Minkowski metrics. The calculation of weights for the proposed classification algorithm is performed by the Fisher method. Results. The effectiveness of the proposed method was investigated in the course of experiments with an image database containing photos of paintings by world, European and Ukrainian artists. Algorithm configuration parameters that provide high classification accuracy are found. Conclusions. The performed experiments have shown the effectiveness of the selected descriptors for the formation of vector descriptions of images of paintings. The greatest accuracy is provided by descriptor merging, which reveals significant differences in the structural properties of images. This approach to the description of objects in combination with the proposed classification algorithm and the chosen main criterion ensures high accuracy of the obtained solutions. The direction of further research may include the use of convolutional neural networks to improve the accuracy of classification under the condition of a static database.

Region-Based Convolutional Neural Network-Based Spine Model Positioning of X-Ray Images.

Background Idiopathic scoliosis accounts for over 80% of all cases of scoliosis but has an unclear pathogenic mechanism. Many studies have introduced conventional image processing methods, but the results often fail to meet expectations. With the improvement and evolution of research in neural networks in the field of deep learning, many research efforts related to spinal reconstruction using the convolutional neural network (CNN) architecture of deep learning have shown promise. Purpose To investigate the use of CNN for spine modeling. Methods The primary technique used in this study involves Mask Region-based CNN (R-CNN) image segmentation and object detection methods as applied to spine model positioning of radiographs. The methods were evaluated based on common evaluation criteria for vertebral segmentation and object detection. Evaluations were performed using the loss function, mask loss function, classification loss function, target box loss function, average accuracy, and average recall. Results Many bony structures were directly identified in one step, including the lumbar spine (L1-L5) and thoracic spine (T1-T12) in frontal and lateral radiographs, thereby achieving initial positioning of the statistical spine model to provide spine model positioning for future reconstruction and classification prediction. An average detection box accuracy of 97.4% and an average segmentation accuracy of 96.8% were achieved for the prediction efficacy of frontal images, with good image visualization. Moreover, the results for lateral images were satisfactory considering the evaluation parameters and image visualization. Conclusion Mask R-CNN can be used for effective positioning in spine model studies for future reconstruction and classification prediction.

A novel convolutional neural network with gated recurrent unit for automated speech emotion recognition and classification

Automated Speech Emotion Recognition (SER) becomes more popular and has increased applicability. SER concentrates on the automatic identification of the emotional state of a human being using speech signals. It mainly depends upon the in-depth analysis of the speech signal, extracts features containing emotional details from the speech signal, and utilises pattern recognition techniques for emotional state identification. The major problem in automatic SER is to extract discriminate, powerful, and emotional salient features from the acoustical content of speech signals. The proposed model aims to detect and classify three emotional states of speech such as happy, neutral, and sad. The presented model makes use of Convolution neural network – Gated Recurrent unit (CNN-GRU) based feature extraction technique which derives a set of feature vectors. A comprehensive simulation takes place using the Berlin German Database and SJTU Chinese Database which comprises numerous audio files under a collection of different emotion labels.

Video annotation with metadata: A case study of soccer game video annotation with players

In recent years, the use of convolutional neural networks for video processing has become very attractive. The reason lies in the computational power for data processing which is available today. There are many well-defined research areas where neural networks have brought higher reliability than other conventional approaches; for example, traffic sign recognition and isolated number recognition. In this paper, we will describe the architecture and the implementation of the process of soccer game annotation. The game is annotated with data about players. The technology of convolutional neural networks is used for number recognition. The process runs in real-time on a streaming video. Content enriched with metadata is given to the user in parallel with the real-time video. In the paper, we will describe in some detail the following modules: Image binarization, shot localization, the selection and recognition of numbers on players` jerseys.

Analysis and construction by convolution neural network of link prediction model on social network

Estimating similarity using multiple similarity measures or machine learning prediction models is a popular solution to the link prediction problem. The Relation Pattern Deep Learning Classification (RPDLC) technique is proposed in this study, and it is based on multiple neighbor-based similarity metrics and convolution neural networks. The RPDLC first calculates the characteristics for a pair of nodes using neighbor-based metrics and impact nodes. Second, the RPDLC creates a heat map using node characteristics to assess the similarity of the nodes’ connection patterns. Third, the RPDLC uses convolution neural network architecture to build a prediction model for missing relationship prediction. On three separate social network datasets, this method is compared to other state-of-the-art algorithms. On all three datasets, the suggested method achieves the greatest AUC, hovering around 99 percent. The use of convolution neural networks and features via relational patterns to create a prediction model are the paper’s primary contributions.

A surrogate machine learning model for advanced gas-cooled reactor graphite core safety analysis

A surrogate machine learning model was developed with the aim of predicting seismic graphite core displacements from crack configurations for the advanced gas-cooled reactor. The model was trained on a dataset generated by a software package which simulates the behaviour of the graphite core during a severe earthquake. Several machine learning techniques, such as the use of convolutional neural networks, were identified as highly applicable to this particular problem. Through the development of the model, several observations and insights were garnered which may be of interest from a graphite core analysis and safety perspective. The best performing model was capable of making 95% of test set predictions within a 20 percentage point margin of the ground truth.

The Use of Convolutional Neural Networks in the Prediction of Invasive Ductal Carcinoma in Histological Images of Breast Cancer.

This paper presents a deep learning approach for automatic detection and visual analysis of Invasive Ductal Carcinoma (IDC) tissue regions. The method proposed in this work is a convolutional neural network (CNN) for visual semantic analysis of tumor regions for diagnostic support. Detection of IDC is a time-consuming and challenging task, mainly because a pathologist needs to examine large tissue regions to identify areas of malignancy. Deep Learning approaches are particularly suitable for dealing with this type of problem, especially when many samples are available for training, ensuring high quality of the learned features by the classifier and, consequently, its generalization capacity. A 3-hidden-layer CNN with data balancing reached both accuracy and F1-Score of 0.85 and outperforming other approaches from the literature. Thus, the proposed method in this article can serve as a support tool for the identification of invasive breast cancer.

Odonata identification using Customized Convolutional Neural Networks

The alarming decline of the Odonates (Dragonflies and Damselflies) population needs urgent conservation efforts to balance nature’s food cycle and reduce the negative impact on humans, such as the increasing number of mosquito-borne diseases. The existing odonates identification approaches are not efficient and are mainly based on domain expertise which is highly time-consuming and expensive. The fast-growing Artificial Intelligence technologies have shown impressive solutions to many real-world problems, including wildlife conservation and management. In this study, we proposed the use of Convolutional Neural Networks (CNN) for the identification of odonates. Firstly, we create a large Odonates image dataset comprising 54,176 images belonging to 256 species of odonates. Secondly, we employ nine Customized CNN architectures namely AlexNet, DenseNet121, DenseNet161, DenseNet201, ResNet18, ResNet101, ResNet152, SqueezeNet v1.1, and VGG19 to train our odonates image dataset. Finally, we then compared and empirically studied the performance of our proposed CNN architectures at various image resolutions (224px × 224px, 350px × 350px, 450px × 450px and 550px × 550px). Our pretrained custom DenseNet161 (450px × 450px) performed with highest top-1 accuracy (93.53%), ResNet152 (224px × 224px) with the highest top-3 accuracy (98.07%), DenseNet121 (450px × 450px), and ResNet152 (224px × 224px) with highest top-5 accuracy (98.85%) and DenseNet201 (550px × 550px) with highest F1 score (86.17%). SqueezeNet v1.1 performed the least with highest top-1 accuracy of 83.3%, highest top-3 accuracy (94.24%), highest top-5 accuracy (96.59%), and the highest F1 score (77.68%). The results of our study show that odonates can be reliably and instantly identified using our proposed methodology and the customized model.

Literature Survey of Plant Disease Detection using CNN

Abstract: Modern farming practises have the potential to feed the world's 7.6 billion inhabitants. Despite the availability of sufficient food, individuals continue to suffer from malnutrition. Plant diseases reduce both the amount and the quality of total production. Building an image processing model for prediction or classification applications presents several obstacles. We present a deep learning model for illness detection that makes use of CNN and Capsule Network (CapsNet). The backbone of image processing is a convolutional neural network (CNN). The new architecture called as CapsNet is suggested to reduce the limitations and to achieve greater performance than standard neural networks. In this project, we analyze CNN and CapsNet for tomato plant disease datasets. The performance of both the models are measured and analysed. As a result, this approach may be used to a variety of plants and on a huge scale. Keywords: Leaf disease detection; Convolutional Neural Network (CNN); Capsule Network (CapsNet); Performance Evaluation

A Systematic Approach for Explaining Time and Frequency Features Extracted by Convolutional Neural Networks From Raw Electroencephalography Data.

In recent years, the use of convolutional neural networks (CNNs) for raw resting-state electroencephalography (EEG) analysis has grown increasingly common. However, relative to earlier machine learning and deep learning methods with manually extracted features, CNNs for raw EEG analysis present unique problems for explainability. As such, a growing group of methods have been developed that provide insight into the spectral features learned by CNNs. However, spectral power is not the only important form of information within EEG, and the capacity to understand the roles of specific multispectral waveforms identified by CNNs could be very helpful. In this study, we present a novel model visualization-based approach that adapts the traditional CNN architecture to increase interpretability and combines that inherent interpretability with a systematic evaluation of the model via a series of novel explainability methods. Our approach evaluates the importance of spectrally distinct first-layer clusters of filters before examining the contributions of identified waveforms and spectra to cluster importance. We evaluate our approach within the context of automated sleep stage classification and find that, for the most part, our explainability results are highly consistent with clinical guidelines. Our approach is the first to systematically evaluate both waveform and spectral feature importance in CNNs trained on resting-state EEG data.

SEMANTIC URBAN MESH SEGMENTATION BASED ON AERIAL OBLIQUE IMAGES AND POINT CLOUDS USING DEEP LEARNING

Abstract. The use of deep machine learning methods for semantic classification of city mesh models is one of the current trends in geoscience development. Thanks to the thriving development of Convolutional Neural Networks (CNNs) it is now achievable to conduct fully automated process of building aforementioned 3D model by means of photogrammetric techniques and supplement it with additional semantic information obtained by Artificial Intelligence (AI) algorithms. In order to guarantee the comprehensiveness of said information it is essential to use an extensive range of 3D data including oblique aerial imagery and aerial laser scanning (ALS). Such comprehensive 3D mesh models may be later implemented in many Digital Twin class solutions additionally supported with modern GIS systems and its algorithms. To proof the validity of this thesis, the article showcases results of research conducted using deep learning based solutions tested on two datasets - real-world data in the form of oblique aerial images and ALS point clouds acquired in Bordeaux, France using novel Leica CityMapper-1 multisensoral system and large-scale dataset from SUM: A Benchmark Dataset of Semantic Urban Meshes. Both subalgorithms make use of CNNs as its core-feature. To perform accurate classification of oblique aerial scenes PSP-Net architecture accelerated by techniques of transfer learning has been used. Second algorithm destined for ALS point clouds utilizes CNN as well, but in this case implementation is based on proprietary architecture. The results of the experiments demonstrate that the utilizing these two mutually complementary solutions to extract new semantic information for city mesh models in proposed manner compared with the state-of-the-art methods achieves competitive classification performance.

Spectral-Spatial Attention Transformer with Dense Connection for Hyperspectral Image Classification.

In recent years, deep learning has been widely used in hyperspectral image (HSI) classification and has shown good capabilities. Particularly, the use of convolutional neural network (CNN) in HSI classification has achieved attractive performance. However, HSI contains a lot of redundant information, and the CNN-based model is limited by the receptive field of CNN and cannot balance the performance and depth of the model. Furthermore, considering that HSI can be regarded as sequence data, CNN-based models cannot mine sequence features well. In this paper, we propose a model named SSA-Transformer to address the above problems and extract spectral-spatial features of HSI more efficiently. The SSA-Transformer model combines a modified CNN-based spectral-spatial attention mechanism and a self-attention-based transformer with dense connection. The SSA-Transformer model can combine the local and global features of HSI to improve the performance of the model. A series of experiments showed that the SSA-Transformer achieved competitive classification accuracy compared with other CNN-based classification methods using three HSI datasets: University of Pavia (PU), Salinas (SA), and Kennedy Space Center (KSC).

Generating a labeled data set to train machine learning algorithms for lithologic classification of drill cuttings

Despite significant developments in the past few years in the application of machine learning algorithms for the lithologic classification of rock samples, publicly available labeled data sets are very scarce. We open source a fully labeled data set containing more than 16,000 scanning electron microscopy (SEM) images of drill cutting samples—mounted on thin sections—from a low-permeability reservoir in western Canada. We develop a simplified image processing workflow to segment and isolate the rock chips into individual SEM images, which in turn are used to identify, classify, and quantify rock types based on textural characteristics. In addition, using this data set, we explore the use of convolutional neural networks (CNNs) as a baseline tool for acceleration and automatization of rock-type classification. Without significant modifications to popular CNN models, we obtain an accuracy of approximately 90% for the test set. Results demonstrate the potential of CNN as a fast approach for lithologic classification in low-permeability siltstone reservoirs. In addition to making the data set publicly available, we believe our workflow to segment and isolate drill cutting samples in individual images of rock chips will facilitate future research of drill cuttings properties (e.g., lithology, porosity, and particle size) using machine learning algorithms.

A Caps-Ubi Model for Protein Ubiquitination Site Prediction.

Ubiquitination, a widespread mechanism of regulating cellular responses in plants, is one of the most important post-translational modifications of proteins in many biological processes and is involved in the regulation of plant disease resistance responses. Predicting ubiquitination is an important technical method for plant protection. Traditional ubiquitination site determination methods are costly and time-consuming, while computational-based prediction methods can accurately and efficiently predict ubiquitination sites. At present, capsule networks and deep learning are used alone for prediction, and the effect is not obvious. The capsule network reflects the spatial position relationship of the internal features of the neural network, but it cannot identify long-distance dependencies or focus on amino acids in protein sequences or their degree of importance. In this study, we investigated the use of convolutional neural networks and capsule networks in deep learning to design a novel model “Caps-Ubi,” first using the one-hot and amino acid continuous type hybrid encoding method to characterize ubiquitination sites. The sequence patterns, the dependencies between the encoded protein sequences and the important amino acids in the captured sequences, were then focused on the importance of amino acids in the sequences through the proposed Caps-Ubi model and used for multispecies ubiquitination site prediction. Through relevant experiments, the proposed Caps-Ubi method is superior to other similar methods in predicting ubiquitination sites.

Vibration-based structural damage detection via phase-based motion estimation using convolutional neural networks

Detection of structural damage is a major concern for engineers. In recent years, convolutional neural networks (CNNs) have been used for feature extraction and classification of vibration signals that reveal structural damage. Damage detection by CNNs greatly depends on high-quality learning data which are usually difficult to be obtained in actual engineering scenarios. To solve this problem, we combine phase-based motion estimation (PME) with the use of CNNs. By PME method, each pixel in a video can be regarded as a separate displacement sensor. Thus, it is possible to obtain millions of vibration signals from a single video, greatly facilitating CNN applications. We used a two-story steel structure for experimental validation. It was demonstrated that only one measured video sample obtained under each structural condition is possible to train a CNN model accurately detecting the location and severity of bolt looseness damage. This verified the outstanding performance of the proposed method.

Neural Networks to Recognize Patterns in Topographic Images of Cortical Electrical Activity of Patients with Neurological Diseases.

Software such as EEGLab has enabled the treatment and visualization of the tracing and cortical topography of the electroencephalography (EEG) signals. In particular, the topography of the cortical electrical activity is represented by colors, which make it possible to identify functional differences between cortical areas and to associate them with various diseases. The use of cortical topography with EEG origin in the investigation of diseases is often not used due to the representation of colors making it difficult to classify the disease. Thus, the analyses have been carried out, mainly, based on the EEG tracings. Therefore, a computer system that recognizes disease patterns through cortical topography can be a solution to the diagnostic aid. In view of this, this study compared five models of Convolutional Neural Networks (CNNs), namely: Inception v3, SqueezeNet, LeNet, VGG-16 and VGG-19, in order to know the patterns in cortical topography images obtained with EEG, in Parkinson's disease, Depression and Bipolar Disorder. SqueezeNet performed better in the 3 diseases analyzed, with Parkinson's disease being better evaluated for Accuracy (88.89%), Precison (86.36%), Recall (91.94%) and F1 Score (89.06%), the other CNNs had less performance. In the analysis of the values of the Area under ROC Curve (AUC), SqueezeNet reached (93.90%) for Parkinson's disease, (75.70%) for Depression and (72.10%) for Bipolar Disorder. We understand that there is the possibility of classifying neurological diseases from cortical topographies with the use of CNNs and, thus, creating a computational basis for the implementation of software for screening and possible diagnostic assistance.

Adjustable super-resolution network via deep supervised learning and progressive self-distillation

With the use of convolutional neural networks, Single-Image Super-Resolution (SISR) has advanced dramatically in recent years. However, we notice a phenomenon that the structure of all these models must be consistent during training and testing. This severely limits the flexibility of the model, making the same model difficult to be deployed on different sizes of platforms (e.g., computers, smartphones, and embedded devices). Therefore, it is crucial to develop a model that can adapt to different needs without retraining. To achieve this, we propose a lightweight Adjustable Super-Resolution Network (ASRN). Specifically, ASRN consists of a series of Multi-scale Aggregation Blocks (MABs), which is a lightweight and efficient module specially designed for feature extraction. Meanwhile, the Deep Supervised Learning (DSL) strategy is introduced into the model to guarantee the performance of each sub-network and a novel Progressive Self-Distillation (PSD) strategy is proposed to further improve the intermediate results of the model. With the help of DSL and PSD strategies, ASRN can achieve elastic image reconstruction. Meanwhile, ASRN is the first elastic SISR model, which shows good results after directly changing the model size without retraining.

A CNN-BASED FLOOD MAPPING APPROACH USING SENTINEL-1 DATA

Abstract. The adverse effects of flood events have been increasing in the world due to the increasing occurrence frequency and their severity due to urbanization and the population growth. All weather sensors, such as satellite synthetic aperture radars (SAR) enable the extent detection and magnitude analysis of such events under cloudy atmospheric conditions. Sentinel-1 satellite from European Space Agency (ESA) facilitate such studies thanks to the free distribution, the regular data acquisition scheme and the availability of open source software. However, various difficulties in the visual interpretation and processing exist due to the size and the nature of the SAR data. The supervised machine learning algorithms have increasingly been used for automatic flood extent mapping. However, the use of Convolutional Neural Networks (CNNs) for this purpose is relatively new and requires further investigations. In this study, the U-Net architecture for multi-class segmentation of flooded areas and flooded vegetation was employed by using Sentinel-1 SAR data and altitude information as input. The training data was produced by an automatic thresholding approach using OTSU method in Sardoba, Uzbekistan and Sagaing, Myanmar. The results were validated in Ordu, Turkey and in Ca River, Vietnam by visual comparison with previously produced flood maps. The results show that CNNs have great potential in classifying flooded areas and flooded vegetation even when trained in areas with different geographical setting. The F1 scores obtained in the study for flood and flooded vegetation classes were 0.91 and 0.85, respectively.

DEEP LEARNING BASED BUILDING FOOTPRINT EXTRACTION FROM VERY HIGH RESOLUTION TRUE ORTHOPHOTOS AND NDSM

Abstract. A challenging aspect of developing deep learning-based models for extracting building footprints from very high resolution (< 0.1 m) aerial imagery is the amount of details contained within the images. The use of convolutional neural networks (CNNs) to tackle semantic image segmentation has been shown to outperform conventional computer vision and machine learning approaches in various applications. Here, we investigated the performances of two different CNN architectures, U-Net and LinkNet by implementing them on various backbones and by using a number of building footprint vectors in a part of Turkey for training. The dataset includes red-green-blue (RGB) true orthophotos and normalized digital surface model (nDSM) data. The performances of the implemented methods were assessed comparatively by using the RGB data only and the RGB + nDSM. The results show that by adding nDSM as the fourth band to the RGB, the accuracy values obtained from the RGB only results were improved by 3.27% and 5.90% expressed in F1-Score and Jaccard (IoU) values, respectively. The highest accuracy reflected by the F1-Score of the validation data was 97.31%, while the F1-Score of the test data that was excluded from the model training was 96.14%. A vectorization process using the GDAL and Douglas-Peucker simplification algorithm was also performed to obtain the building footprints as polygons.

A Machine Learning Strategy for Race-Tracking Detection During Manufacturing of Composites by Liquid Moulding

This work presents a supervised machine learning (ML) model to detect race-tracking disturbances during the liquid moulding manufacturing of structural composites. Race-tracking is generated by unexpected resin channels at mould edges that may induce dry spots and porosity formation. The ML model uses the pressure signals recorded by a sensor network as input, providing a classification of the race-tracking event from a set of possible scenarios, and a subsequent variable regression for their position, size and strength. Such a model is based on the residual network (ResNet), a well-known artificial intelligence architecture that makes use of convolutional neural networks for image recognition. Training of the ML classifier and regressors was carried out with the aid of a synthetically generated simulation data set obtained throughout computational fluid dynamics simulations. The time evolution of the pressure sensors was used as grey-level images, or footprints, as inputs to the ResNet ML. The trained model was able to recognise the presence of race-tracking channels from the pressure data yielding good accuracy in terms of label prediction as well as position, size and strength. The model correlation was carried out with a set of injection experiments performed with a constant thickness closed mould containing induced race-tracking channels. The ability of ML models to provide an approximation to the inverse problem, relating the pressure sensor distortions to the cause of such events, is analysed and discussed.