Multilayer Convolutional Neural Network Research Articles

Intelligent video surveillance using enhanced deep belief based multilayered convolution neural network classification techniques

AbstractVideo surveillance has undergone numerous changes in the past few years and several pieces of research have been carried out in this field. Object tracking is the significant task in such systems, and hence it is essential to review the standard approaches dealing with object detection, classification and tracking. This work proposes a novel classification technique for a detected object of a moving scene from a video dataset. Initially the dataset has been processed and prepared for data training based on neural networks. The data has been classified using the proposed enhanced deep belief based multilayered convolutional neural network (EBMCNN). The major focus is on deep learning applications involved in estimating the count, total persons involved and the activities in a crowd where all criteria are taken into consideration, thereby achieving security through video analysis. Identifying theft and the detection of violence are some security measures where video is converted to frames which are then processed to analyse the individuals along with their activities. The classification is performed through comparative analysis of a real‐time dataset. Experimental results show the accuracy 97%, precision 93.8%, recall 87.7% and F‐1 score 87.5%.

Survey of Recent Applications of Artificial Intelligence for Detection and Analysis of COVID-19 and Other Infectious Diseases

The purpose is to illustrate how artificial intelligence (AI) technologies have been used for detection and analysis of COVID-19 and other infectious diseases such as breast, lung, and skin cancers; heart disease; and others. Specifically, the use of neural networks (NN) and machine learning (ML) are described along with which countries are creating these techniques and how these are being used for COVID-19 or other disease diagnosis and detection. Illustrations of multi-layer convolutional neural networks (CNN), recurrent neural networks (RNN), and deep neural networks (DNN) are provided to show how these are used for COVID-19 or other disease detection and prediction. A summary of big data analytics for COVID-19 and some available COVID-19 open-source data sets and repositories and their characteristics for research and analysis is also provided. An example is also shown for artificial intelligence (AI) and neural network (NN) applications using real-time COVID-19 data.

Predicting In-Vitro DNA-Protein Binding With a Spatially Aligned Fusion of Sequence and Shape.

Discovery of transcription factor binding sites (TFBSs) is of primary importance for understanding the underlying binding mechanic and gene regulation process. Growing evidence indicates that apart from the primary DNA sequences, DNA shape landscape has a significant influence on transcription factor binding preference. To effectively model the co-influence of sequence and shape features, we emphasize the importance of position information of sequence motif and shape pattern. In this paper, we propose a novel deep learning-based architecture, named hybridShape eDeepCNN, for TFBS prediction which integrates DNA sequence and shape information in a spatially aligned manner. Our model utilizes the power of the multi-layer convolutional neural network and constructs an independent subnetwork to adapt for the distinct data distribution of heterogeneous features. Besides, we explore the usage of continuous embedding vectors as the representation of DNA sequences. Based on the experiments on 20 in-vitro datasets derived from universal protein binding microarrays (uPBMs), we demonstrate the superiority of our proposed method and validate the underlying design logic.

A Multifeatured Time–Frequency Neural Network System for Classifying sEMG

Human gesture-related electromyographic signal in intelligent recognition systems has attracted widespread attention. The EMG signal is the temporal and spatial superposition of motor unit neural potentials and it contains nonlinear and complex characteristics. Despite the previous efforts, recognizing more different actions and building an online recognition system remain challenging. To address these issues, we construct a multi-feature time-frequency neural network system and a dataset which contains twenty kinds of hand movements. In a multi-featured time-frequency neural network system, the multi-layer CNN structure is used to obtain faster recognition and better classification results. The experimental results demonstrate that equipment and algorithmic model can reach a 94.66% accuracy. Our approach focuses on processing multiple time and frequency domain features of sEMG using CNN. Importantly, we also perform real-time validation. The system can accurately recognize hand movements to control UAV motion in real-time, in which case system classification takes 0.05s. This demonstrates the practicality and effectiveness of the sEMG acquisition system based on the multifeatured time-frequency neural network.

High Resolution Interferometric Imaging of Liquid-Solid Interfaces with HOTNNET

BackgroundA variety of imaging methods are available to obtain kinematic data at an interface, with a widely varying range of spatial and temporal resolution. These methods require a trade-off between imaging rate and resolution.ObjectiveA deep learning framework trained on synchronous profilometry data acquired using two imaging modalities at two different spatial resolutions to enhance spatial resolution while maintaining temporal resolution is desired.MethodsFizeau interferometry (FIF) and frustrated total internal reflection (FTIR) are used to overcome the resolution-rate trade-off via a deep learning framework. The FTIR imaging data are recorded at high resolution, while the FIF imaging data are recorded with a lesser resolved, larger field of view. We apply a deep learning framework using a multi-layer convolutional neural network to enhance the FIF image resolution.ResultsWith the deep learning framework, we achieve the high spatial resolution of measurements obtained by FTIR imaging in all three dimensions from the lower resolution FIF data. A high-order overset technique ultimately yields full up-scaled images from the network outputs without losing precision. The accuracy of the super-resolved image is evaluated using test data.ConclusionsThis hybrid framework, called HOTNNET, is implemented in its entirety on high-speed imaging profilometry data acquired in the study of droplet impacts on a smooth, solid surface, and is used to recover full, high-resolution images at high rates by unwrapping the phase of the interferometry. This framework can be readily adapted to other paired datasets by retraining the network on the novel data.

Prophetic Energy Assessment with Smart Implements in Hydroelectricity Entities Using Artificial Intelligence Algorithm

An encouraging development is the quick expansion of renewable energy extraction. Harnessing renewable energy is economically feasible at the current rate of technological advancement. Traditional energy sources, such as coal, petroleum, and hydrocarbons, which have negative effects on the environment, are coming under more social and financial pressure. Companies need more solar and wind power because this calls for a well-balanced mix of renewable resources and a higher proportion of alternative energy sources. Sustainable energy can be captured using a variety of techniques. Massive scale and small-sized are the two most prevalent techniques. No renewable energy source possesses an inherent property that restricts how it may be managed or how it can be planned to produce electricity. A number of factors have contributed to a growth in the use of alternative sources, one of which is to mitigate the effects of rising temperatures. To improve the ability to estimate renewable energy, various modeling approaches have been created. This region might use an HRES to give many sources with the inclusion of different energy sources. The inventiveness of solar and wind power and the brilliant ability of neural networks to handle complex time-series data signals have both aided in the prediction of sustainable energy. Therefore, this research will examine the numerous information models in order to determine which proposed models can provide accurate projections of renewable energy output, such as sunlight, wind, or pumped storage. In the fields of sustainable energy predictions, a number of machine learning methods, such as multilayer perceptions MLP, RNN CNN, and LSTM designs, are frequently utilized. This form of modeling uses historical data to predict potential values and can predict short-term patterns in solar and wind generation.

Static-gesture word recognition in Bangla sign language using convolutional neural network

Sign language is the communication process of people with hearing impairments. For hearing-impaired communication in Bangladesh and parts of India, Bangla sign language (BSL) is the standard. While Bangla is one of the most widely spoken languages in the world, there is a scarcity of research in the field of BSL recognition. The few research works done so far focused on detecting BSL alphabets. To the best of our knowledge, no work on detecting BSL words has been conducted till now for the unavailability of BSL word dataset. In this research, a small static-gesture word dataset has been developed, and a deep learning-based method has been introduced that can detect BSL static-gesture words from images. The dataset, “BSLword” contains 30 static-gesture BSL words with 1200 images for training. The training is done using a multi-layered convolutional neural network with the Adam optimizer. OpenCV is used for image processing and TensorFlow is used to build the deep learning models. This system can recognize BSL static-gesture words with 92.50% accuracy on the word dataset.

Official logo recognition based on multilayer convolutional neural network model

Deep learning has gained high popularity in the field of image processing and computer vision applications due to its unique feature extraction property. For this characteristic, deep learning networks used to solve different issues in computer vision applications. In this paper the issue has been raised is classification of logo of formal directors in Iraqi government. The paper proposes a multi-layer convolutional neural network (CNN) to classify and recognize these official logos by train the CNN model on several logos. The experimental show the effectiveness of the proposed method to recognize the logo with high accuracy rate about 99.16%. The proposed multi-layers CNN model proves the effectiveness to classify different logos with various conditions.

Number of Convolution Layers and Convolution Kernel Determination and Validation for Multilayer Convolutional Neural Network: Case Study in Breast Lesion Screening of Mammographic Images

Mammography is a low-dose X-ray imaging technique that can detect breast tumors, cysts, and calcifications, which can aid in detecting potential breast cancer in the early stage and reduce the mortality rate. This study employed a multilayer convolutional neural network (MCNN) to screen breast lesions with mammographic images. Within the region of interest, a specific bounding box is used to extract feature maps before automatic image segmentation and feature classification are conducted. These include three classes, namely, normal, benign tumor, and malignant tumor. Multiconvolution processes with kernel convolution operations have noise removal and sharpening effects that are better than other image processing methods, which can strengthen the features of the desired object and contour and increase the classifier’s classification accuracy. However, excessive convolution layers and kernel convolution operations will increase the computational complexity, computational time, and training time for training the classifier. Thus, this study aimed to determine a suitable number of convolution layers and kernels to achieve a classifier with high learning performance and classification accuracy, with a case study in the breast lesion screening of mammographic images. The Mammographic Image Analysis Society Digital Mammogram Database (United Kingdom National Breast Screening Program) was used for experimental tests to determine the number of convolution layers and kernels. The optimal classifier’s performance is evaluated using accuracy (%), precision (%), recall (%), and F1 score to test and validate the most suitable MCNN model architecture.

Analysis of Smart Lung Tumour Detector and Stage Classifier Using Deep Learning Techniques with Internet of Things.

The use of artificial intelligence (AI) and the Internet of Things (IoT), which is a developing technology in medical applications that assists physicians in making more informed decisions regarding patients' courses of treatment, has become increasingly widespread in recent years in the field of healthcare. On the other hand, the number of PET scans that are being performed is rising, and radiologists are getting significantly overworked as a result. As a direct result of this, a novel approach that goes by the name “computer-aided diagnostics” is now being investigated as a potential method for reducing the tremendous workloads. A Smart Lung Tumor Detector and Stage Classifier (SLD-SC) is presented in this study as a hybrid technique for PET scans. This detector can identify the stage of a lung tumour. Following the development of the modified LSTM for the detection of lung tumours, the proposed SLD-SC went on to develop a Multilayer Convolutional Neural Network (M-CNN) for the classification of the various stages of lung cancer. This network was then modelled and validated utilising standard benchmark images. The suggested SLD-SC is now being evaluated on lung cancer pictures taken from patients with the disease. We observed that our recommended method gave good results when compared to other tactics that are currently being used in the literature. These findings were outstanding in terms of the performance metrics accuracy, recall, and precision that were assessed. As can be shown by the much better outcomes that were achieved with each of the test images that were used, our proposed method excels its rivals in a variety of respects. In addition to this, it achieves an average accuracy of 97 percent in the categorization of lung tumours, which is much higher than the accuracy achieved by the other approaches.

Dermoscopy lesion classification based on GANs and a fuzzy rank-based ensemble of CNN models

Background and Objective. Skin lesion classification by using deep learning technologies is still a considerable challenge due to high similarity among classes and large intraclass differences, serious class imbalance in data, and poor classification accuracy with low robustness. Approach. To address these issues, a two-stage framework for dermoscopy lesion classification using adversarial training and a fuzzy rank-based ensemble of multilayer feature fusion convolutional neural network (CNN) models is proposed. In the first stage, dermoscopy dataset augmentation based on generative adversarial networks is proposed to obtain realistic dermoscopy lesion images, enabling significant improvement for balancing the number of lesions in each class. In the second stage, a fuzzy rank-based ensemble of multilayer feature fusion CNN models is proposed to classify skin lesions. In addition, an efficient channel integrating spatial attention module, in which a novel dilated pyramid pooling structure is designed to extract multiscale features from an enlarged receptive field and filter meaningful information of the initial features. Combining the cross-entropy loss function with the focal loss function, a novel united loss function is designed to reduce the intraclass sample distance and to focus on difficult and error-prone samples to improve the recognition accuracy of the proposed model. Main results. In this paper, the common dataset (HAM10000) is selected to conduct simulation experiments to evaluate and verify the effectiveness of the proposed method. The subjective and objective experimental results demonstrate that the proposed method is superior over the state-of-the-art methods for skin lesion classification due to its higher accuracy, specificity and robustness. Significance. The proposed method effectively improves the classification performance of the model for skin diseases, which will help doctors make accurate and efficient diagnoses, reduce the incidence rate and improve the survival rates of patients.

Prediction of Drug-Related Diseases Through Integrating Pairwise Attributes and Neighbor Topological Structures.

Identifying new disease indications for the approved drugs can help reduce the cost and time of drug development. Most of the recent methods focus on exploiting the various information related to drugs and diseases for predicting the candidate drug-disease associations. However, the previous methods failed to deeply integrate the neighborhood topological structure and the node attributes of an interested drug-disease node pair. We propose a new prediction method, ANPred, to learn and integrate pairwise attribute information and neighbor topology information from the similarities and associations related to drugs and diseases. First, a bi-layer heterogeneous network with intra-layer and inter-layer connections is established to combine the drug similarities, the disease similarities, and the drug-disease associations. Second, the embedding of a pair of drug and disease is constructed based on integrating multiple biological premises about drugs and diseases. The learning framework based on multi-layer convolutional neural networks is designed to learn the attribute representation of the pair of drug and disease nodes from its embedding. The sequences composed of neighbor nodes are formed based on random walk on the heterogeneous network. A framework based on fully-connected autoencoder and skip-gram module is constructed to learn the neighbor topological representations of nodes. The cross-validation results indicate the performance of ANPred is superior to several state-of-the-art methods. The case studies on 5 drugs further confirm the ability of ANPred in discovering the potential drug-disease association candidates.

A disaster classification application using convolutional neural network by performing data augmentation

Natural disasters are catastrophic events and cause havoc to human life. These events occur in the most unpredictable times and are beyond human control. The aftermath of the disasters is devastating ranging from loss of life to relocation of large groups of the population. With the development in the domains of computer vision (CV) and Image processing, machine learning and deep learning models can integrate images and perform predictions. Deep learning techniques employ many robust techniques and provide significant results even in the case of images. The detection of natural disasters without human intervention requires the help of deep learning techniques. The project aims to employ a multi-layered convolutional neural network (CNN) organization to classify the images related to natural disasters related to earthquakes, floods, cyclones, and wildfires.

An enhanced CNN-LSTM remaining useful life prediction model for aircraft engine with attention mechanism

Remaining useful life (RUL) prediction is one of the key technologies of aircraft prognosis and health management (PHM) which could provide better maintenance decisions. In order to improve the accuracy of aircraft engine RUL prediction under real flight conditions and better meet the needs of PHM system, we put forward an improved CNN-LSTM model based on the convolutional block attention module (CBAM). First, the features of aircraft engine operation data are extracted by multi-layer CNN network, and then the attention mechanism is processed by CBAM in channel and spatial dimensions to find key variables related to RUL. Finally, the hidden relationship between features and service time is learned by LSTM and the predicted RUL is output. Experiments were conducted using C-MPASS dataset. Experimental results indicate that our prediction model has feasibility. Compared with other state-of-the-art methods, the RMSE of our method decreased by 17.4%, and the score of the prediction model was improved by 25.9%.

Hand Activity Recognition From Automatic Estimated Egocentric Skeletons Combining Slow Fast and Graphical Neural Networks

In this paper, we present an unified framework for understanding hand action from the first-person video. The proposed framework composes two main components. The first component estimates three-dimensional (3D) hand joints from RGB images. Two network structures derived from the baseline HopeNet network are proposed: convolutional neural networks (CNNs) which are traditional multi-layer CNN and CNN combining with GraphCNN to perform 3D hand pose estimation, without the use of GraphUNet as in baseline HopeNet method. The second component of the framework recognizes hand action from skeleton stream. We first deploy two recent advanced neuronal networks that are PA-ResGCN and Double-feature Double-motion (DDNet). To focus more on the hand pose changes, we improve DDNet with two normalization strategies of the hand joints. Finally, we fuse PA-ResGCN with our improved DDNet to still boost the recognition performance. We evaluate our proposed methods on First-Person Hand Action Benchmark dataset. Experiments show that our model for 3D hand joints estimation achieves the best precision (36.6 mm). Our hand joint normalization strategies improve the original DDNet from 0.71% to 4.05% of accuracy with the ground-truth hand pose while the improvement is significantly larger (from 2.96% to 10.98%) with the estimated hand pose. The late fusion schemes outperform different state-of-the-art methods for the hand action recognition with the highest accuracy of 86.67%. These experimental results show potential and extendable possibilities for developing practical first-person vision applications.

Automatic BASED scoring on scalp EEG in children with infantile spasms using convolutional neural network.

In recent years, the Burden of Amplitudes and Epileptiform Discharges (BASED) score has been used as a reliable, accurate, and feasible electroencephalogram (EEG) grading scale for infantile spasms. However, manual EEG annotation is, in general, very time-consuming, and BASED scoring is no exception. Convolutional neural networks (CNNs) have proven their great potential in many EEG classification problems. However, very few research studies have focused on the use of CNNs for BASED scoring, a challenging but vital task in the diagnosis and treatment of infantile spasms. This study proposes an automatic BASED scoring framework using EEG and a deep CNN. The feasibility of using CNN for automatic BASED scoring was investigated in 36 patients with infantile spasms by annotating their long-term EEG data with four levels of the BASED score (scores 5, 4, 3, and ≤2). In the validation set, the accuracy was 96.9% by applying a multi-layer CNN to classify the EEG data as a 4-label problem. The extensive experiments have demonstrated that our proposed approach offers high accuracy and, hence, is an important step toward an automatic BASED scoring algorithm. To the best of our knowledge, this is the first attempt to use a CNN to construct a BASED-based scoring model.

DCSE:Double-Channel-Siamese-Ensemble model for protein protein interaction prediction

BackgroundProtein-protein interaction (PPI) is very important for many biochemical processes. Therefore, accurate prediction of PPI can help us better understand the role of proteins in biochemical processes. Although there are many methods to predict PPI in biology, they are time-consuming and lack accuracy, so it is necessary to build an efficiently and accurately computational model in the field of PPI prediction.ResultsWe present a novel sequence-based computational approach called DCSE (Double-Channel-Siamese-Ensemble) to predict potential PPI. In the encoding layer, we treat each amino acid as a word, and map it into an N-dimensional vector. In the feature extraction layer, we extract features from local and global perspectives by Multilayer Convolutional Neural Network (MCN) and Multilayer Bidirectional Gated Recurrent Unit with Convolutional Neural Networks (MBC). Finally, the output of the feature extraction layer is then fed into the prediction layer to output whether the input protein pair will interact each other. The MCN and MBC are siamese and ensemble based network, which can effectively improve the performance of the model. In order to demonstrate our model’s performance, we compare it with four machine learning based and three deep learning based models. The results show that our method outperforms other models in all evaluation criteria. The Accuracy, Precision, F_{1}, Recall and MCC of our model are 0.9303, 0.9091, 0.9268, 0.9452, 0.8609. For the other seven models, the highest Accuracy, Precision, F_{1}, Recall and MCC are 0.9288, 0.9243, 0.9246, 0.9250, 0.8572. We also test our model in the imbalanced dataset and transfer our model to another species. The results show our model is excellent.ConclusionOur model achieves the best performance by comparing it with seven other models. NLP-based coding method has a good effect on PPI prediction task. MCN and MBC extract protein sequence features from local and global perspectives and these two feature extraction layers are based on siamese and ensemble network structures. Siamese-based network structure can keep the features consistent and ensemble based network structure can effectively improve the accuracy of the model.

Measuring Waste Recyclability Level Using Convolutional Neural Network and Fuzzy Inference System

This paper presents a hybrid model that is used to measure the waste recyclability level using a convolutional neural network (CNN) and fuzzy inference system (FIS; WRL-CNNFIS). The proposed system uses waste images to train a multilayer convolutional neural network to extract the most relevant features that were used in a rule-based fuzzy system to give an accurate percentage of the recyclability level of these images. The proposed model did overcome many challenges in transfer learning models alone, like overfitting and low accuracy. The use of fuzzy rules improved the performance even with a small data set. Results have shown the effectiveness of the proposed model in terms of all four metrics: accuracy, precision, recall, and F1 score. The performance was measured under two testing scenarios. For all evaluation measurements in all experiments, the validation was conducted using the cross validation in the last step. The proposed approach is a robust and consistent approach for classifying organic and recyclable waste types. WRL-CNNFIS has achieved an accuracy rate of more than 98%.

High-Speed Multi-Layer Convolutional Neural Network Based on Free-Space Optics

Convolutional neural networks (CNNs) are at the heart of several machine learning applications, while they suffer from computational complexity due to their large number of parameters and operations. Recently, all-optical implementation of the CNNs has achieved many attentions, however, the recently proposed optical architectures for CNNs cannot fully utilize the tremendous capabilities of optical processing, due to the required electro-optical conversions in-between successive layers. To implement an all-optical multi-layer CNN, it is essential to optically implement all required operations, namely convolution, summation of channels’ output for each convolutional kernel feeding the nonlinear unit, nonlinear activation function, and finally, pooling operations. Considering the lack of multi-layer photonic CNN implementation, in this paper, we explore a fully-optical design for implementing successive convolutional layers in an optical CNN. As a proof of concept, and without loss of generality, we considered two successive optical layers in the proposed network, named as 2L-OPCNN, for comparative studies against electrical counterpart and single optical layer CNN. Our simulation results confirm nearly the same accuracies for classifying images of Kaggle Cats and Dogs challenge, CIFAR-10, and MNIST datasets, compared to the electrical counterpart, as well as improved accuracies compared to single optical layer CNN.

Image Captioning - A Deep Learning Approach

Abstract: Image captioning is a brand-new study area in the science of computer vision. The primary goal of picture captioning is to create a natural language description for the input image. In recent years, research on natural language processing and computer vision has become increasingly interested in the problem of automatically synthesising descriptive phrases for photos. Image captioning is a crucial task that demands both the ability to create precise and accurate description phrases as well as a semantic understanding of the images. Long Short Term Memory (LSTM) is used to precisely organise data using the available keywords to form meaningful sentences. The authors of this research propose a hybrid system based on multilayer Convolutional Neural Networks to create a lexicon for characterising the visuals. The convolutional neural network employs trained captions to deliver an accurate description after comparing the target image to a sizable dataset of training images. We demonstrate the effectiveness of our suggested methodology using the Flickr 8K datasets.