Convolutional Neural Network Framework Research Articles

A Non-Intrusive Load Monitoring Model for Electric Vehicles Based on Multi-Kernel Conventional Neural Network

With the widespread use of electric vehicles (EVs), the charging behavior of these resources has brought a large amount of load growth to the grid, leading to a series of problems such as increased peak valley load difference and line flow violation. Non-intrusive load monitoring (NILM) is a key technology that can be employed to monitor the multi-source load data information in the power grid and support the high-proportion access of electric vehicles. However, traditional NILM approaches are designed to identify the operation of household appliances and cannot be applied at the substation level directly due to frequent and intricate switching events of electrical equipment at this stage. In this paper, a NILM algorithm that can be applied for the monitoring of the charging behavior of electric vehicles at the substation level is proposed to support the high-proportion injection of distributed energy resources. The proposed approach employs a deep learning framework and a multi-kernel convolutional neural network (multi-kernel CNN) framework is used. The performance of the proposed method is verified on the self-organized datasets based on Pecan Street data and results showed that the obtained f1 score is over 90% for both the training sets and testing sets.

Recent advances in plant disease severity assessment using convolutional neural networks

In modern agricultural production, the severity of diseases is an important factor that directly affects the yield and quality of plants. In order to effectively monitor and control the entire production process of plants, not only the type of disease, but also the severity of the disease must be clarified. In recent years, deep learning for plant disease species identification has been widely used. In particular, the application of convolutional neural network (CNN) to plant disease images has made breakthrough progress. However, there are relatively few studies on disease severity assessment. The group first traced the prevailing views of existing disease researchers to provide criteria for grading the severity of plant diseases. Then, depending on the network architecture, this study outlined 16 studies on CNN-based plant disease severity assessment in terms of classical CNN frameworks, improved CNN architectures and CNN-based segmentation networks, and provided a detailed comparative analysis of the advantages and disadvantages of each. Common methods for acquiring datasets and performance evaluation metrics for CNN models were investigated. Finally, this study discussed the major challenges faced by CNN-based plant disease severity assessment methods in practical applications, and provided feasible research ideas and possible solutions to address these challenges.

BX2S-Net: Learning to reconstruct 3D spinal structures from bi-planar X-ray images

Grasping good understanding of the weight-bearing spatial structure of the spine of a human subject in a standing position is critical for the treatment of spinal disorders. Such disorders are commonly diagnosed via 2D X-ray imaging of the human subject in a standing position. However, 3D reconstruction techniques based on bi-planar X-ray imaging can enable better exploration and analysis of the spinal structure. In particular, compared to earlier deformable modeling approaches, the recently-developed deep-learning-based 3D reconstruction methods exhibit higher efficiency and generalizability. But these methods usually employ simple architectures with 2D encoders and 3D decoders. Thus, these methods have several drawbacks, namely, the existence of a semantic gap between dimensionally-inconsistent feature maps, the difficulty of jointly handling multi-view inputs, and the information source limitations for the decoding process. In order to better assist clinicians and tackle these problems, we propose a novel convolutional neural network framework, which we call BX2S-Net, to effectively achieve 3D spine reconstruction based on bi-planar X-ray images. In particular, a dimensionally-consistent encoder–decoder architecture is designed in conjunction with a dimensionality enhancement method in order to reduce the semantic gap between feature maps and achieve information fusion for multi-view inputs. A feature-guided progressive decoding process is developed on the decoder side, where a full-scale feature attention guidance (FFAG) module is introduced to efficiently aggregate image features and guide the decoding process at each level. In addition, a class augmentation method and a spatially-weighted cross-entropy loss function are used for network training with improved reconstruction quality for the vertebral edge region. The experimental results demonstrate the effectiveness of our model in reconstructing high-quality 3D spinal structures from bi-planar X-ray images. The code is available at https://github.com/NBU-CVMI/bx2s-net.

3D multi-view squeeze-and-excitation convolutional neural network for lung nodule classification.

Early screening is crucial to improve the survival rate and recovery rate of lung cancer patients. Computer-aided diagnosis system (CAD) is a powerful tool to assist clinicians in early diagnosis. Lung nodules are characterized by spatial heterogeneity. However, many attempts use the two-dimensional multi-view (MV) framework to learn and simply integrate multiple view features. These methods suffer from the problems of not capturing the spatial characteristics effectively and ignoring the variability of multiple views. In this paper, we propose a three-dimensional MV convolutional neural network (3D MVCNN) framework and embed the squeeze-and-excitation (SE) module in it to further address the variability of each view in the MV framework. First, the 3D multiple view samples of lung nodules are extracted by the spatial sampling method, and a 3D CNN is established to extract 3D abstract features. Second, build a 3D MVCNN framework according to the 3D multiple view samples and 3D CNN. This framework can learn more features of different views of lung nodules, taking into account the characteristics of spatial heterogeneity of lung nodules. Finally, to further address the variability of each view in the MV framework, a 3D MVSECNN model is constructed by introducing a SE module in the feature fusion stage. For training and testing purposes we used independent subsets of the public LIDC-IDRI dataset. For the LIDC-IDRI dataset, this study achieved 96.04% accuracy and 98.59% sensitivity in the binary classification, and 87.76% accuracy in the ternary classification, which was higher than other state-of-the-art studies. The consistency score of 0.948 between the model predictions and pathological diagnosis was significantly higher than that between the clinician's annotations and pathological diagnosis. The results show that our proposed method can effectively learn the spatial heterogeneity of nodules and solve the problem of multiple view variability. Moreover, the consistency analysis indicates that our method can provide clinicians with more accurate results of benign-malignant lung nodule classification for auxiliary diagnosis, which is important for assisting clinicians in clinical diagnosis.

I-VITAL: Information aided visual tracking with adversarial learning

With the advent of convolutional neural networks (CNN), MDNet and the Siamese trackers posed tracking as supervised learning. They model an object’s presence using classification (foreground and background) and location using regression. For the first time, we have brought probability distribution into the CNN framework for tracking. We have selected “Information maximization Generative Adversarial Network (InfoGAN)” to couple the target and background classes with two unique Gaussian distributions. This paper highlights the use of InfoGAN in information extraction & feedback to improve the tracking framework. Specifically, the novel features proposed in this tracking framework are (i) Coupling of unique probability distributions to target and background classes and (ii) Unsupervised tracker status (success/ failure) identification and correction through information feedback. We demonstrated the efficacy of the proposed I-VITAL tracker in visual tracking with experimental comparisons on well-known data sets such as GOT10K, VOT2020, and OTB-2015. Compared with base works, the proposed tracker has improved performance in locating the object of interest.

Deep Learning Architecture for UAV Traffic-Density Prediction

The research community has paid great attention to the prediction of air traffic flows. Nonetheless, research examining the prediction of air traffic patterns for unmanned aircraft traffic management (UTM) is relatively sparse at present. Thus, this paper proposes a one-dimensional convolutional neural network and encoder-decoder LSTM framework to integrate air traffic flow prediction with the intrinsic complexity metric. This adapted complexity metric takes into account the important differences between ATM and UTM operations, such as dynamic flow structures and airspace density. Additionally, the proposed methodology has been evaluated and verified in a simulation scenario environment, in which a drone delivery system that is considered essential in the delivery of COVID-19 sample tests, package delivery services from multiple post offices, an inspection of the railway infrastructure and fire-surveillance tasks. Moreover, the prediction model also considers the impacts of other significant factors, including emergency UTM operations, static no-fly zones (NFZs), and variations in weather conditions. The results show that the proposed model achieves the smallest RMSE value in all scenarios compared to other approaches. Specifically, the prediction error of the proposed model is 8.34% lower than the shallow neural network (on average) and 19.87% lower than the regression model on average.

Toward a generalizable deep CNN for neural drive estimation across muscles and participants

Objective. High-density electromyography (HD-EMG) decomposition algorithms are used to identify individual motor unit (MU) spike trains, which collectively constitute the neural code of movements, to predict motor intent. This approach has advanced from offline to online decomposition, from isometric to dynamic contractions, leading to a wide range of neural-machine interface applications. However, current online methods need offline retraining when applied to the same muscle on a different day or to a different person, which limits their applications in a real-time neural-machine interface. We proposed a deep convolutional neural network (CNN) framework for neural drive estimation, which takes in frames of HD-EMG signals as input, extracts general spatiotemporal properties of MU action potentials, and outputs the number of spikes in each frame. The deep CNN can generalize its application without retraining to HD-EMG data recorded in separate sessions, muscles, or participants. Approach. We recorded HD-EMG signals from the vastus medialis and vastus lateralis muscles from five participants while they performed isometric contractions during two sessions separated by ∼20 months. We identified MU spike trains from HD-EMG signals using a convolutive blind source separation (BSS) method, and then used the cumulative spike train (CST) of these MUs and the HD-EMG signals to train and validate the deep CNN. Main results. On average, the correlation coefficients between CST from the BSS and that from deep CNN were for leave-one-out across-sessions-and-muscles validation and for leave-one-out across-participants validation. When trained with more than four datasets, the performance of deep CNN saturated at for cross validations across muscles, sessions, and participants. Significance. We can conclude that the deep CNN is generalizable across the aforementioned conditions without retraining. We could potentially generate a robust deep CNN to estimate neural drive to muscles for neural-machine interfaces.

Lightweight tomato real-time detection method based on improved YOLO and mobile deployment

The current deep-learning-based tomato target detection algorithm has many parameters; it has drawbacks of large computation, long time consumption, and reliance on high-computing-power devices such as graphics processing units (GPU). In this study, we propose a lightweight improved YOLOv5 (You Only Look Once) based algorithm to achieve real-time localization and ripeness detection of tomato fruits. Initially, this algorithm used a down-sampling convolutional layer instead of the original focus layer, reconstructing the backbone network of YOLOv5 using the bneck module of MobileNetV3. Then, it performs channel pruning for the neck layer to further reduce the model size and uses a genetic algorithm for hyperparameter optimization to improve detection accuracy. We evaluate the improved algorithm using a homemade tomato dataset. The experimental results demonstrated that the improved model number of parameters and floating point operations per second (FLOPs) were compressed by 78% and 84.15% compared to the original YOLOv5s, while the mAP reached 0.969. Meanwhile, the detection speed on the central processing unit (CPU) platform was 42.5 ms, which was 64.88% better. This study further utilized the Nihui convolutional neural network (NCNN) framework to quantize the improved model and developed an Android-based real-time tomato monitoring application (app). Experimental results demonstrated that the 16-bit quantized model achieved an average detection frame rate of 26.5 frames per second (fps) on the mobile side with lower arithmetic power, which was 268% better than the original YOLOv5s, and the model size was reduced by 51.1% while achieving a 93% true detection rate.

A decomposition-based hybrid ensemble CNN framework for driver fatigue recognition

Electroencephalogram (EEG) has become increasingly popular in driver fatigue monitoring systems. Several decomposition methods have been attempted to analyze the EEG signals that are complex, nonlinear and non-stationary and improve the EEG decoding performance in different applications. However, it remains challenging to extract more distinguishable features from different decomposed components for driver fatigue recognition. In this work, we propose a novel decomposition-based hybrid ensemble convolutional neural network (CNN) framework to enhance the capability of decoding EEG signals. Four decomposition methods are employed to disassemble the EEG signals into components of different complexity. Instead of handcraft features, the CNNs in this framework directly learn from the decomposed components. In addition, a component-specific batch normalization layer is employed to reduce subject variability. Moreover, we employ two ensemble modes to integrate the outputs of all CNNs, comprehensively exploiting the diverse information of the decomposed components. Against the challenging cross-subject driver fatigue recognition task, the models under the framework all showed better performance than the strong baselines. Specifically, the performance of different decomposition methods and ensemble modes was further compared. The results indicated that discrete wavelet transform-based ensemble CNN achieved the highest average classification accuracy of 83.48% among the compared methods. The proposed framework can be extended to any CNN architecture and be applied to any EEG-related tasks, opening the possibility of extracting more beneficial features from complex EEG data.

Enhanced E-commerce Fraud Prediction Based on a Convolutional Neural Network Model

The rapidly escalating sophistication of e-commerce fraud in recent years has led to an increasing reliance on fraud detection methods based on machine learning. However, fraud detection methods based on conventional machine learning approaches suffer from several problems, including an excessively high number of network parameters, which decreases the efficiency and increases the difficulty of training the network, while simultaneously leading to network overfitting. In addition, the sparsity of positive fraud incidents relative to the overwhelming proportion of negative incidents leads to detection failures in trained networks. The present work addresses these issues by proposing a convolutional neural network (CNN) framework for detecting e-commerce fraud, where network training is conducted using historical market transaction data. The number of network parameters reduces via the local perception field and weight sharing inherent in the CNN framework. In addition, this deep learning framework enables the use of an algorithmic-level approach to address dataset imbalance by focusing the CNN model on minority data classes. The proposed CNN model is trained and tested using a large public e-commerce service dataset from 2018, and the test results demonstrate that the model provides higher fraud prediction accuracy than existing state-of-the-art methods.

Retracted: Coupled Attention Framework of Convolutional Neural Network Based on Computer Intelligence.

[This retracts the article DOI: 10.1155/2022/7859287.].

Deep Learning Framework of Convolutional Neural Network (CNN) and Attention CNN for Early Diagnosis of Alzheimer's Disease

One of the biggest killers in the industrialized world is Alzheimer's disease (AD). Although computer-aided techniques have shown promising outcomes in laboratory experiments, they have yet to be used in a clinical setting. Recently, deep neural networks have gained traction, particularly for image processing tasks. There has been a dramatic increase in the number of publications written on the topic of identifying AD using deep learning since 2017. It has been observed that deep networks are more efficient than standard machine learning methods for detecting AD. It remains difficult to identify AD because distinguishing between comparable brain signals during categorization needs an extremely discriminative depiction of features. This paper proposed a deep neural network method for prediction the AD. Low-level computer vision has been a hotspot for research into deep convolutional neural networks (CNNs). Studies often focus on enhancing performance through the use of very deep CNNs. Yet, as one goes deeper, the effect of the shallow layers on the deeper ones gradually diminishes. Prompted by reality. This paper compared with the CNN and attention CNN models. The proposed model applied in the AD dataset which contains 5121 images for the train set. The results showed the attention CNN model is better than the CNN model in accuracy, precision, recall, loss, and AUC.

Detection of Breast Cancer Based on Feature Extraction Using WPSO in Conjunction with CNN

According to cancer reports from the past few years in India, thirty percent of instances are breast cancer, and furthermore, it is possible that this percentage would increase in the near future. In addition, one woman is given a diagnosis every two minutes, and another woman passes away every nine minutes as a result of her condition. People who are diagnosed with cancer at an earlier stage have a better chance of survival. Micro calcifications are one of the most important symptoms to look out for when trying to diagnose breast cancer in its earlier stages. Several scientific investigations have been carried out in an effort to combat this illness, for which techniques related to machine learning can be utilized to a significant extent. Particle swarm optimization, often known as PSO, is acknowledged as one of several effective and promising methods for identifying breast cancer. This method helps medical professionals administer treatment that is both timely and appropriate. The weighted particle swarm optimization (WPSO) approach is utilized in this work for the purpose of extracting textural information from the segmented mammography picture for the purpose of classifying micro calcifications as normal, benign, or malignant, hence increasing the accuracy. A portion of the cancerous growth is removed from the breast region using optimizing techniques. In this article, Convolutional Neural Networks (CNNs) are presented for the purpose of identifying breast cancer in order to cut down on the amount of manual overhead. The CNN framework is built in order to extract features as effectively as possible. This algorithm was developed to identify areas in mammograms (MG) that are suspicious for cancer and to classify those areas as normal or abnormal as quickly as possible. This model makes use of MG pictures that were gathered from a variety of hospitals in the surrounding area.

DL4DS—Deep learning for empirical downscaling

Abstract A common task in Earth Sciences is to infer climate information at local and regional scales from global climate models. Dynamical downscaling requires running expensive numerical models at high resolution, which can be prohibitive due to long model runtimes. On the other hand, statistical downscaling techniques present an alternative approach for learning links between the large- and local-scale climate in a more efficient way. A large number of deep neural network-based approaches for statistical downscaling have been proposed in recent years, mostly based on convolutional architectures developed for computer vision and super-resolution tasks. This paper presents deep learning for empirical downscaling (DL4DS), a python library that implements a wide variety of state-of-the-art and novel algorithms for downscaling gridded Earth Science data with deep neural networks. DL4DS has been designed with the goal of providing a general framework for training convolutional neural networks with configurable architectures and learning strategies to facilitate the conduction of comparative and ablation studies in a robust way. We showcase the capabilities of DL4DS on air quality Copernicus Atmosphere Monitoring Service (CAMS) data over the western Mediterranean area. The DL4DS library can be found in this repository: https://github.com/carlos-gg/dl4ds

Enhancing remote sensing satellite image classification through parameter optimization in hybrid CNN-LSTM model

Satellite imagery is a vital resource in Earth observation, offering valuable insights for applications ranging from environmental monitoring to disaster response. In the pursuit of accurate and robust classification, we present a study focused on parameter optimization within the framework of hybrid Convolutional Neural Network (CNN) and Long Short-Term Memory (LSTM) models. This research seeks to maximize the potential of hybrid CNN-LSTM architecture, which inherently combine spatial feature extraction capabilities of CNNs with LSTM’s ability to model temporal dependencies. We delve into a systematic exploration of hyper-parameters, including the number of CNN layers, LSTM units, learning rates, and dropout rates, fine-tuning them to achieve peak classification performance. These parameters are meticulously adjusted and validated on a diverse and representative satellite image dataset. Beyond hyper-parameter optimization, we address the challenge of class imbalance frequently encountered in satellite image datasets. By incorporating specialized techniques during the optimization process, we mitigate class imbalance effects and bolster the model’s accuracy. The study’s findings not only unveil optimal parameter configurations but also shed light on the intricate interplay between architecture complexity and model performance. The resulting optimized hybrid model demonstrates exceptional classification accuracy, validating its applicability to real-world remote sensing tasks.

Instance-Aware Deep Graph Learning for Multi-Label Classification

Graph convolutional neural network (GCN) has effectively boosted the multi-label image recognition task by modeling correlation among labels. In previous methods, label correlation is computed based on statistical information through label diffusion, and therefore the same for all samples. This, however, makes graph inference on labels insufficient to handle huge variations among numerous image instances. In this paper, we propose an instance-aware graph convolutional neural network (IA_GCN) framework for the multi-label classification. As a whole, two fused branches of sub-networks are involved in the framework: a global branch modeling the whole image and a local branch exploring dependencies among regions of interests (ROIs). For both the branches, an image-dependent label correlation matrix (ID_LCM), fusing both the statistical label correlation matrix (LCM) and an individual one of each image instance, is constructed to inject adaptive information of label-awareness into the learned features of the model through graph convolution. Specifically, the individual LCM of each image is obtained by mining the label dependencies based on the predicted label scores of those detected ROIs. In this process, considering the contribution differences of ROIs to multi-label classification, variational inference is introduced to learn adaptive scaling factors for those ROIs by considering their complex distribution. Finally, extensive experiments on MS-COCO and VOC datasets show that our proposed approach outperforms existing state-of-the-art methods.

A Hybrid CRNN Model for Multi-Class Violence Detection in Text and Video

Gender-based violence is a critical issue that not only poses a threat to physical safety but also has significant impacts on mental health. Shockingly, up to 1 billion children aged 2-17 years are estimated to have experienced gender-based violence globally, making it a pressing concern for the machine learning and deep learning communities to address. To end this, a novel approach has been proposed in the form of a Convolutional Neural Network and bi-directional LSTM (CRNN) to classify three types of violence present in both text and video data, thereby making the internet a safer space for individuals. The proposed approach utilises two datasets consisting of 400 and 600 samples each for videos and text, respectively, to improve the precision and accuracy of the model. The use of a Convolutional Recurrent Neural Network framework combined with LSTM layers has resulted in an accuracy of 97% on text and 96% on videos, surpassing the performance of existing RNN models. Additionally, the inclusion of dropout and regularizer layers has helped the model avoid overfitting and generalise better on unseen data. Overall, the CRNN-based approach presents a promising solution to the problem of gender-based violence detection, with the potential to significantly improve the safety of individuals online. By leveraging the power of machine learning and deep learning, we can contribute towards creating a safer and more equitable world for all.

Diagnosis of lung and colon cancer based on clinical pathology images using convolutional neural network and CLAHE framework

Diagnosis of lung and colon cancer based on clinical pathology images using convolutional neural network and CLAHE framework

TetCNN: Convolutional Neural Networks on Tetrahedral Meshes.

Convolutional neural networks (CNN) have been broadly studied on images, videos, graphs, and triangular meshes. However, it has seldom been studied on tetrahedral meshes. Given the merits of using volumetric meshes in applications like brain image analysis, we introduce a novel interpretable graph CNN framework for the tetrahedral mesh structure. Inspired by ChebyNet, our model exploits the volumetric Laplace-Beltrami Operator (LBO) to define filters over commonly used graph Laplacian which lacks the Riemannian metric information of 3D manifolds. For pooling adaptation, we introduce new objective functions for localized minimum cuts in the Graclus algorithm based on the LBO. We employ a piece-wise constant approximation scheme that uses the clustering assignment matrix to estimate the LBO on sampled meshes after each pooling. Finally, adapting the Gradient-weighted Class Activation Mapping algorithm for tetrahedral meshes, we use the obtained heatmaps to visualize discovered regions-of-interest as biomarkers. We demonstrate the effectiveness of our model on cortical tetrahedral meshes from patients with Alzheimer's disease, as there is scientific evidence showing the correlation of cortical thickness to neurodegenerative disease progression. Our results show the superiority of our LBO-based convolution layer and adapted pooling over the conventionally used unitary cortical thickness, graph Laplacian, and point cloud representation.

Food Image Classification and Data Extraction Using Convolutional Neural Network and Web Crawlers

Food image recognition is one among the various propitious applications in the area of computer vision. An application with the ability to identify all kinds of food images along with its nutritious value will help people in maintaining a balanced diet. The proposed convolutional neural network (CNN) model can be used for the identification and classification of food images. A pre-trained Inception v3 CNN model is employed via transfer learning to galvanize the original custom-made CNN framework. With the aid of this pre-trained model, the learning process is boosted and is hence more efficient. Data augmentation is performed on the training set as it improves the robustness of the model and as it also helps avoid overfitting. The predicted food label generated by the model is forwarded to the web crawlers for information retrieval. The web crawlers are deployed on the browser through automation for the retrieval of relevant information such as the food item's origin, nutritional details, its recipe, and even the nearby restaurants that serve the dish. The crawlers are built using Python Scrapy and the process of scraping data from the websites is automated through Selenium. The model achieved an accuracy of 97.00% for 20 classes. This model can be further enhanced in terms of scalability by building a deeper more advanced neural network, collecting more images per class for each of the respective food items and by fine-tuning the model hyperparameters.