Conventional CNN Research Articles

A wavelet CNN with appropriate feed-allocation and PSO optimized activations for diabetic retinopathy grading

This work modifies the architecture of conventional CNN with the integration of Multi-resolution Analysis (MRA) in a CNN framework for Diabetic Retinopathy (DR) diagnosis and grading. Here, the HF sub-bands are subjected to optimized activations and are directly fed to the fully connected layers, as it encompasses edge features. Unlike FD-Relu, the proposed function preserves significant negative coefficients, compared to the S-Relu, the proposed third-order S-Relu is optimized such that it sustains the activations in the range suitable for the wavelet coefficients. The coefficients of higher-order terms of the proposed 3rd-order S-Relu are optimized with PSO, fitting the maximum energy of the wavelet sub-bands to ensure High Frequency (HF) edge preservation. The authors re-architecture 3 different CNNs published in the Retinal Image analysis field, with spatial and wavelet inputs with optimized activations. The highest accuracy of 96% is attained with the AlexNet re-architecture, with 35,126 fundus images secured from the Kaggle dataset. As we can infer the proposed re-architecture wavelet CNN outperformed the multiscale shallow CNNs, multiscale attention net, and stacked CNNs with a 6.6, 0.3, 0.7 per cent increase in accuracy. The entire implementation of the wavelet CNN is made available under source code.

IEA-Net: Internal and External Dual-Attention Medical Segmentation Network with High-Performance Convolutional Blocks.

Currently, deep learning is developing rapidly in the field of image segmentation, and medical image segmentation is one of the key applications in this field. Conventional CNN has achieved great success in general medical image segmentation tasks, but it has feature loss in the feature extraction part and lacks the ability to explicitly model remote dependencies, which makes it difficult to adapt to the task of human organ segmentation. Although methods containing attention mechanisms have made good progress in the field of semantic segmentation, most of the current attention mechanisms are limited to a single sample, while the number of samples of human organ images is large, ignoring the correlation between the samples is not conducive to image segmentation. In order to solve these problems, an internal and external dual-attention segmentation network (IEA-Net) is proposed in this paper, and the ICSwR (interleaved convolutional system with residual) module and the IEAM module are designed in this network. The ICSwR contains interleaved convolution and hopping connection, which are used for the initial extraction of the features in the encoder part. The IEAM module (internal and external dual-attention module) consists of the LGGW-SA (local-global Gaussian-weighted self-attention) module and the EA module, which are in a tandem structure. The LGGW-SA module focuses on learning local-global feature correlations within individual samples for efficient feature extraction. Meanwhile, the EA module is designed to capture inter-sample connections, addressing multi-sample complexities. Additionally, skip connections will be incorporated into each IEAM module within both the encoder and decoder to reduce feature loss. We tested our method on the Synapse multi-organ segmentation dataset and the ACDC cardiac segmentation dataset, and the experimental results show that the proposed method achieves better performance than other state-of-the-art methods.

Enhanced heart disease prediction through hybrid CNN-TLBO-GA optimization: a comparative study with conventional CNN and optimized CNN using FPO algorithm

Enhanced heart disease prediction through hybrid CNN-TLBO-GA optimization: a comparative study with conventional CNN and optimized CNN using FPO algorithm

Bidirectional Hybrid LSTM Based Recurrent Neural Network for Multi-View Stereo.

Recently, deep learning based multi-view stereo (MVS) networks have demonstrated their excellent performance on various benchmarks. In this paper, we present an effective and efficient recurrent neural network (RNN) for accurate and complete dense point cloud reconstruction. Instead of regularizing the cost volume via conventional 3D CNN or unidirectional RNN like previous attempts, we adopt a bidirectional hybrid Long Short-Term Memory (LSTM) based structure for cost volume regularization. The proposed bidirectional recurrent regularization is able to perceive full-space context information comparable to 3D CNNs while saving runtime memory. For post-processing, we introduce a visibility based approach for depth map refinement to obtain more accurate dense point clouds. Extensive experiments on DTU, Tanks and Temples and ETH3D datasets demonstrate that our method outperforms previous state-of-the-art MVS methods and exhibits high memory efficiency at runtime.

Leukemia Detection using Machine Learning Algorithms: Current trends and future directions

The process of medical diagnosis of leukemia cases is a complex process and requires diligent efforts. It is done by examining samples under a microscope and distinguishing the number, shape, size, and morphological features of white blood cells. This process takes a long time to predict leukemia. The professional skills and experience of the pathologist may also influence this procedure as a large number of overlapping structures and conditions, distractions, fatigue, and limitations in the human visual system can lead to an inappropriate diagnosis. Machine and deep learning approaches have been found to be effective strategies for enhancing the precision and efficiency of diagnosis and classification of medical images, including those of microscopic blood cells. In this paper, we will review the recent studies of leukemia detection and/or classification in the period of (2015 - 2023) in machine and deep learning and the combining of them. The review of these schemes is carried out in a systematic manner. In order to achieve the desired objective, segmentation schemes can be classified into four main categories: supervised machine learning techniques, unsupervised machine learning techniques, deep learning approaches, and traditional image processing techniques. Classification approaches can be categorized into three main groups: classical machine learning, deep learning, and a combination of both. CNN-based categorization systems can be further classified into three categories: conventional CNN, transfer learning, and additional developments in CNN. This paper also provides a concise analysis of these strategies and their significance in the categorization of leukemia. Ultimately, a comprehensive examination is conducted to elucidate the current state of research in this particular domain.

Leukemia Detection using Machine Learning Algorithms: Current trends and future directions

The process of medical diagnosis of leukemia cases is a complex process and requires diligent efforts. It is done by examining samples under a microscope and distinguishing the number, shape, size, and morphological features of white blood cells. This process takes a long time to predict leukemia. The professional skills and experience of the pathologist may also influence this procedure as a large number of overlapping structures and conditions, distractions, fatigue, and limitations in the human visual system can lead to an inappropriate diagnosis. Machine and deep learning approaches have been found to be effective strategies for enhancing the precision and efficiency of diagnosis and classification of medical images, including those of microscopic blood cells. In this paper, we will review the recent studies of leukemia detection and/or classification in the period of (2015 - 2023) in machine and deep learning and the combining of them. The review of these schemes is carried out in a systematic manner. In order to achieve the desired objective, segmentation schemes can be classified into four main categories: supervised machine learning techniques, unsupervised machine learning techniques, deep learning approaches, and traditional image processing techniques. Classification approaches can be categorized into three main groups: classical machine learning, deep learning, and a combination of both. CNN-based categorization systems can be further classified into three categories: conventional CNN, transfer learning, and additional developments in CNN. This paper also provides a concise analysis of these strategies and their significance in the categorization of leukemia. Ultimately, a comprehensive examination is conducted to elucidate the current state of research in this particular domain.

DeepGAN: Utilizing generative adversarial networks for improved deep learning

In the realm of deep learning, Generative Adversarial Networks (GANs) have emerged as a topic of significant interest for their potential to enhance model performance and enable effective data augmentation. This paper addresses the existing challenges in synthesizing high-quality data and harnessing the capabilities of GANs for improved deep learning outcomes. Unlike traditional approaches that heavily rely on manually engineered data augmentation techniques, our work introduces a novel framework that leverages DeepGANs to autonomously generate diverse and high-fidelity data. Our experiments encompass a diverse spectrum of datasets, including images, text, and time series data. In the context of image classification tasks, we conduct experiments on the widely recognized CIFAR-10 dataset, which consists of 50,000 image samples. Our results demonstrate the remarkable efficacy of DeepGANs in enhancing model performance across various data domains. Notably, in image classification using the CIFAR-10 dataset, our innovative approach achieves an impressive accuracy of 97.2%. This represents a substantial advancement beyond conventional CNN models, underscoring the profound impact of DeepGANs in the realm of deep learning. In summary, this research sheds light on DeepGANs as a fundamental component in the pursuit of enhanced deep learning performance. Our framework not only overcomes existing limitations but also heralds a new era of data augmentation, with generative adversarial networks leading the way. The attainment of an accuracy rate of 97.2% on CIFAR-10 serves as a compelling testament to the transformative potential of DeepGANs, solidifying their pivotal role in the future of deep learning. This promises the development of more robust, adaptive, and accurate models across a myriad of applications, marking a significant contribution to the field.

Learning Chebyshev neural network-based spacecraft attitude tracking control ensuring finite-time prescribed performance

This article presents a finite-time prescribed performance (FTPP) control approach based on a learning Chebyshev neural network (LCNN) for spacecraft attitude tracking with modeling uncertainties, actuator faults, and external disturbances. An FTPP function is designed to specify the desired accuracy boundary and finite-time convergence. Further, an FTPP-based learning sliding mode controller (LSMC) is constructed, where the lumped disturbance is approximated and compensated via a novel LCNN model. Unlike conventional adaptive CNN models, the LCNN model employs an iterative learning mechanism for adjusting the weights of the CNN model, reducing computing costs. The FTPP-based LSMC approach is presented with a detailed stability analysis. The proposed method offers a broad range of applications with the FTPP criteria satisfied. A series of simulations are performed to verify the validity and applicability of the proposed approach.

Greater accuracy of radiomics compared to deep learning to discriminate normal subjects from patients with dementia: a whole brain 18FDG PET analysis.

18F-FDG brain PET and clinical score were collected in 85 patients with dementia and 125 healthy controls (HC). Patients were assigned to various form of dementia on the basis of clinical evaluation, follow-up and voxels comparison with HC using a two-sample Student's t -test, to determine the regions of brain involved. Radiomic analysis was performed on the whole brain after normalization to an optimized template. After feature selection using the minimum redundancy maximum relevance method and Pearson's correlation coefficients, a Neural Network model was tested to find the accuracy to classify HC and demented patients. Twenty subjects not included in the training were used to test the models. The results were compared with those obtained by conventional CNN model. Four radiomic features were selected. The validation and test accuracies were 100% for both models, but the probability scores were higher with radiomics, in particular for HC group ( P = 0.0004). Radiomic features extracted from standardized PET whole brain images seem to be more accurate than CNN to distinguish patients with and without dementia.

Nondestructive detection of fiber content in steel fiber reinforced concrete through percussion method coordinated with a hybrid deep learning network

Accurate inspection of the steel fiber content is necessary to assure the designed performance and required quality of steel fiber reinforced concrete (SFRC). This paper presents a percussion-based nondestructive methodology in detecting the steel fiber content in SFRC components, incorporating a deep neural network capable of intelligent detection and automated classification of steel fiber content without the need for signal transformation or manual feature extraction. This research makes several noteworthy contributions. First of all, the percussion acoustic signals were captured on a smartphone from SFRC cubes with varying fiber volumes in a range of 0–2%. Subsequently, a hybrid deep learning framework named 1D CNN-BiLSTM, which harnesses advantages from both the one-dimensional conventional neural network (1D CNN) and the bidirectional long short-term memory (BiLSTM) network, was established and trained to classify these acoustic signals labeled with specific fiber contents. The results well manifest the excellent performances of the proposed network in classifying the steel fiber content, achieving a testing accuracy of 98%, which outperforms conventional machine learning methods, 1D CNN and 1D CNN-LSTM. Moreover, the noise resilience and adaptability of 1D CNN-BiLSTM were also greatly enhanced in contrast to these baseline methods. The testing accuracy exceeds 90.0% under strong noisy conditions in an SNR range of 2–10 dB. In addition, the trained model also presents an exceptional adaptability to identifying previously unseen signals from SFRC beams, yielding an accuracy of 90.34%.

Empowering edge devices: FPGA‐based 16‐bit fixed‐point accelerator with SVD for CNN on 32‐bit memory‐limited systems

AbstractConvolutional neural networks (CNNs) are now often used in deep learning and computer vision applications. Its convolutional layer accounts for most calculations and should be computed fast in a local edge device. Field‐programmable gate arrays (FPGAs) have been adequately explored as promising hardware accelerators for CNNs due to their high performance, energy efficiency, and reconfigurability. This paper developed an efficient FPGA‐based 16‐bit fixed‐point hardware accelerator unit for deep learning applications on the 32‐bit low‐memory edge device (PYNQ‐Z2 board). Additionally, singular value decomposition is applied to the fully connected layer for dimensionality reduction of weight parameters. The accelerator unit was designed for all five layers and employed eight processing elements in convolution layers 1 and 2 for parallel computations. In addition, array partitioning, loop unrolling, and pipelining are the techniques used to increase the speed of calculations. The AXI‐Lite interface was also used to communicate between IP and other blocks. Moreover, the design is tested with grayscale image classification on MNIST handwritten digit dataset and color image classification on the Tumor dataset. The experimental results show that the proposed accelerator unit implementation performs faster than the software‐based implementation. Its inference speed is 89.03% more than INTEL 3‐core CPU, 86.12% higher than Haswell 2‐core CPU, and 82.45% more than NVIDIA Tesla K80 GPU. Furthermore, the throughput of the proposed design is 4.33GOP/s, which is better than the conventional CNN accelerator architectures.

Bionic study of distance-azimuth discrimination of multi-scattered point objects in bat bio-sonar

This paper presents a novel approach to enhance the discrimination capacity of multi-scattered point objects in bat bio-sonar. A broadband interferometer mathematical model is developed, incorporating both distance and azimuth information, to simulate the transmitted and received signals of bats. The Fourier transform is employed to simulate the preprocessing step of bat information for feature extraction. Furthermore, the bat bio-sonar model based on convolutional neural network (BS-CNN) is constructed to compensate for the limitations of conventional machine learning and CNN networks, including three strategies: Mix-up data enhancement, joint feature and hybrid atrous convolution module. The proposed BS-CNN model emulates the perceptual nerves of the bat brain for distance-azimuth discrimination and compares with four conventional classifiers to assess its discrimination efficacy. Experimental results demonstrate that the overall discrimination accuracy of the BS-CNN model is 93.4%, surpassing conventional CNN networks and machine learning methods by at least 5.9%. This improvement validates the efficacy of the BS-CNN bionic model in enhancing the discrimination accuracy in bat bio-sonar and offers valuable references for radar and sonar target classification.

Informatization Exploration of Wushu Teaching Management Platform in Colleges and Universities under the Concept of Modern Education

Abstract In the evolving landscape of higher education, integrating information technology into Wushu teaching management presents an opportunity to elevate teaching standards. This paper investigates the development of an innovative platform for Wushu instruction, utilizing cutting-edge video analysis techniques to enhance learning outcomes. By analyzing video data from Hikvision DS-2CD2051-1 webcams with advanced preprocessing methods, we demonstrate the effectiveness of the Single Shot MultiBox Detector (SSD) algorithm in improving action recognition over conventional CNN approaches. Our results reveal a notable enhancement in the quality of Wushu movement evaluation, evidenced by a 72.02% accuracy rate and a 0.3865 Mean Squared Error (MSE). The deployment of this technology in a university setting not only fosters a significant uplift in student performance but also boosts their engagement and satisfaction levels. This paper contributes to the discourse on educational technology by offering a successful model for the digital Transformation of Wushu teaching in academic institutions.

Optimized CNN Using Manta-Ray Foraging Optimization for Brain Tumour Detection

Brain tumors pose a significant medical challenge, demanding early detection and diagnosis to enhance patient outcomes. However, existing brain tumor detection methods often suffer from inefficiency and inaccuracy. Presently, deep learning models have gained prominence in classifying brain MRI images due to their superior accuracy compared to traditional classifiers. Nonetheless, optimizing hyperparameters manually to achieve precise image classification is a daunting and time-consuming task. To address these challenges, we present a novel approach in this paper—the Manta Ray Foraging Optimized Convolutional Neural Network (MRFO-CNN) model to classify brain MRIs into tumorous and non-tumorous categories. We leverage the Manta Ray Foraging Optimization technique to automatically determine the optimal hyperparameters, specifically batch size and epoch settings. Our model is trained on a dataset comprising 3000 brain MR images and validated on an additional 351 brain MR images. The results unequivocally demonstrate the superiority of our proposed MRFO-CNN model over conventional CNN approaches, achieving an impressive training accuracy of 99.3% and a validation accuracy of 98.7%. This research not only showcases the potential of deep learning in brain tumours classification but also underscores the efficiency and effectiveness of automated hyperparameter optimization in medical image analysis.

Distracted driver behavior recognition using modified capsule networks

Human activity recognition (HAR) is an increasingly active study field within the computer vision community. In HAR, driver behavior can be detected to ensure safe travel. Detect driver behaviors using a capsule network with leave-one-subject-out validation. The study was done using CapsNet with leave-one-subject-out validation to identify driving habits. The proposed method in this study consists of two parts, namely encoder and decoder. The encoder used in this study modifies Sabour’s capsule network architecture by adding a convolution layer before going to the primary capsule layer. The proposed method is evaluated using a primary dataset with 10 classes and 300 images for each class. The dataset is split based on hold-out validation and leave-one-subject-out validation. The resulting models were then compared to conventional CNN architecture. The objective of the research is to identify driving behavior. In this study, the proposed method results an accuracy rate of 97.83 % in the split dataset using hold-out validation. However, the accuracy decreased by 53.11 % when the proposed method was used on a split dataset using leave-one-subject-out validation. This is because the proposed method extracts all features including the attributes of each participant contained in the input image (user-independent). Thus, the resulting model in this study tends to overfit.

A surface electromyography based hand gesture recognition framework leveraging variational mode decomposition technique and deep learning classifier

Deciphering and classifying surface electromyography (sEMG) signals is highly essential in rehabilitation robotics, myoelectric prosthetic control, sign languages and human–computer-interface. Researchers have generally focused on utilizing multi-channel sEMG signals for gesture classification. However, in case of amputees, the residual muscles are limited and the gesture classification needs to be reframed with minimum sEMG channels. Moreover, another key problem in the gesture classification system is the nonlinear and non-stationary nature of sEMG signals, leading to poor generalization ability. Hence, to address the aforementioned problems, this study puts forward a novel deep learning classifier framework which leverages the potentials of variational mode decomposition (VMD) technique and a hybrid convolutional neural network–long short term memory (CNN–LSTM) classifier model to recognize the hand gestures from single channel sEMG signals. Collecting the sEMG signals from forearm muscles of 25 intact subjects for ten functional and grasping actions, this work implements a VMD technique to identify the prominent frequency modes in the sEMG signals. From the decomposed modes of sEMG signals, the prominent intrinsic mode functions (IMFs) are extracted through spectral analysis to minimize the computation burden on the hybrid classifier model. Furthermore, as the muscle contractions result in substantial temporal dependencies, this work exploits the potentials of CNN and LSTM networks and extracts the spatiotemporal features of the sEMG signals for various hand gestures. The experimental results corroborate that the proposed classifier framework can achieve an average classification accuracy of 98.04% and provide 3% improvement in the classification accuracy compared to conventional CNN classifier.

A lightweight convolutional neural network with end-to-end learning for three-dimensional mineral prospectivity modeling: A case study of the Sanhetun Area, Heilongjiang Province, Northeastern China

With the continuous exploitation of surface and shallow mineral resources, the global demand for concealed ore deposit exploration is increasing. However, concealed mineral prospectivity modeling (MPM) requires significant efforts, particularly in terms of methods and technologies for three-dimensional modeling. In this study, we propose a lightweight three-dimensional convolutional neural network (3D CNN) for MPM, which adopts the inception structure of GoogleNet and combines the idea of end-to-end learning. We replace the fully connected layer in the conventional CNN with deconvolution, which can greatly reduce the required parameters during the training process and accelerate the convergence. The proposed method overcomes the disadvantage that other shallow machine learning methods cannot extract spatial neighborhood information, while it can extract cross-correlations among geological factors and generates less parameters by a lightweight network when facing massive data. Additionally, compared with the patch-wise method commonly used in previous studies, we use the pixel-wise method for end-to-end learning, which not only overcomes the drawbacks of random sampling but also considers the influence of each voxel when calculating the loss function. The three-dimensional multi-source geoscientific characteristics obtained from the geophysical inversion and 3D geological models are not discretized in order to promote effective CNN training while facilitating the ore-controlling representation. Comparing the predicted results between the 3D weight of evidence (WofE) and our proposed 3D CNN method for MPM, our proposed method and WofE delineated 100% of the known mineralization in high-favorability areas with voxel numbers of 70% and 95%, respectively. A case study of a structure-controlled hydrothermal gold deposit in the Sanhetun area of Heilongjiang Province demonstrates that the proposed 3D CNN method performs better than WofE in terms of prediction effectiveness and efficiency and effectively reveals the correlation between mineralization and adjacent ore-controlling characteristics. Moreover, the proposed 3D CNN method can simulate non-linear metallogenic processes and mine hidden relationships to reveal complex ore-controlling characteristics. In conclusion, the proposed 3D CNN method can reduce the exploration effort in 3D MPM, thus greatly improving the efficiency of discovering concealed ore deposits.

Assessment of hybrid transfer learning method for forecasting EV profile and system voltage using limited EV charging data

The number of electric vehicles (EV) is increasing exponentially, significantly affecting the planning and operation of future electricity grids, albeit the availability of EV data is very limited to perform power system studies. To overcome the poor forecasting accuracy associated with limited available data, this research proposes a time-series-based hybrid transfer learning forecasting approach, namely CNN-BiLSTM (Convolutional Neural Network – Bidirectional Long Short-Term Memory) to forecast EV charging profile. Additionally, the proposed algorithm has been applied to forecast network voltage from the EV data without performing power flow. EV charging demand datasets collected over a year for residential, slow commercial, and fast commercial charging stations and their corresponding voltage profiles have been used to test the effectiveness of the proposed hybrid transfer learning framework. The results confirm the improved accuracy of the proposed hybrid CNN-BiLSTM model compared to the conventional CNN model, newly created models in predicting the EV charging demand and voltage profiles.

Facial expression recognition based on ResNet and transfer learning

With its potential to revolutionize a wide range of applications, including lie detection, social robotics, and driver fatigue detection, facial expression recognition is a field that is rapidly expanding. However, traditional machine learning methods have struggled with facial expression recognition due to limitations such as manual feature selection and limited representation capabilities. Additionally, these methods require large amounts of annotated data, which can be time-consuming and expensive to obtain. In order to overcome these difficulties, this paper suggests a novel method that builds recognition models using a multi-layer perceptron (MLP) and ResNet. This hybrid model offers improved performance over conventional CNN models, achieving an impressive accuracy rate of 85.71% on the FER_2013 dataset. Additionally, migration learning is used to increase the model's precision and avoid over-fitting. The FER_2013 dataset is used to train and test the model. The results of the trials show that the suggested model can recognize facial expressions while minimizing the overfitting problem typically associated with deep learning. The model will eventually include a self-attentive mechanism in the study in an effort to improve model resolution. By using it with color images, the team also hopes to increase the model's capacity for generalization.

Voice pathology detection using optimized convolutional neural networks and explainable artificial intelligence-based analysis

This article proposes a noninvasive computer-aided assessment approach based on optimized convolutional neural network for healthy and pathological voice detection. Firstly, the input voice samples are first transformed into mel-spectrogram time-frequency visual representations and fed for training the CNN model. The time-frequency image captures inherent speech variations beneficial for healthy and pathological voice sample detection. The weights and biases of trained CNN network are further optimized using artificial bee colony (ABC) optimization algorithm resulting in optimum CNN network employed for testing unseen data. The proposed approach is evaluated using three popular and publicly available datasets: SVD, AVPD and VOICED. Experimental results emphasize that proposed ABC optimized CNN model shows improved accuracy performance by 1.02% compared to conventional CNN network illustrating data-independent discriminative representation ability. Finally, gradient-weighted class activation mapping (Grad-CAM) explainable artificial intelligence (XAI) is utilized to make the decision understandable.