Simple Convolutional Neural Network Research Articles

A good result of brain tumor classification based on simple convolutional neural network architecture

A good result of brain tumor classification based on simple convolutional neural network architecture

Real Time Object Detection using TensorFlow

In recent years, deep learning has been used in image classification, object tracking, action recognition and scene labeling. Traditionally, Image Processing techniques were used to solve any Computer Vision problems occurred in an artificial intelligence system. However, in real-time identification, image processing cannot be used. This is where Deep Learning concepts are applied. We built a simple Convolutional Neural Network for object detection. The model is trained and multiple test cases are implemented in the TensorFlow environment so as to obtain accurate results

Partial Discharge Pattern Recognition Based on an Ensembled Simple Convolutional Neural Network and a Quadratic Support Vector Machine

Partial discharge (PD) is a crucial and intricate electrical occurrence observed in various types of electrical equipment. Identifying and characterizing PDs is essential for upholding the integrity and reliability of electrical assets. This paper proposes an ensemble methodology aiming to strike a balance between the model complexity and the predictive performance in PD pattern recognition. A simple convolutional neural network (SCNN) was constructed to efficiently decrease the model parameters (quantities). A quadratic support vector machine (QSVM) was established and ensembled with the SCNN model to effectively improve the PD recognition accuracy. The input for QSVM consisted of the circular local binary pattern (CLBP) extracted from the enhanced image. A testing prototype with three types of PD was constructed and 3D phase-resolved pulse sequence (PRPS) spectrograms were measured and recorded by ultra-high frequency (UHF) sensors. The proposed methodology was compared with three existing lightweight CNNs. The experiment results from the collected dataset emphasize the benefits of the proposed method, showcasing its advantages in high recognition accuracy and relatively few mode parameters, thereby rendering it more suitable for PD pattern recognition on resource-constrained devices.

Prediction of treatment response after stereotactic radiosurgery of brain metastasis using deep learning and radiomics on longitudinal MRI data

We developed artificial intelligence models to predict the brain metastasis (BM) treatment response after stereotactic radiosurgery (SRS) using longitudinal magnetic resonance imaging (MRI) data and evaluated prediction accuracy changes according to the number of sequential MRI scans. We included four sequential MRI scans for 194 patients with BM and 369 target lesions for the Developmental dataset. The data were randomly split (8:2 ratio) for training and testing. For external validation, 172 MRI scans from 43 patients with BM and 62 target lesions were additionally enrolled. The maximum axial diameter (Dmax), radiomics, and deep learning (DL) models were generated for comparison. We evaluated the simple convolutional neural network (CNN) model and a gated recurrent unit (Conv-GRU)-based CNN model in the DL arm. The Conv-GRU model performed superior to the simple CNN models. For both datasets, the area under the curve (AUC) was significantly higher for the two-dimensional (2D) Conv-GRU model than for the 3D Conv-GRU, Dmax, and radiomics models. The accuracy of the 2D Conv-GRU model increased with the number of follow-up studies. In conclusion, using longitudinal MRI data, the 2D Conv-GRU model outperformed all other models in predicting the treatment response after SRS of BM.

Simplified Convolutional Neural Network Model for Automatic Classification of Retinal Diseases from Optical Coherence Tomography Images

Optical coherence tomography (OCT) allows for direct and immediate imaging of the morphology of retinal tissue. It has become a crucial imaging modality for diagnosing eye problems in ophthalmology. One of the most significant morphological characteristics of the retina is the structure of the retinal layers, which provides important evidence for diagnostic purposes and is related to a variety of retinal diseases. In this paper, a convolutional neural network (CNN) model is proposed that can identify the difference between a normal retina and three common macular diseases: Diabetic macular edema (DME), Drusen, and Choroidal neovascularization (CNV). This proposed model was trained and tested on an open source dataset of OCT images also with professional disease classifications such as DME, CNV, Drusen, and Normal. The suggested model has achieved 98.3% overall classification accuracy, with only 7 wrong classifications out of 368 test samples. The suggested model significantly outperforms other models that made use of the identical dataset. The final results show that the suggested model is particularly adapted to the detection of retinal disorders in ophthalmology centers.

Electrical impedance tomography image reconstruction for lung monitoring based on ensemble learning algorithms1

AbstractElectrical impedance tomography (EIT) is a promising non‐invasive imaging technique that visualizes the electrical conductivity of an anatomic structure to form based on measured boundary voltages. However, the EIT inverse problem for the image reconstruction is nonlinear and highly ill‐posed. Therefore, in this work, a simulated dataset that mimics the human thorax was generated with boundary voltages based on given conductivity distributions. To overcome the challenges of image reconstruction, an ensemble learning method was proposed. The ensemble method combines several convolutional neural network models, which are the simple Convolutional Neural Network (CNN) model, AlexNet, AlexNet with residual block, and the modified AlexNet model. The ensemble models’ weights selection was based on average technique giving the best coefficient of determination (R2 score). The reconstruction quality is quantitatively evaluated by calculating the root mean square error (RMSE), the coefficient of determination (R2 score), and the image correlation coefficient (ICC). The proposed method's best performance is an RMSE of 0.09404, an R2 score of 0.926186, and an ICC of 0.95783 using an ensemble model. The proposed method is promising as it can construct valuable images for clinical EIT applications and measurements compared to previous studies.

Classification of Simple CNN Model and ResNet50

Abstract: In recent years, Convolutional Neural Networks (CNNs) have emerged as powerful tools for image classification tasks, achieving state-of-the-art performance in various domains. Among the plethora of CNN architectures, the Simple CNN model and ResNet50 stand out as widely used architectures with distinct characteristics. In this study, we present a comparative analysis of these two architectures in terms of their performance, computational efficiency, and robustness for classification tasks.The Simple CNN model represents a straightforward convolutional neural network architecture with a sequential arrangement of convolutional, pooling, and fully connected layers. On the other hand, ResNet50 introduces the concept of residual learning, leveraging skip connections to address the vanishing gradient problem and facilitate the training of deeper networks.We conduct experiments on benchmark datasets, evaluating the classification accuracy, training time, and model complexity of both architectures. Our findings reveal insights into the strengths and weaknesses of each model. While the Simple CNN model demonstrates simplicity and ease of implementation, ResNet50 exhibits superior performance, particularly in scenarios with a large amount of training data and complex feature representations.

CBin-NN: An Inference Engine for Binarized Neural Networks

Binarization is an extreme quantization technique that is attracting research in the Internet of Things (IoT) field, as it radically reduces the memory footprint of deep neural networks without a correspondingly significant accuracy drop. To support the effective deployment of Binarized Neural Networks (BNNs), we propose CBin-NN, a library of layer operators that allows the building of simple yet flexible convolutional neural networks (CNNs) with binary weights and activations. CBin-NN is platform-independent and is thus portable to virtually any software-programmable device. Experimental analysis on the CIFAR-10 dataset shows that our library, compared to a set of state-of-the-art inference engines, speeds up inference by 3.6 times and reduces the memory required to store model weights and activations by 7.5 times and 28 times, respectively, at the cost of slightly lower accuracy (2.5%). An ablation study stresses the importance of a Quantized Input Quantized Kernel Convolution layer to improve accuracy and reduce latency at the cost of a slight increase in model size.

Low complex CNN for semiconductor wafer defect detection

Fabrication of semiconductor wafers is a complex process and chances of defect wafers are high. Because of defective wafers the circuit patterns will not be created correctly and it is necessary to identify them. Manual identification of defects are time consuming and expensive. Deep learning methods are widely used for defect detection. In this paper we propose a simple Convolutional Neural Network (CNN) model for classification of nine defects in wafers. A custom CNN consisting of 9 layers is used for the classification of defects as Center, Donut, Edge-Loc, Edge-Ring, Loc, Random, Scratch, Near-full, and None. Performance of the model is evaluated using WM-811K dataset. Results shows that the model classifies the defects with high confidence score and an accuracy of 99.1% is achieved using this method. Further, the convolution operation in the CNN is realized using Coordinate Rotation Digital Computer (CORDIC) algorithm. The model is implemented in Field Programmable Gate Arrays (FPGA) and proved less complex method and consume less computational power than conventional methods.

Robust two-dimensional PCANet with F-norm distance metric

The principal component analysis network (PCANet) is a simplified convolutional neural network (CNN) implemented using a deep subspace learning model. Since its typical PCA filters utilize the squared Frobenius-norm (F-norm) as the distance metric, it exhibits extreme sensitivity to outliers hidden within the data. Furthermore, PCANet faces challenges in extracting structural information in the row and column directions of images as the filters need to be pre-vectorized. To address these issues, we propose a robust bi-directional two-dimensional PCANet (RBDPCANet), wherein the filters utilize the F-norm instead of the squared F-norm to mitigate the impact of outliers. Additionally, the relationship between reconstruction errors and projection distances is simultaneously considered in the objective function. After a concise analytical description of the PCANet concept and related work, we present the methodology for generating three convolutional kernels in the form of RBDPCANet-1, RBDPCANet-2, and RBDPCANet-3 algorithms. These convolution kernels are iteratively obtained and are suitable for numerical implementation. Each of the three algorithms uses another optimization objective function. It has been shown that in image recognition tasks, the developed methods are characterized by better resistance to noise compared to the classic PCANet approach. The rotation invariance and convergence of RBDPCANet are demonstrated according to theoretical principles. The efficacy of the proposed method is validated using images from publicly available datasets. Extensive experimental results on these datasets demonstrate that the proposed RBDPCANet not only significantly boosts the efficiency of image classification but also has strong resistance to occlusions, changes in lighting, changes in orientation, etc. Our implementation code is available at https://github.com/Wesley-li1/RBDPCANet-.

DBGAN: Dual Branch Generative Adversarial Network for Multi-modal MRI Translation

Existing Magnetic resonance imaging (MRI) translation models rely on Generative Adversarial Networks, primarily employing simple convolutional neural networks. Unfortunately, these networks struggle to capture global representations and contextual relationships within MRI images. While the advent of Transformers enables capturing long-range feature dependencies, they often compromise the preservation of local feature details. To address these limitations and enhance both local and global representations, we introduce a novel Dual-Branch Generative Adversarial Network (DBGAN). In this framework, the Transformer branch comprises sparse attention blocks and dense self-attention blocks, allowing for a wider receptive field while simultaneously capturing local and global information. The CNN branch, built with integrated residual convolutional layers, enhances local modeling capabilities. Additionally, we propose a fusion module that cleverly integrates features extracted from both branches. Extensive experimentation on two public datasets and one clinical dataset validates significant performance improvements with DBGAN. On Brats2018, it achieves a 10%improvement in MAE, 3.2% in PSNR, and 4.8% in SSIM for image generation tasks compablack to RegGAN. Notably, the generated MRIs receive positive feedback from radiologists, underscoring the potential of our proposed method as a valuable tool in clinical settings.

Garbage Classification Using Deep Learning

Garbage disposal is a critical responsibility in maintaining a healthy and green environment. The recycling business is thriving as inhabitants of India become more aware of the necessity of a clean environment in terms of reducing natural resource use and rubbish disposal. The amount of trash generated in daily life has an impact on land, water, and air, posing a major threat to aquatic animals and their habitats, as well as humans, if not properly managed. Conventional garbage disposal systems are on the rise, requiring accurate and efficient segmentation and recognition processes. This demand corresponds to an increase in the computing capacity of modern computer systems as well as more efficient picture recognition methods. To address this issue, the garbage categorization process is automated by using an image classifier with a convolutional neural network (CNN), which reduces waste segregation time and costs. The goal of automating the process is to reduce human intervention and increase productivity in the waste segregation process. In this paper, three distinct models are tested for greater accuracy: Simple CNN, ResNet50, and VGG16 are trained on various image datasets and used to extract features from images, which are then fed into a classifier for dump/trash categorization. The learned models are then loaded into the mobile application.

Convolutional neural network for estimation of harvest time of forage sorghum (sorghum bicolor) cultivar samurai-1

One of the economic alternatives to improve the quality of ruminant feed is combining grass as the main feed with high-protein forages such as sorghum. To get a quality sorghum harvest during the period, it must be right when it has good biomass content, nutrients, and digestibility. The problem is that measuring quality in the laboratory has additional costs and time, which is not short, causing delays. An approach with machine learning using a convolutional neural network can be a better solution. This research uses a convolutional neural network algorithm with the right architecture to estimate sorghum harvest time from imaging results of unmanned aerial vehicles. The stages of this research include data collection, pre-processing, modeling, and finally, the evaluation stage. This research compares the results of several convolutional neural network (CNN) algorithm architectural models: simple CNN, ResNet50 V2, visual geometry group-16 (VGG-16), MobileNet V2, and Inception V3. The result is determining the CNN algorithm architectural model that can estimate sorghum harvest time with maximum accuracy. The best result is the simple CNN architectural model with an accuracy of 0.95. This research shows that the classification model obtained from the CNN algorithm with a simple CNN architecture is the choice model for estimating sorghum harvest time.

SCOV-CNN: A Simple CNN Architecture for COVID-19 Identification Based on the CT Images

Since the coronavirus was first discovered in Wuhan, it has widely spread and was finally declared a global pandemic by the WHO. Image processing plays an essential role in examining the lungs of affected patients. Computed Tomography (CT) and X-ray images have been popularly used to examine the lungs of COVID-19 patients. This research aims to design a simple Convolution Neural Network (CNN) architecture called SCOV-CNN for the classification of the virus based on CT images and implementation on the web-based application. The data used in this work were CT images of 120 patients from hospitals in Brazil. SCOV-CNN was inspired by the LeNet architecture, but it has a deeper convolution and pooling layer structure. Combining seven and five kernel sizes for convolution and padding schemes can preserve the feature information from the images. Furthermore, it has three fully connected (FC) layers with a dropout of 0.3 on each. In addition, the model was evaluated using the sensitivity, specificity, precision, F1 score, and ROC curve values. The results showed that the architecture we proposed was comparable to some prominent deep learning techniques in terms of accuracy (0.96), precision (0.98), and F1 score (0.95). The best model was integrated into a website-based system to help and facilitate the users' activities. We use Python Flask Pam tools as a web server on the server side and JavaScript for the User Interface (UI) Design

ResDiff: Combining CNN and Diffusion Model for Image Super-resolution

Adapting the Diffusion Probabilistic Model (DPM) for direct image super-resolution is wasteful, given that a simple Convolutional Neural Network (CNN) can recover the main low-frequency content. Therefore, we present ResDiff, a novel Diffusion Probabilistic Model based on Residual structure for Single Image Super-Resolution (SISR). ResDiff utilizes a combination of a CNN, which restores primary low-frequency components, and a DPM, which predicts the residual between the ground-truth image and the CNN predicted image. In contrast to the common diffusion-based methods that directly use LR space to guide the noise towards HR space, ResDiff utilizes the CNN’s initial prediction to direct the noise towards the residual space between HR space and CNN-predicted space, which not only accelerates the generation process but also acquires superior sample quality. Additionally, a frequency-domain-based loss function for CNN is introduced to facilitate its restoration, and a frequency-domain guided diffusion is designed for DPM on behalf of predicting high-frequency details. The extensive experiments on multiple benchmark datasets demonstrate that ResDiff outperforms previous diffusion based methods in terms of shorter model convergence time, superior generation quality, and more diverse samples.

Best performance with fewest resources: Unveiling the most resource-efficient Convolutional Neural Network for P300 detection with the aid of Explainable AI

Convolutional Neural Networks (CNNs) have shown remarkable prowess in detecting P300, an Event-Related Potential (ERP) crucial in Brain–Computer Interfaces (BCIs). Researchers persistently seek simple and efficient CNNs for P300 detection, exemplified by models like DeepConvNet, EEGNet, and SepConv1D. Noteworthy progress has been made, manifesting in reducing parameters from millions to hundreds while sustaining state-of-the-art performance. However, achieving further simplification or performance improvement beyond SepConv1D appears challenging due to inherent oversimplification. This study explores landmark CNNs and P300 data with the aid of Explainable AI, proposing a simpler yet superior-performing CNN architecture which incorporates (1) precise separable convolution for feature extraction of P300 data, (2) adaptive activation function tailored for P300 data, and (3) customized large learning rate schedules for training P300 data. Termed the Minimalist CNN for P300 detection (P300MCNN), this novel model is characterized by its requirement of the fewest filters and epochs to date, concurrently achieving best performance in cross-subject P300 detection. P300MCNN not only introduces groundbreaking concepts for CNN architectures in P300 detection but also showcases the importance of Explainable AI in demystifying the “black box” design of CNNs.

On the pitfalls of Batch Normalization for end-to-end video learning: A study on surgical workflow analysis

Batch Normalization’s (BN) unique property of depending on other samples in a batch is known to cause problems in several tasks, including sequence modeling. Yet, BN-related issues are hardly studied for long video understanding, despite the ubiquitous use of BN in CNNs (Convolutional Neural Networks) for feature extraction. Especially in surgical workflow analysis, where the lack of pretrained feature extractors has led to complex, multi-stage training pipelines, limited awareness of BN issues may have hidden the benefits of training CNNs and temporal models end to end. In this paper, we analyze pitfalls of BN in video learning, including issues specific to online tasks such as a ’cheating’ effect in anticipation. We observe that BN’s properties create major obstacles for end-to-end learning. However, using BN-free backbones, even simple CNN–LSTMs beat the state of the art on three surgical workflow benchmarks by utilizing adequate end-to-end training strategies which maximize temporal context. We conclude that awareness of BN’s pitfalls is crucial for effective end-to-end learning in surgical tasks. By reproducing results on natural-video datasets, we hope our insights will benefit other areas of video learning as well. Code is available at: https://gitlab.com/nct_tso_public/pitfalls_bn.

Pattern Augmented Lightweight Convolutional Neural Network for Intrusion Detection System

As the world increasingly becomes more interconnected, the demand for safety and security is ever-increasing, particularly for industrial networks. This has prompted numerous researchers to investigate different methodologies and techniques suitable for intrusion detection systems (IDS) requirements. Over the years, many studies have proposed various solutions in this regard, including signature-based and machine learning (ML)-based systems. More recently, researchers are considering deep learning (DL)-based anomaly detection approaches. Most proposed works in this research field aim to achieve either one or a combination of high accuracy, considerably low false alarm rates (FARs), high classification specificity and detection sensitivity, lightweight DL models, or other ML and DL-related performance measurement metrics. In this study, we propose a novel method to convert a raw dataset to an image dataset to magnify patterns by utilizing the Short-Term Fourier transform (STFT). The resulting high-quality image dataset allowed us to devise an anomaly detection system for IDS using a simple lightweight convolutional neural network (CNN) that classifies denial of service and distributed denial of service. The proposed methods were evaluated using a modern dataset, CSE-CIC-IDS2018, and a legacy dataset, NSLKDD. We have also applied a combined dataset to assess the generalization of the proposed model across various datasets. Our experimental results have demonstrated that the proposed methods achieved high accuracy and considerably low FARs with high specificity and sensitivity. The resulting loss and accuracy curves have demonstrated the efficacy of our raw dataset to image dataset conversion methodology, which is evident as an excellent generalization of the proposed lightweight CNN model was observed, effectively avoiding overfitting. This holds for both the modern and legacy datasets, including their mixed versions.

Effective time-series Data Augmentation with Analytic Wavelets for bearing fault diagnosis

In the realm of rotary machine maintenance, rolling bearings emerge as crucial yet frequently vulnerable components. Ensuring their operational integrity is pivotal for the overall safety and performance of the machinery. Traditional diagnostic methods have utilized a variety of signal processing techniques, ranging from blind source separation to wavelet analysis, and more recently, deep neural networks (DNNs). However, a significant challenge in this domain is the scarcity of labeled failure data in real-world applications, leading to a reliance on data augmentation (DA) techniques to enhance the quality and quantity of the training data. Compounding this problem is the use of DA techniques tailored for images, which are not suitable for the time–frequency representations. This study proposes an innovative Data Augmentation with Analytic Wavelets (DAAW) approach, tailored specifically for time-series (TS) data classification in bearing fault diagnosis. The essence of DAAW lies in its ability to generate synthetic scalogram samples that closely mirror the properties of original samples. The proposed DAAW is directly applied at the input stage of a learning model by successively adjusting predefined parameters to generate scalograms, without the need for auxiliary DA algorithms. This technique is versatile in generating synthetic time–frequency data samples, and the parameterization enables it to be tailored to different time series datasets. Through experimentation with publicly available datasets, we show that combining a simple Convolutional Neural Network (CNN) with the proposed approach results in improved performance. It outperforms existing methods, achieving a mean classification accuracy of 99.49%. The critical distance for average accuracy, as determined by the Nemenyi post-hoc test, is 0.4619. Our results also show enhanced performance with accuracy rates of 90.14%, 78.94%, and 62.86% even under low signal-to-noise ratio (SNR) conditions, using 80%, 60%, and 50% of the dataset, respectively, for model training. Notably, its efficacy remains robust even under challenging low SNR conditions. The proposed DAAW significantly improves classification accuracy in bearing fault diagnosis, simultaneously obviating the need for proxy DA tasks.

Deep learning cosmic ray transport from density maps of simulated, turbulent gas

The coarse-grained propagation of galactic cosmic rays (CRs) is traditionally constrained by phenomenological models of Milky Way CR propagation fit to a variety of direct and indirect observables; however, constraining the fine-grained transport of CRs along individual magnetic field lines—for instance, diffusive vs streaming transport models—is an unsolved challenge. Leveraging a recent training set of magnetohydrodynamic turbulent box simulations, with CRs spanning a range of transport parameters, we use convolutional neural networks (CNNs) trained solely on gas density maps to classify CR transport regimes. We find that even relatively simple CNNs can quite effectively classify density slices to corresponding CR transport parameters, distinguishing between streaming and diffusive transport, as well as magnitude of diffusivity, with class accuracies between 92% and 99%. As we show, the transport-dependent imprints that CRs leave on the gas are not all tied to the resulting density power spectra: classification accuracies are still high even when image spectra are flattened (85%–98% accuracy), highlighting CR transport-dependent changes to turbulent phase information. We interpret our results with saliency maps and image modifications, and we discuss physical insights and future applications.