Shallow Convolutional Neural Network Research Articles

On the effects of recursive convolutional layers in convolutional neural networks

The Recursive Convolutional Layer (RCL) is a module that wraps a recursive feedback loop around a convolutional layer (CL). The RCL has been proposed to address some of the shortcomings of Convolutional Neural Networks (CNNs), as its unfolding increases the depth of a network without increasing the number of weights. We investigated the “naïve” substitution of CL with RCL on three base models: a 4-CL model, ResNet, DenseNet and their RCL-ized versions: C-FRPN, R-ResNet, and R-DenseNet using five image classification datasets. We find that this one-to-one replacement significantly improves the performances of the 4-CL model, but not those of ResNet or DenseNet. This led us to investigate the implication of the RCL substitution on the 4-CL model which reveals, among a number of properties, that RCLs are particularly efficient in shallow CNNs. We proceeded to re-visit the first set of experiments by gradually transforming the 4-CL model and the C-FRPN into respectively ResNet and R-ResNet, and find that the performance improvement is largely driven by the training regime whereas any depth increase negatively impacts the RCL-ized version. We conclude that the replacement of CLs by RCLs shows great potential in designing high-performance shallow CNNs.

Deep learning-assisted concentration gradient generation for the study of 3D cell cultures in hydrogel beads of varying stiffness.

The study of dose-response relationships underpins analytical biosciences. Droplet microfluidics platforms can automate the generation of microreactors encapsulating varying concentrations of an assay component, providing datasets across a large chemical space in a single experiment. A classical method consists in varying the flow rate of multiple solutions co-flowing into a single microchannel (producing different volume fractions) before encapsulating the contents into water-in-oil droplets. This process can be automated through controlling the pumping elements but lacks the ability to adapt to unpredictable experimental scenarios, often requiring constant human supervision. In this paper, we introduce an image-based, closed-loop control system for assessing and adjusting volume fractions, thereby generating unsupervised, uniform concentration gradients. We trained a shallow convolutional neural network to assess the position of the laminar flow interface between two co-flowing fluids and used this model to adjust flow rates in real-time. We apply the method to generate alginate microbeads in which HEK293FT cells could grow in three dimensions. The stiffnesses ranged from 50Pa to close to 1kPa in Young modulus and were encoded with a fluorescent marker. We trained deep learning models based on the YOLOv4 object detector to efficiently detect both microbeads and multicellular spheroids from high-content screening images. This allowed us to map relationships between hydrogel stiffness and multicellular spheroid growth.

Evaluation of Shallow Convolutional Neural Network in Open-World Chart Image Classification

Evaluation of Shallow Convolutional Neural Network in Open-World Chart Image Classification

Turbulence Closure With Small, Local Neural Networks: Forced Two‐Dimensional and β‐Plane Flows

AbstractWe parameterize sub‐grid scale (SGS) fluxes in sinusoidally forced two‐dimensional turbulence on the β‐plane at high Reynolds numbers (Re ∼25,000) using simple 2‐layer convolutional neural networks (CNN) having only O(1000) parameters, two orders of magnitude smaller than recent studies employing deeper CNNs with 8–10 layers; we obtain stable, accurate, and long‐term online or a posteriori solutions at 16× downscaling factors. Our methodology significantly improves training efficiency and speed of online large eddy simulations runs, while offering insights into the physics of closure in such turbulent flows. Our approach benefits from extensive hyperparameter searching in learning rate and weight decay coefficient space, as well as the use of cyclical learning rate annealing, which leads to more robust and accurate online solutions compared to fixed learning rates. Our CNNs use either the coarse velocity or the vorticity and strain fields as inputs, and output the two components of the deviatoric stress tensor, Sd. We minimize a loss between the SGS vorticity flux divergence (computed from the high‐resolution solver) and that obtained from the CNN‐modeled Sd, without requiring energy or enstrophy preserving constraints. The success of shallow CNNs in accurately parameterizing this class of turbulent flows implies that the SGS stresses have a weak non‐local dependence on coarse fields; it also aligns with our physical conception that small‐scales are locally controlled by larger scales such as vortices and their strained filaments. Furthermore, 2‐layer CNN‐parameterizations are more likely to be interpretable.

Swift Diagnose: A High-Performance Shallow Convolutional Neural Network for Rapid and Reliable SARS-COV-2 Induced Pneumonia Detection

INTRODUCTION: The SARS-COV-2 pandemic has led to a significant increase in the number of infected individuals and a considerable loss of lives. Identifying SARS-COV-2-induced pneumonia cases promptly is crucial for controlling the virus's spread and improving patient care. In this context, chest X-ray imaging has become an essential tool for detecting pneumonia caused by the novel coronavirus.
 OBJECTIVES: The primary goal of this research is to differentiate between pneumonia cases induced specifically by the SARS-COV-2 virus and other types of pneumonia or healthy cases. This distinction is vital for the effective treatment and isolation of affected patients.
 METHODS: A streamlined stacked Convolutional Neural Network (CNN) architecture was employed for this study. The dataset, meticulously curated from Johns Hopkins University's medical database, comprised 2292 chest X-ray images. This included 542 images of COVID-19-infected cases and 1266 non-COVID cases for the training phase, and 167 COVID-infected images plus 317 non-COVID images for the testing phase. The CNN's performance was assessed against a well-established CNN model to ensure the reliability of the findings.
 RESULTS: The proposed CNN model demonstrated exceptional accuracy, with an overall accuracy rate of 98.96%. In particular, the model achieved a per-class accuracy of 99.405% for detecting SARS-COV-2-infected cases and 98.73% for identifying non-COVID cases. These results indicate the model's significant potential in distinguishing between COVID-19-related pneumonia and other conditions.
 CONCLUSION: The research validates the efficacy of using a specialized CNN architecture for the rapid and precise identification of SARS-COV-2-induced pneumonia from chest X-ray images. The high accuracy rates suggest that this method could be a valuable tool in the ongoing fight against the COVID-19 pandemic, aiding in the swift diagnosis and effective treatment of patients.

Siamese Networks for Clinically Relevant Bacteria Classification Based on Raman Spectroscopy.

Identifying bacterial strains is essential in microbiology for various practical applications, such as disease diagnosis and quality monitoring of food and water. Classical machine learning algorithms have been utilized to identify bacteria based on their Raman spectra. However, convolutional neural networks (CNNs) offer higher classification accuracy, but they require extensive training sets and retraining of previous untrained class targets can be costly and time-consuming. Siamese networks have emerged as a promising solution. They are composed of two CNNs with the same structure and a final network that acts as a distance metric, converting the classification problem into a similarity problem. Classical machine learning approaches, shallow and deep CNNs, and two Siamese network variants were tailored and tested on Raman spectral datasets of bacteria. The methods were evaluated based on mean sensitivity, training time, prediction time, and the number of parameters. In this comparison, Siamese-model2 achieved the highest mean sensitivity of 83.61 ± 4.73 and demonstrated remarkable performance in handling unbalanced and limited data scenarios, achieving a prediction accuracy of 73%. Therefore, the choice of model depends on the specific trade-off between accuracy, (prediction/training) time, and resources for the particular application. Classical machine learning models and shallow CNN models may be more suitable if time and computational resources are a concern. Siamese networks are a good choice for small datasets and CNN for extensive data.

CN-BSRIQA: Cascaded network - blind super-resolution image quality assessment

High resolution (HR) images consist of higher quality and more detail information in comparison to low-resolution images. But obtaining HR images entails higher costs and requires a larger workforce. The solution is using super-resolution (SR) images. Single image SR is the process of creating HR images from a single low-resolution image. Obtaining SR images from low-resolution images is a well-known problem in the domains of computer vision and image processing. Advancement in technology has given rise to many SR algorithms. The perceptual quality assessment of super-resolved images can be used to benchmark the techniques employed for image SR. In this work, a cascaded network-blind super-resolution image quality assessment (CN-BSRIQA) methodology is proposed. The proposed approach works under the cascaded architecture where a convolutional neural network (CNN) is cascaded with a deep belief network (DBN). CNN is employed as a shallow network in the proposed methodology to extract low-level information after partitioning the input image into patches. To assess the visual quality of the SR images, the features retrieved from CNN are incorporated into a DBN. Three databases are used to evaluate the performance of proposed CN-BSRIQA i.e., SR Quality Database (SRQD), SR Image Quality Database (SRID), and visual quality evaluation for super-resolved images (QADS). When compared to other state-of-the-art methodologies for assessing the visual quality of SR images, CN-BSRIQA outperforms them. For the perceptual quality assessment of SR images, the experimental results reveal that CNN-based techniques outperform techniques based on hand-crafted features. Furthermore, the shallow CNN proposed in the CN-BSRIQA can extract features that are content-independent i.e., they show better performance over cross-database evaluation in comparison to existing state-of-the-art techniques.

SVG-CNN: A shallow CNN based on VGGNet applied to intra prediction partition block in HEVC

SVG-CNN: A shallow CNN based on VGGNet applied to intra prediction partition block in HEVC

CoSev: Data-Driven Optimizations for COVID-19 Severity Assessment in Low-Sample Regimes

Given the pronounced impact COVID-19 continues to have on society—infecting 700 million reported individuals and causing 6.96 million deaths—many deep learning works have recently focused on the virus’s diagnosis. However, assessing severity has remained an open and challenging problem due to a lack of large datasets, the large dimensionality of images for which to find weights, and the compute limitations of modern graphics processing units (GPUs). In this paper, a new, iterative application of transfer learning is demonstrated on the understudied field of 3D CT scans for COVID-19 severity analysis. This methodology allows for enhanced performance on the MosMed Dataset, which is a small and challenging dataset containing 1130 images of patients for five levels of COVID-19 severity (Zero, Mild, Moderate, Severe, and Critical). Specifically, given the large dimensionality of the input images, we create several custom shallow convolutional neural network (CNN) architectures and iteratively refine and optimize them, paying attention to learning rates, layer types, normalization types, filter sizes, dropout values, and more. After a preliminary architecture design, the models are systematically trained on a simplified version of the dataset-building models for two-class, then three-class, then four-class, and finally five-class classification. The simplified problem structure allows the model to start learning preliminary features, which can then be further modified for more difficult classification tasks. Our final model CoSev boosts classification accuracies from below 60% at first to 81.57% with the optimizations, reaching similar performance to the state-of-the-art on the dataset, with much simpler setup procedures. In addition to COVID-19 severity diagnosis, the explored methodology can be applied to general image-based disease detection. Overall, this work highlights innovative methodologies that advance current computer vision practices for high-dimension, low-sample data as well as the practicality of data-driven machine learning and the importance of feature design for training, which can then be implemented for improvements in clinical practices.

Heterogeneous fusion of biometric and deep physiological features for accurate porcine cough recognition.

Accurate identification of porcine cough plays a vital role in comprehensive respiratory health monitoring and diagnosis of pigs. It serves as a fundamental prerequisite for stress-free animal health management, reducing pig mortality rates, and improving the economic efficiency of the farming industry. Creating a representative multi-source signal signature for porcine cough is a crucial step toward automating its identification. To this end, a feature fusion method that combines the biological features extracted from the acoustic source segment with the deep physiological features derived from thermal source images is proposed in the paper. First, acoustic features from various domains are extracted from the sound source signals. To determine the most effective combination of sound source features, an SVM-based recursive feature elimination cross-validation algorithm (SVM-RFECV) is employed. Second, a shallow convolutional neural network (named ThermographicNet) is constructed to extract deep physiological features from the thermal source images. Finally, the two heterogeneous features are integrated at an early stage and input into a support vector machine (SVM) for porcine cough recognition. Through rigorous experimentation, the performance of the proposed fusion approach is evaluated, achieving an impressive accuracy of 98.79% in recognizing porcine cough. These results further underscore the effectiveness of combining acoustic source features with heterogeneous deep thermal source features, thereby establishing a robust feature representation for porcine cough recognition.

Comparison of convolutional neural network models for user’s facial recognition

This paper compares well-known convolutional neural networks (CNN) models for facial recognition. For this, it uses its database created from two registered users and an additional category of unknown persons. Eight different base models of convolutional architectures were compared by transfer of learning, and two additional proposed models called shallow CNN and shallow directed acyclic graph with CNN (DAG-CNN), which are architectures with little depth (six convolution layers). Within the tests with the database, the best results were obtained by the GoogLeNet and ResNet-101 models, managing to classify 100% of the images, even without confusing people outside the two users. However, in an additional real-time test, in which one of the users had his style changed, the models that showed the greatest robustness in this situation were the Inception and the ResNet-101, being able to maintain constant recognition. This demonstrated that the networks of greater depth manage to learn more detailed features of the users' faces, unlike those of shallower ones; their learning of features is more generalized. Declare the full term of an abbreviation/acronym when it is mentioned for the first time.

Impact of PSF misestimation and galaxy population bias on precision shear measurement using a CNN

ABSTRACT Weak gravitational lensing of distant galaxies provides a powerful probe of dark energy. The aim of this study is to investigate the application of convolutional neural networks (CNNs) to precision shear estimation. In particular, using a shallow CNN, we explore the impact of point spread function (PSF) misestimation and ‘galaxy population bias’ (including ‘distribution bias’ and ‘morphology bias’), focusing on the accuracy requirements of next generation surveys. We simulate a population of noisy disc and elliptical galaxies and adopt a PSF that is representative of a Euclid-like survey. We quantify the accuracy achieved by the CNN, assuming a linear relationship between the estimated and true shears and measure the multiplicative (m) and additive (c) biases. We make use of an unconventional loss function to mitigate the effects of noise bias and measure m and c when we use either: (i) an incorrect galaxy ellipticity distribution or size–magnitude relation, or the wrong ratio of morphological types, to describe the population of galaxies (distribution bias); (ii) an incorrect galaxy light profile (morphology bias); or (iii) a PSF with size or ellipticity offset from its true value (PSF misestimation). We compare our results to the Euclid requirements on the knowledge of the PSF model shape and size. Finally, we outline further work to build on the promising potential of CNNs in precision shear estimation.

Optimized Convolutional Forest by Particle Swarm Optimizer for Pothole Detection

Poor road maintenance leads to potholes on the road. Potholes are responsible for road accidents and even deaths in developed and developing countries. Detecting and filling road potholes is an essential part of road maintenance. Sustaining a reliable and safe road for communication depends on pothole detection. This study presents a novel combination of a convolutional neural network and an optimized machine-learning model by a heuristic algorithm for pothole detection. The proposed method comprises a shallow convolutional neural network for feature extraction and an optimized random forest model for pothole detection. The proposed model initially uses the shallow convolutional layer to extract feature sets from input pictures. Then, the particle swarm optimizer is used to eliminate irrelevant features. Finally, a combination of random forest and a particle swarm optimizer is used for pothole detection. Particle swarm optimization indicates the best subset of the extracted feature set for final pothole detection. We added 171 pictures to the already available 665 pothole pictures to evaluate the proposed method. The test set was isolated from the training set, and we trained the model on k-fold cross-validation. The experimental result indicates 99.37% accuracy, 99.37% precision, 99.38% sensitivity, and 99.38% F1-score for discriminating potholes from roads without potholes by proposed methods. The response time of the proposed method for pothole detection is 0.02 s. The proposed method can be utilized for real-time pothole detection.

FPGA based Flexible Implementation of Light Weight Inference on Deep Convolutional Neural Networks

Standard Convolution (StdConv) is the main technique used in the state of the art Deep Convolutional Neural Networks (DCNNs). Fewer computations are achieved if Depthwise Separable Convolution technique (SepConv) is used as an alternative. A crucial issue in many applications like smart cameras and autonomous vehicles where low latency is essential stems from deploying a lightweight and low cost inference models. An acceptable accuracy should be kept with tolerable computations and memory access load. A flexible architecture for different DCNN convolution types and models is proposed. The flexibility comes from the sharing of one memory access unit with different types of layers regardless of the selected kernel size, by multiplying each weight vector by local operators with variant aperture. Moreover, one depthwise computation unit can be used for both standard and pointwise layers. The learnable parameters are quantized to 8-bits fixed point representation and that gives very limited reduction of accuracy and a considerable reduction of the Field-Programmable Gate Array (FPGA) resources. To reduce processing time, inter layer parallel computations are performed. The experiment is conducted by using grey scale ORL database with shallow Convolutional Neural Network (CNN) and the colored Canadian Institute for Advanced Research 10 classes (CIFAR-10) database with DCNN, and a comparable accuracies of 93% and 85.7% are achieved respectively using very low cost of Spartan 3E and moderate cost of zynq FPGA platforms

Dual‐branch feature extraction network combined with Transformer and CNN for polyp segmentation

AbstractTo overcome the difficulty of accurate polyp segmentation, a novel encoder–decoder network DFETC‐Net is proposed, in which two encoders based on Swin Transformer and CNN are utilized to extract the global and local features respectively. Further, a new self‐attention and convolution feature fusion module is designed to fuse the two branch features to enhance the feature representative capability and alleviate the influence of the semantic gap. In the bottleneck, a new multi‐feature pyramid pooling module fuses all deep features from two branches to obtain multi‐scale information and promote segmentation accuracy. The coordinate attention is used in the skip connections between two shallow CNN blocks and corresponding decoder blocks to pay more attention to doubtful and complicated regions. Extensive experiments demonstrate the proposed network outperforms several state‐of‐the‐art methods in terms of both qualitative effects and quantitative measurements. All codes are available on https://github.com/LYJieH/DFETC-NET.

An AI approach to operationalise global daily PlanetScope satellite imagery for river water masking

Monitoring rivers is vital to manage the invaluable ecosystem services they provide, and also to mitigate the risks they pose to property and life through flooding and drought. Due to the vast extent and dynamic nature of river systems, Earth Observation (EO) is one of the best ways to measure river characteristics. As a first step, EO-based river monitoring often requires extraction of accurate pixel-level water masks, but satellite images traditionally used for this purpose suffer from limited spatial and/or temporal resolution. We address this problem by applying a novel Convolutional Neural Network (CNN)-based model to automate water mask extraction from daily 3 m resolution PlanetScope satellite imagery. Notably, this approach overcomes radiometric issues that frequently present limitations when working with CubeSat data. We test our classification model on 36 rivers across 12 global terrestrial biomes (as proxies for the environmental and physical characteristics that lead to the variability in catchments around the globe). Using a relatively shallow CNN classification model, our approach produced a median F1 accuracy score of 0.93, suggesting that a compact and efficient CNN-based model can work as well as, if not better than, the very deep neural networks conventionally used in similar studies, whilst requiring less training data and computational power. We further show that our model, specialised to the task at hand, performs better than a state-of-the-art Fully Convolutional Neural Network (FCN) that struggles with the highly variable image quality from PlanetScope. Although classifying rivers that were narrower than 60 m, anastomosed or highly urbanised was slightly less successful than our other test images, we showed that fine tuning could circumvent these limitations to some degree. Indeed, fine tuning carried out on the Ottawa River, Canada, by including just 5 additional site-specific training images significantly improved classification accuracy (F1 increased from 0.81 to 0.90, p < 0.01). Overall, our results show that CNN-based classification applied to PlanetScope imagery is a viable tool for producing accurate, temporally dynamic river water masks, opening up possibilities for river monitoring investigations where high temporal variability data is essential.

Category-aware Recycle Classification using Convolutional Neural Networks

Generally, the filtration of non-recyclable garbage is conducted in order to make a material recycling process efficient. However, this filtration task is very time-consuming and costly, since it is generally done manually. Thus, it is significantly considered that an automation technique is necessary in solving this problem. There have been a number of attempts to resolve this problem. The previous methods used the shallow convolutional neural network to compose their network. However, these methods tend to display poor accuracy, which makes it impossible to be applied in real-life situations. To solve this problem, I propose a novel category-aware recycle classification system. The proposed system is composed of three models, which are garbage feature extractor, garbage classifier, and recycle classifier. The input image is fed to the garbage feature extractor and then converted into feature maps. The feature maps are then fed to each the garbage classifier and the recycle classifier. The garbage classifier predicts the category of the input garbage image while the recycle classifier determines if the input is recyclable or not. Through experiments, the proposed method outperforms the other state-of-the-art methods by a maximum 20.2% difference in accuracy on Garbage Classification and Industrial and Residential Waste datasets.

Selecting hyperspectral bands and extracting features with a custom shallow convolutional neural network to classify citrus peel defects

Citrus Black Spot (CBS) causes considerable damage to the Florida citrus industry. Early detection of CBS, especially in the presence of other peel blemishes, would enable better mapping and control of CBS spread, reduce wasted fruit, and permit early removal of culls from the packing stream. Oranges whose peels bore the symptoms of four defects/disease (CBS, greasy spot, melanose, and wind scar), as well as a normal control group, were imaged with a hyperspectral imaging system. Principal Component Analysis- (PCA) and Linear Discriminant Analysis (LDA) -based methods were employed to select bands from these images, and a custom convolutional neural network (CNN) for feature extraction was trained with these bands. The extracted features permitted classification of the peel conditions with four classifiers: SoftMax, Support Vector Machines (SVM), Random Forest Classifier (RFC), and K-Nearest Neighbors (KNN). A mean overall accuracy of 94.9 % was achieved using an SVM classifier on five bands selected with PCA, and 90.2 % with LDA-selected bands. These results show the potential of CNNs to extract features for automated postharvest citrus inspection.

A CNN-SVM Study Based on the Fusion of Spectrogram and Thermal Imaging Heterogeneous Features for Pig Cough Recognition in Field Situation

Accurate identification of pig cough is essential for comprehensive monitoring and diagnosis of the respiratory health status of pigs. It contributes to stress-free animal health management, reduces pig mortality and improves the economics of farming. Creating a representative multisource signal signature of pig cough is a critical step in achieving automatic recognition of pig cough. For this reason, in this paper, we propose a feature fusion classification method that combines the spectrogram deep features and thermal image deep features to be fed into a support vector machine (SVM) classifier to accomplish cough classification. First, we use a time–frequency transformation algorithm to convert a one-dimensional cough sound signal into a two-dimensional acoustic spectrogram. Then, the corresponding heterogeneous deep features are extracted from the cough spectrogram and thermal image by fine-tuning Lenet-5 and a customized CoughRNet shallow convolutional neural network. Finally, we employ an early fusion technique to align and splice the extracted heterogeneous deep features and feed them into an SVM for the automatic classification task of pig cough. Our study evaluates the classification performance, recognition speed and model size of the proposed deep feature fusion classification network with satisfactory results. Experimental results show that the method achieves 99.77% accuracy in pig cough recognition. This further demonstrates the effectiveness of combining abstract heterogeneous sound and thermal image deep features as a method for automated detection of pig respiratory health.

Automated breast tumor ultrasound image segmentation with hybrid UNet and classification using fine-tuned CNN model

IntroductionBreast cancer stands as the second most deadly form of cancer among women worldwide. Early diagnosis and treatment can significantly mitigate mortality rates. PurposeThe study aims to classify breast ultrasound images into benign and malignant tumors. This approach involves segmenting the breast's region of interest (ROI) employing an optimized UNet architecture and classifying the ROIs through an optimized shallow CNN model utilizing an ablation study. MethodSeveral image processing techniques are utilized to improve image quality by removing text, artifacts, and speckle noise, and statistical analysis is done to check the enhanced image quality is satisfactory. With the processed dataset, the segmentation of breast tumor ROI is carried out, optimizing the UNet model through an ablation study where the architectural configuration and hyperparameters are altered. After obtaining the tumor ROIs from the fine-tuned UNet model (RKO-UNet), an optimized CNN model is employed to classify the tumor into benign and malignant classes. To enhance the CNN model's performance, an ablation study is conducted, coupled with the integration of an attention unit. The model's performance is further assessed by classifying breast cancer with mammogram images. ResultThe proposed classification model (RKONet-13) results in an accuracy of 98.41 %. The performance of the proposed model is further compared with five transfer learning models for both pre-segmented and post-segmented datasets. K-fold cross-validation is done to assess the proposed RKONet-13 model's performance stability. Furthermore, the performance of the proposed model is compared with previous literature, where the proposed model outperforms existing methods, demonstrating its effectiveness in breast cancer diagnosis. Lastly, the model demonstrates its robustness for breast cancer classification, delivering an exceptional performance of 96.21 % on a mammogram dataset. ConclusionThe efficacy of this study relies on image pre-processing, segmentation with hybrid attention UNet, and classification with fine-tuned robust CNN model. This comprehensive approach aims to determine an effective technique for detecting breast cancer within ultrasound images.