Shallow Convolutional Neural Network Research Articles

CoSev: Data-Driven Optimizations for COVID-19 Severity Assessment in Low-Sample Regimes

Given the pronounced impact COVID-19 continues to have on society—infecting 700 million reported individuals and causing 6.96 million deaths—many deep learning works have recently focused on the virus’s diagnosis. However, assessing severity has remained an open and challenging problem due to a lack of large datasets, the large dimensionality of images for which to find weights, and the compute limitations of modern graphics processing units (GPUs). In this paper, a new, iterative application of transfer learning is demonstrated on the understudied field of 3D CT scans for COVID-19 severity analysis. This methodology allows for enhanced performance on the MosMed Dataset, which is a small and challenging dataset containing 1130 images of patients for five levels of COVID-19 severity (Zero, Mild, Moderate, Severe, and Critical). Specifically, given the large dimensionality of the input images, we create several custom shallow convolutional neural network (CNN) architectures and iteratively refine and optimize them, paying attention to learning rates, layer types, normalization types, filter sizes, dropout values, and more. After a preliminary architecture design, the models are systematically trained on a simplified version of the dataset-building models for two-class, then three-class, then four-class, and finally five-class classification. The simplified problem structure allows the model to start learning preliminary features, which can then be further modified for more difficult classification tasks. Our final model CoSev boosts classification accuracies from below 60% at first to 81.57% with the optimizations, reaching similar performance to the state-of-the-art on the dataset, with much simpler setup procedures. In addition to COVID-19 severity diagnosis, the explored methodology can be applied to general image-based disease detection. Overall, this work highlights innovative methodologies that advance current computer vision practices for high-dimension, low-sample data as well as the practicality of data-driven machine learning and the importance of feature design for training, which can then be implemented for improvements in clinical practices.

Heterogeneous fusion of biometric and deep physiological features for accurate porcine cough recognition.

Accurate identification of porcine cough plays a vital role in comprehensive respiratory health monitoring and diagnosis of pigs. It serves as a fundamental prerequisite for stress-free animal health management, reducing pig mortality rates, and improving the economic efficiency of the farming industry. Creating a representative multi-source signal signature for porcine cough is a crucial step toward automating its identification. To this end, a feature fusion method that combines the biological features extracted from the acoustic source segment with the deep physiological features derived from thermal source images is proposed in the paper. First, acoustic features from various domains are extracted from the sound source signals. To determine the most effective combination of sound source features, an SVM-based recursive feature elimination cross-validation algorithm (SVM-RFECV) is employed. Second, a shallow convolutional neural network (named ThermographicNet) is constructed to extract deep physiological features from the thermal source images. Finally, the two heterogeneous features are integrated at an early stage and input into a support vector machine (SVM) for porcine cough recognition. Through rigorous experimentation, the performance of the proposed fusion approach is evaluated, achieving an impressive accuracy of 98.79% in recognizing porcine cough. These results further underscore the effectiveness of combining acoustic source features with heterogeneous deep thermal source features, thereby establishing a robust feature representation for porcine cough recognition.

Comparison of convolutional neural network models for user’s facial recognition

This paper compares well-known convolutional neural networks (CNN) models for facial recognition. For this, it uses its database created from two registered users and an additional category of unknown persons. Eight different base models of convolutional architectures were compared by transfer of learning, and two additional proposed models called shallow CNN and shallow directed acyclic graph with CNN (DAG-CNN), which are architectures with little depth (six convolution layers). Within the tests with the database, the best results were obtained by the GoogLeNet and ResNet-101 models, managing to classify 100% of the images, even without confusing people outside the two users. However, in an additional real-time test, in which one of the users had his style changed, the models that showed the greatest robustness in this situation were the Inception and the ResNet-101, being able to maintain constant recognition. This demonstrated that the networks of greater depth manage to learn more detailed features of the users' faces, unlike those of shallower ones; their learning of features is more generalized. Declare the full term of an abbreviation/acronym when it is mentioned for the first time.

Impact of PSF misestimation and galaxy population bias on precision shear measurement using a CNN

ABSTRACT Weak gravitational lensing of distant galaxies provides a powerful probe of dark energy. The aim of this study is to investigate the application of convolutional neural networks (CNNs) to precision shear estimation. In particular, using a shallow CNN, we explore the impact of point spread function (PSF) misestimation and ‘galaxy population bias’ (including ‘distribution bias’ and ‘morphology bias’), focusing on the accuracy requirements of next generation surveys. We simulate a population of noisy disc and elliptical galaxies and adopt a PSF that is representative of a Euclid-like survey. We quantify the accuracy achieved by the CNN, assuming a linear relationship between the estimated and true shears and measure the multiplicative (m) and additive (c) biases. We make use of an unconventional loss function to mitigate the effects of noise bias and measure m and c when we use either: (i) an incorrect galaxy ellipticity distribution or size–magnitude relation, or the wrong ratio of morphological types, to describe the population of galaxies (distribution bias); (ii) an incorrect galaxy light profile (morphology bias); or (iii) a PSF with size or ellipticity offset from its true value (PSF misestimation). We compare our results to the Euclid requirements on the knowledge of the PSF model shape and size. Finally, we outline further work to build on the promising potential of CNNs in precision shear estimation.

Optimized Convolutional Forest by Particle Swarm Optimizer for Pothole Detection

Poor road maintenance leads to potholes on the road. Potholes are responsible for road accidents and even deaths in developed and developing countries. Detecting and filling road potholes is an essential part of road maintenance. Sustaining a reliable and safe road for communication depends on pothole detection. This study presents a novel combination of a convolutional neural network and an optimized machine-learning model by a heuristic algorithm for pothole detection. The proposed method comprises a shallow convolutional neural network for feature extraction and an optimized random forest model for pothole detection. The proposed model initially uses the shallow convolutional layer to extract feature sets from input pictures. Then, the particle swarm optimizer is used to eliminate irrelevant features. Finally, a combination of random forest and a particle swarm optimizer is used for pothole detection. Particle swarm optimization indicates the best subset of the extracted feature set for final pothole detection. We added 171 pictures to the already available 665 pothole pictures to evaluate the proposed method. The test set was isolated from the training set, and we trained the model on k-fold cross-validation. The experimental result indicates 99.37% accuracy, 99.37% precision, 99.38% sensitivity, and 99.38% F1-score for discriminating potholes from roads without potholes by proposed methods. The response time of the proposed method for pothole detection is 0.02 s. The proposed method can be utilized for real-time pothole detection.

FPGA based Flexible Implementation of Light Weight Inference on Deep Convolutional Neural Networks

Standard Convolution (StdConv) is the main technique used in the state of the art Deep Convolutional Neural Networks (DCNNs). Fewer computations are achieved if Depthwise Separable Convolution technique (SepConv) is used as an alternative. A crucial issue in many applications like smart cameras and autonomous vehicles where low latency is essential stems from deploying a lightweight and low cost inference models. An acceptable accuracy should be kept with tolerable computations and memory access load. A flexible architecture for different DCNN convolution types and models is proposed. The flexibility comes from the sharing of one memory access unit with different types of layers regardless of the selected kernel size, by multiplying each weight vector by local operators with variant aperture. Moreover, one depthwise computation unit can be used for both standard and pointwise layers. The learnable parameters are quantized to 8-bits fixed point representation and that gives very limited reduction of accuracy and a considerable reduction of the Field-Programmable Gate Array (FPGA) resources. To reduce processing time, inter layer parallel computations are performed. The experiment is conducted by using grey scale ORL database with shallow Convolutional Neural Network (CNN) and the colored Canadian Institute for Advanced Research 10 classes (CIFAR-10) database with DCNN, and a comparable accuracies of 93% and 85.7% are achieved respectively using very low cost of Spartan 3E and moderate cost of zynq FPGA platforms

Can micro-expressions be used as a biomarker for autism spectrum disorder?

Early and accurate diagnosis of autism spectrum disorder (ASD) is crucial for effective intervention, yet it remains a significant challenge due to its complexity and variability. Micro-expressions are rapid, involuntary facial movements indicative of underlying emotional states. It is unknown whether micro-expression can serve as a valid bio-marker for ASD diagnosis. This study introduces a novel machine-learning (ML) framework that advances ASD diagnostics by focusing on facial micro-expressions. We applied cutting-edge algorithms to detect and analyze these micro-expressions from video data, aiming to identify distinctive patterns that could differentiate individuals with ASD from typically developing peers. Our computational approach included three key components: (1) micro-expression spotting using Shallow Optical Flow Three-stream CNN (SOFTNet), (2) feature extraction via Micron-BERT, and (3) classification with majority voting of three competing models (MLP, SVM, and ResNet). Despite the sophisticated methodology, the ML framework's ability to reliably identify ASD-specific patterns was limited by the quality of video data. This limitation raised concerns about the efficacy of using micro-expressions for ASD diagnostics and pointed to the necessity for enhanced video data quality. Our research has provided a cautious evaluation of micro-expression diagnostic value, underscoring the need for advancements in behavioral imaging and multimodal AI technology to leverage the full capabilities of ML in an ASD-specific clinical context.

Cost Volume Aggregation in Stereo Matching Revisited: A Disparity Classification Perspective.

Cost aggregation plays a critical role in existing stereo matching methods. In this paper, we revisit cost aggregation in stereo matching from disparity classification and propose a generic yet efficient Disparity Context Aggregation (DCA) module to improve the performance of CNN-based methods. Our approach is based on an insight that a coarse disparity class prior is beneficial to disparity regression. To obtain such a prior, we first classify pixels in an image into several disparity classes and treat pixels within the same class as homogeneous regions. We then generate homogeneous region representations and incorporate these representations into the cost volume to suppress irrelevant information while enhancing the matching ability for cost aggregation. With the help of homogeneous region representations, efficient and informative cost aggregation can be achieved with only a shallow 3D CNN. Our DCA module is fully-differentiable and well-compatible with different network architectures, which can be seamlessly plugged into existing networks to improve performance with small additional overheads. It is demonstrated that our DCA module can effectively exploit disparity class priors to improve the performance of cost aggregation. Based on our DCA, we design a highly accurate network named DCANet, which achieves state-of-the-art performance on several benchmarks.

Dual‐branch feature extraction network combined with Transformer and CNN for polyp segmentation

AbstractTo overcome the difficulty of accurate polyp segmentation, a novel encoder–decoder network DFETC‐Net is proposed, in which two encoders based on Swin Transformer and CNN are utilized to extract the global and local features respectively. Further, a new self‐attention and convolution feature fusion module is designed to fuse the two branch features to enhance the feature representative capability and alleviate the influence of the semantic gap. In the bottleneck, a new multi‐feature pyramid pooling module fuses all deep features from two branches to obtain multi‐scale information and promote segmentation accuracy. The coordinate attention is used in the skip connections between two shallow CNN blocks and corresponding decoder blocks to pay more attention to doubtful and complicated regions. Extensive experiments demonstrate the proposed network outperforms several state‐of‐the‐art methods in terms of both qualitative effects and quantitative measurements. All codes are available on https://github.com/LYJieH/DFETC-NET.

An AI approach to operationalise global daily PlanetScope satellite imagery for river water masking

Monitoring rivers is vital to manage the invaluable ecosystem services they provide, and also to mitigate the risks they pose to property and life through flooding and drought. Due to the vast extent and dynamic nature of river systems, Earth Observation (EO) is one of the best ways to measure river characteristics. As a first step, EO-based river monitoring often requires extraction of accurate pixel-level water masks, but satellite images traditionally used for this purpose suffer from limited spatial and/or temporal resolution. We address this problem by applying a novel Convolutional Neural Network (CNN)-based model to automate water mask extraction from daily 3 m resolution PlanetScope satellite imagery. Notably, this approach overcomes radiometric issues that frequently present limitations when working with CubeSat data. We test our classification model on 36 rivers across 12 global terrestrial biomes (as proxies for the environmental and physical characteristics that lead to the variability in catchments around the globe). Using a relatively shallow CNN classification model, our approach produced a median F1 accuracy score of 0.93, suggesting that a compact and efficient CNN-based model can work as well as, if not better than, the very deep neural networks conventionally used in similar studies, whilst requiring less training data and computational power. We further show that our model, specialised to the task at hand, performs better than a state-of-the-art Fully Convolutional Neural Network (FCN) that struggles with the highly variable image quality from PlanetScope. Although classifying rivers that were narrower than 60 m, anastomosed or highly urbanised was slightly less successful than our other test images, we showed that fine tuning could circumvent these limitations to some degree. Indeed, fine tuning carried out on the Ottawa River, Canada, by including just 5 additional site-specific training images significantly improved classification accuracy (F1 increased from 0.81 to 0.90, p < 0.01). Overall, our results show that CNN-based classification applied to PlanetScope imagery is a viable tool for producing accurate, temporally dynamic river water masks, opening up possibilities for river monitoring investigations where high temporal variability data is essential.

Category-aware Recycle Classification using Convolutional Neural Networks

Generally, the filtration of non-recyclable garbage is conducted in order to make a material recycling process efficient. However, this filtration task is very time-consuming and costly, since it is generally done manually. Thus, it is significantly considered that an automation technique is necessary in solving this problem. There have been a number of attempts to resolve this problem. The previous methods used the shallow convolutional neural network to compose their network. However, these methods tend to display poor accuracy, which makes it impossible to be applied in real-life situations. To solve this problem, I propose a novel category-aware recycle classification system. The proposed system is composed of three models, which are garbage feature extractor, garbage classifier, and recycle classifier. The input image is fed to the garbage feature extractor and then converted into feature maps. The feature maps are then fed to each the garbage classifier and the recycle classifier. The garbage classifier predicts the category of the input garbage image while the recycle classifier determines if the input is recyclable or not. Through experiments, the proposed method outperforms the other state-of-the-art methods by a maximum 20.2% difference in accuracy on Garbage Classification and Industrial and Residential Waste datasets.

Selecting hyperspectral bands and extracting features with a custom shallow convolutional neural network to classify citrus peel defects

Citrus Black Spot (CBS) causes considerable damage to the Florida citrus industry. Early detection of CBS, especially in the presence of other peel blemishes, would enable better mapping and control of CBS spread, reduce wasted fruit, and permit early removal of culls from the packing stream. Oranges whose peels bore the symptoms of four defects/disease (CBS, greasy spot, melanose, and wind scar), as well as a normal control group, were imaged with a hyperspectral imaging system. Principal Component Analysis- (PCA) and Linear Discriminant Analysis (LDA) -based methods were employed to select bands from these images, and a custom convolutional neural network (CNN) for feature extraction was trained with these bands. The extracted features permitted classification of the peel conditions with four classifiers: SoftMax, Support Vector Machines (SVM), Random Forest Classifier (RFC), and K-Nearest Neighbors (KNN). A mean overall accuracy of 94.9 % was achieved using an SVM classifier on five bands selected with PCA, and 90.2 % with LDA-selected bands. These results show the potential of CNNs to extract features for automated postharvest citrus inspection.

A CNN-SVM Study Based on the Fusion of Spectrogram and Thermal Imaging Heterogeneous Features for Pig Cough Recognition in Field Situation

Accurate identification of pig cough is essential for comprehensive monitoring and diagnosis of the respiratory health status of pigs. It contributes to stress-free animal health management, reduces pig mortality and improves the economics of farming. Creating a representative multisource signal signature of pig cough is a critical step in achieving automatic recognition of pig cough. For this reason, in this paper, we propose a feature fusion classification method that combines the spectrogram deep features and thermal image deep features to be fed into a support vector machine (SVM) classifier to accomplish cough classification. First, we use a time–frequency transformation algorithm to convert a one-dimensional cough sound signal into a two-dimensional acoustic spectrogram. Then, the corresponding heterogeneous deep features are extracted from the cough spectrogram and thermal image by fine-tuning Lenet-5 and a customized CoughRNet shallow convolutional neural network. Finally, we employ an early fusion technique to align and splice the extracted heterogeneous deep features and feed them into an SVM for the automatic classification task of pig cough. Our study evaluates the classification performance, recognition speed and model size of the proposed deep feature fusion classification network with satisfactory results. Experimental results show that the method achieves 99.77% accuracy in pig cough recognition. This further demonstrates the effectiveness of combining abstract heterogeneous sound and thermal image deep features as a method for automated detection of pig respiratory health.

Automated breast tumor ultrasound image segmentation with hybrid UNet and classification using fine-tuned CNN model

IntroductionBreast cancer stands as the second most deadly form of cancer among women worldwide. Early diagnosis and treatment can significantly mitigate mortality rates. PurposeThe study aims to classify breast ultrasound images into benign and malignant tumors. This approach involves segmenting the breast's region of interest (ROI) employing an optimized UNet architecture and classifying the ROIs through an optimized shallow CNN model utilizing an ablation study. MethodSeveral image processing techniques are utilized to improve image quality by removing text, artifacts, and speckle noise, and statistical analysis is done to check the enhanced image quality is satisfactory. With the processed dataset, the segmentation of breast tumor ROI is carried out, optimizing the UNet model through an ablation study where the architectural configuration and hyperparameters are altered. After obtaining the tumor ROIs from the fine-tuned UNet model (RKO-UNet), an optimized CNN model is employed to classify the tumor into benign and malignant classes. To enhance the CNN model's performance, an ablation study is conducted, coupled with the integration of an attention unit. The model's performance is further assessed by classifying breast cancer with mammogram images. ResultThe proposed classification model (RKONet-13) results in an accuracy of 98.41 %. The performance of the proposed model is further compared with five transfer learning models for both pre-segmented and post-segmented datasets. K-fold cross-validation is done to assess the proposed RKONet-13 model's performance stability. Furthermore, the performance of the proposed model is compared with previous literature, where the proposed model outperforms existing methods, demonstrating its effectiveness in breast cancer diagnosis. Lastly, the model demonstrates its robustness for breast cancer classification, delivering an exceptional performance of 96.21 % on a mammogram dataset. ConclusionThe efficacy of this study relies on image pre-processing, segmentation with hybrid attention UNet, and classification with fine-tuned robust CNN model. This comprehensive approach aims to determine an effective technique for detecting breast cancer within ultrasound images.

Detection and Localization of Albas Velvet Goats Based on YOLOv4.

In order to achieve goat localization to help prevent goats from wandering, we proposed an efficient target localization method based on machine vision. Albas velvet goats from a farm in Ertok Banner, Ordos City, Inner Mongolia Autonomous Region, China, were the main objects of study. First, we proposed detecting the goats using a shallow convolutional neural network, ShallowSE, with the channel attention mechanism SENet, the GeLU activation function and layer normalization. Second, we designed three fully connected coordinate regression network models to predict the spatial coordinates of the goats. Finally, the target detection algorithm and the coordinate regression algorithm were combined to localize the flock. We experimentally confirmed the proposed method using our dataset. The proposed algorithm obtained a good detection accuracy and successful localization rate compared to other popular algorithms. The overall number of parameters in the target detection algorithm model was only 4.5 M. The average detection accuracy reached 95.89% and the detection time was only 8.5 ms. The average localization error of the group localization algorithm was only 0.94 m and the localization time was 0.21 s. In conclusion, the method achieved fast and accurate localization, which helped to rationalize the use of grassland resources and to promote the sustainable development of rangelands.

Identifying a Suitable Signal Processing Technique for MI EEG Data

Motor imagery (MI) electroencephalography (EEG) technology is acquiring great attention from researchers due to its remarkable real-world applications. EEG signals inherit a high degree of non-stationarity, making their analysis not modest. Hence, choosing an appropriate signal processing approach becomes crucial. This comparative paper aims to identify a suitable signal processing method among famous approaches, namely short-time Fourier transform (STFT), continuous wavelet transform (CWT), and two variations of discrete wavelet transform maximal overlap DWT (MODWT) and MODWT multiresolution analysis (MODWTMRA). Different mother wavelet basis filters experimented with wavelet methods: Morse, Amor, Bump, Symlets, Daubechies, Coiflets, and Fejér-Korovkin. The different methods were tested on the classification of the right-hand and left-hand motor imagery tasks using the brain-computer interface (BCI) competition IV 2b dataset. A shallow convolutional neural network containing a single convolution layer was first trained and then used for classification. The experimental outcomes verified that MI EEG signals can be better analyzed and recognized using the maximal overlap-based signal processing methods. The classification accuracy proved that MODWT and MODWTMRA with the Symlets wavelet outperformed the other methods.

A Voice Disease Detection Method Based on MFCCs and Shallow CNN

The incidence rate of voice diseases is increasing year by year. The use of software for remote diagnosis is a technical development trend and has important practical value. Among voice diseases, common diseases that cause hoarseness include spasmodic dysphonia, vocal cord paralysis, vocal nodule, and vocal cord polyp. This paper presents a voice disease detection method that can be applied in a wide range of clinical. We cooperated with Xiangya Hospital of Central South University to collect voice samples from 352 different patients. The Mel Frequency Cepstrum Coefficient (MFCC) parameters are extracted as input features to describe the voice in the form of data. An innovative model combining MFCC parameters and single convolution layer CNN is proposed for fast calculation and classification. The highest accuracy we achieved was 92%, it is fully ahead of the original research results and internationally advanced. And we use advanced voice function assessment databases (AVFAD) to evaluate the generalization ability of the method we proposed, which achieved an accuracy rate of 98%. Experiments on clinical and standard datasets show that for the pathological detection of voice diseases, our method has greatly improved in accuracy and computational efficiency.

S3DCN-OLSR: A shallow 3D CNN method for online learning state recognition

The repeated recurrence of COVID-19 has significantly disrupted learning for students in face-to-face instructional settings. While moving from offline to online instruction has proven to be one of the best solutions, classroom management and capturing students' learning states have emerged as important challenges with the increasing popularity of online instruction. To address these challenges, in this paper we propose an online learning status recognition method based on shallow 3D convolution (S3DC-OLSR) for online students, to identify students' online learning states by analysing their micro-expressions. Specifically, we first use the data augmentation method proposed in this paper to decompose the students' online video file into three features: horizontal component of optical flow, vertical component of optical flow and optical amplitude. Next, the students' online learning status is recognised by feeding the processed data into a shallow 3D convolution neural network. To test the performance of our method, we conduct extensive experiments on the CASME II and SMIC datasets, and the results indicate that our method outperforms the other state-of-the-art methods considered in terms of recognition accuracy, UF1 and UAR, which demonstrates the superiority of our method in identifying students’ online learning states.

Deep learning based effective feature extraction for active sonar target detection

We propose a deep learning (DL) model suitable for classifying target and non-target using a small amount of active sonar data. The proposed model uses two hand-crafted features (STFT and CQT) extracted from the same raw active sonar data, to complement each other and enhance the generalization of DL under insufficient data. The attention-based complementary learning module in proposed model reinforces one feature by referring to the other feature. The comprehensive feature from the complementary learning module pass through a shallow layer CNN and classify targets and non-targets. To verify the performance of the proposed model, we compared it with prevalent deep learning models including ResNet and ViT in terms of generalization performance and learning stability using two real-ocean datasets. The generalization performance of the proposed model having much smaller number of parameters was similar to or superior to the existing deep learning models depending on the training dataset, while the proposed model had the best learning stability for the two datasets.

A New Architecture of a Complex-Valued Convolutional Neural Network for PolSAR Image Classification

Polarimetric synthetic aperture radar (PolSAR) image classification has been an important area of research due to its wide range of applications. Traditional machine learning methods were insufficient in achieving satisfactory results before the advent of deep learning. Results have significantly improved with the widespread use of deep learning in PolSAR image classification. However, the challenge of reconciling the complex-valued inputs of PolSAR images with the real-valued models of deep learning remains unsolved. Current complex-valued deep learning models treat complex numbers as two distinct real numbers, providing limited assistance in PolSAR image classification results. This paper proposes a novel, complex-valued deep learning approach for PolSAR image classification to address this issue. The approach includes amplitude-based max pooling, complex-valued nonlinear activation, and a cross-entropy loss function based on complex-valued probability. Amplitude-based max pooling reduces computational effort while preserving the most valuable complex-valued features. Complex-valued nonlinear activation maps feature into a high-dimensional complex-domain space, producing the most discriminative features. The complex-valued cross-entropy loss function computes the classification loss using the complex-valued model output and dataset labels, resulting in more accurate and robust classification results. The proposed method was applied to a shallow CNN, deep CNN, FCN, and SegNet, and its effectiveness was verified on three public datasets. The results showed that the method achieved optimal classification results on any model and dataset.