Small Convolutional Neural Network Research Articles

A Convolutional Neural Network to Classify Phytoplankton Images Along the West Antarctic Peninsula

AbstractHigh-resolution optical imaging systems are quickly becoming universal tools to characterize and quantify microbial diversity in marine ecosystems. Automated classification systems such as convolutional neural networks (CNNs) are often developed to identify species within the immense number of images (e.g., millions per month) collected. The goal of our study was to develop a CNN to classify phytoplankton images collected with an Imaging FlowCytobot for the Palmer Antarctica Long-Term Ecological Research project. A relatively small CNN (~2 million parameters) was developed and trained using a subset of manually identified images, resulting in an overall test accuracy, recall, and f1-score of 93.8, 93.7, and 93.7%, respectively, on a balanced dataset. However, the f1-score dropped to 46.5% when tested on a dataset of 10,269 new images drawn from the natural environment without balancing classes. This decrease is likely due to highly imbalanced class distributions dominated by smaller, less differentiable cells, high intraclass variance, and interclass morphological similarities of cells in naturally occurring phytoplankton assemblages. As a case study to illustrate the value of the model, it was used to predict taxonomic classifications (ranging from genus to class) of phytoplankton at Palmer Station, Antarctica, from late austral spring to early autumn in 2017‐2018 and 2018‐2019. The CNN was generally able to identify important seasonal dynamics such as the shift from large centric diatoms to small pennate diatoms in both years, which is thought to be driven by increases in glacial meltwater from January to March. This shift in particle size distribution has significant implications for the ecology and biogeochemistry of these waters. Moving forward, we hope to further increase the accuracy of our model to better characterize coastal phytoplankton communities threatened by rapidly changing environmental conditions.

A Convolutional Neural Network to Classify Phytoplankton Images Along the West Antarctic Peninsula

Abstract High-resolution optical imaging systems are quickly becoming universal tools to characterize and quantify microbial diversity in marine ecosystems. Automated classification systems such as convolutional neural networks (CNNs) are often developed to identify species within the immense number of images (e.g., millions per month) collected. The goal of our study was to develop a CNN to classify phytoplankton images collected with an Imaging FlowCytobot for the Palmer Antarctica Long-Term Ecological Research project. A relatively small CNN (~2 million parameters) was developed and trained using a subset of manually identified images, resulting in an overall test accuracy, recall, and f1-score of 93.8, 93.7, and 93.7%, respectively, on a balanced dataset. However, the f1-score dropped to 46.5% when tested on a dataset of 10,269 new images drawn from the natural environment without balancing classes. This decrease is likely due to highly imbalanced class distributions dominated by smaller, less differentiable cells, high intraclass variance, and interclass morphological similarities of cells in naturally occurring phytoplankton assemblages. As a case study to illustrate the value of the model, it was used to predict taxonomic classifications (ranging from genus to class) of phytoplankton at Palmer Station, Antarctica, from late austral spring to early autumn in 2017‐2018 and 2018‐2019. The CNN was generally able to identify important seasonal dynamics such as the shift from large centric diatoms to small pennate diatoms in both years, which is thought to be driven by increases in glacial meltwater from January to March. This shift in particle size distribution has significant implications for the ecology and biogeochemistry of these waters. Moving forward, we hope to further increase the accuracy of our model to better characterize coastal phytoplankton communities threatened by rapidly changing environmental conditions.

Training Vision Transformers in Federated Learning with Limited Edge-Device Resources

Vision transformers (ViTs) demonstrate exceptional performance in numerous computer vision tasks owing to their self-attention modules. Despite improved network performance, transformers frequently require significant computational resources. The increasing need for data privacy has encouraged the development of federated learning (FL). Traditional FL places a computing burden on edge devices. However, ViTs cannot be directly applied through FL on resource-constrained edge devices. To utilize the powerful ViT structure, we reformulated FL as a federated knowledge distillation training algorithm called FedVKD. FedVKD uses an alternating minimization strategy to train small convolutional neural networks on edge nodes and periodically transfers their knowledge to a large server-side transformer encoder via knowledge distillation. FedVKD affords the benefits of reduced edge-computing load and improved performance for vision tasks, while preserving FedGKT-like asynchronous training. We used four datasets and their non-IID variations to test the proposed FedVKD. When utilizing a larger dataset, FedVKD achieved higher accuracy than FedGKT and FedAvg.

A walk in the black-box: 3D visualization of large neural networks in virtual reality

Within the last decade Deep Learning has become a tool for solving challenging problems like image recognition. Still, Convolutional Neural Networks (CNNs) are considered black-boxes, which are difficult to understand by humans. Hence, there is an urge to visualize CNN architectures, their internal processes and what they actually learn. Previously, virtual realityhas been successfully applied to display small CNNs in immersive 3D environments. In this work, we address the problem how to feasibly render large-scale CNNs, thereby enabling the visualization of popular architectures with ten thousands of feature maps and branches in the computational graph in 3D. Our software ”DeepVisionVR” enables the user to freely walk through the layered network, pick up and place images, move/scale layers for better readability, perform feature visualization and export the results. We also provide a novel Pytorch module to dynamically link PyTorch with Unity, which gives developers and researchers a convenient interface to visualize their own architectures. The visualization is directly created from the PyTorch class that defines the Pytorch model used for training and testing. This approach allows full access to the network’s internals and direct control over what exactly is visualized. In a use-case study, we apply the module to analyze models with different generalization abilities in order to understand how networks memorize images. We train two recent architectures, CovidResNet and CovidDenseNet on the Caltech101 and the SARS-CoV-2 datasets and find that bad generalization is driven by high-frequency features and the susceptibility to specific pixel arrangements, leading to implications for the practical application of CNNs. The code is available on Github https://github.com/Criscraft/DeepVisionVR.

Smartphone-based artificial intelligence using a transfer learning algorithm for the detection and diagnosis of middle ear diseases: A retrospective deep learning study.

SummaryBackgroundMiddle ear diseases such as otitis media and middle ear effusion, for which diagnoses are often delayed or misdiagnosed, are among the most common issues faced by clinicians providing primary care for children and adolescents. Artificial intelligence (AI) has the potential to assist clinicians in the detection and diagnosis of middle ear diseases through imaging.MethodsOtoendoscopic images obtained by otolaryngologists from Taipei Veterans General Hospital in Taiwan between Jany 1, 2011 to Dec 31, 2019 were collected retrospectively and de-identified. The images were entered into convolutional neural network (CNN) training models after data pre-processing, augmentation and splitting. To differentiate sophisticated middle ear diseases, nine CNN-based models were constructed to recognize middle ear diseases. The best-performing models were chosen and ensembled in a small CNN for mobile device use. The pretrained model was converted into the smartphone-based program, and the utility was evaluated in terms of detecting and classifying ten middle ear diseases based on otoendoscopic images. A class activation map (CAM) was also used to identify key features for CNN classification. The performance of each classifier was determined by its accuracy, precision, recall, and F1-score.FindingsA total of 2820 clinical eardrum images were collected for model training. The programme achieved a high detection accuracy for binary outcomes (pass/refer) of otoendoscopic images and ten different disease categories, with an accuracy reaching 98.0% after model optimisation. Furthermore, the application presented a smooth recognition process and a user-friendly interface and demonstrated excellent performance, with an accuracy of 97.6%. A fifty-question questionnaire related to middle ear diseases was designed for practitioners with different levels of clinical experience. The AI-empowered mobile algorithm's detection accuracy was generally superior to that of general physicians, resident doctors, and otolaryngology specialists (36.0%, 80.0% and 90.0%, respectively). Our results show that the proposed method provides sufficient treatment recommendations that are comparable to those of specialists.InterpretationWe developed a deep learning model that can detect and classify middle ear diseases. The use of smartphone-based point-of-care diagnostic devices with AI-empowered automated classification can provide real-world smart medical solutions for the diagnosis of middle ear diseases and telemedicine.FundingThis study was supported by grants from the Ministry of Science and Technology (MOST110-2622-8-075-001, MOST110-2320-B-075-004-MY3, MOST-110-2634-F-A49 -005, MOST110-2745-B-075A-001A and MOST110-2221-E-075-005), Veterans General Hospitals and University System of Taiwan Joint Research Program (VGHUST111-G6-11-2, VGHUST111c-140), and Taipei Veterans General Hospital (V111E-002-3).

Survey on deep learning based computer vision for sonar imagery

Research on the automatic analysis of sonar images has focused on classical, i.e. non deep learning based, approaches for a long time. Over the past 15 years, however, the application of deep learning in this research field has constantly grown. This paper gives a broad overview of past and current research involving deep learning for feature extraction, classification, detection and segmentation of sidescan and synthetic aperture sonar imagery. Most research in this field has been directed towards the investigation of convolutional neural networks (CNN) for feature extraction and classification tasks, with the result that even small CNNs with up to four layers outperform conventional methods.The purpose of this work is twofold. On one hand, due to the quick development of deep learning it serves as an introduction for researchers, either just starting their work in this specific field or working on classical methods for the past years, and helps them to learn about the recent achievements. On the other hand, our main goal is to guide further research in this field by identifying main research gaps to bridge. We propose to leverage the research in this field by combining available data into an open source dataset as well as carrying out comparative studies on developed deep learning methods.

Adversarial Images Against Super-Resolution Convolutional Neural Networks for Free

Super-Resolution Convolutional Neural Networks (SRCNNs) with their ability to generate highresolution images from low-resolution counterparts, exacerbate the privacy concerns emerging from automated Convolutional Neural Networks (CNNs)-based image classifiers. In this work, we hypothesize and empirically show that adversarial examples learned over CNN image classifiers can survive processing by SRCNNs and lead them to generate poor quality images that are hard to classify correctly. We demonstrate that a user with a small CNN is able to learn adversarial noise without requiring any customization for SRCNNs and thwart the privacy threat posed by a pipeline of SRCNN and CNN classifiers (95.8% fooling rate for Fast Gradient Sign with ε = 0.03). We evaluate the survivability of adversarial images generated in both black-box and white-box settings and show that black-box adversarial learning (when both CNN classifier and SRCNN are unknown) is at least as effective as white-box adversarial learning (when only CNN classifier is known). We also assess our hypothesis on adversarial robust CNNs and observe that the supper-resolved white-box adversarial examples can fool these CNNs more than 71.5% of the time.

Image-Adaptive YOLO for Object Detection in Adverse Weather Conditions

Though deep learning-based object detection methods have achieved promising results on the conventional datasets, it is still challenging to locate objects from the low-quality images captured in adverse weather conditions. The existing methods either have difficulties in balancing the tasks of image enhancement and object detection, or often ignore the latent information beneficial for detection. To alleviate this problem, we propose a novel Image-Adaptive YOLO (IA-YOLO) framework, where each image can be adaptively enhanced for better detection performance. Specifically, a differentiable image processing (DIP) module is presented to take into account the adverse weather conditions for YOLO detector, whose parameters are predicted by a small convolutional neural network (CNN-PP). We learn CNN-PP and YOLOv3 jointly in an end-to-end fashion, which ensures that CNN-PP can learn an appropriate DIP to enhance the image for detection in a weakly supervised manner. Our proposed IA-YOLO approach can adaptively process images in both normal and adverse weather conditions. The experimental results are very encouraging, demonstrating the effectiveness of our proposed IA-YOLO method in both foggy and low-light scenarios. The source code can be found at https://github.com/wenyyu/Image-Adaptive-YOLO.

Classification of Underwater Fish Images and Videos via Very Small Convolutional Neural Networks

The automatic classification of fish species appearing in images and videos from underwater cameras is a challenging task, albeit one with a large potential impact in environment conservation, marine fauna health assessment, and fishing policy. Deep neural network models, such as convolutional neural networks, are a popular solution to image recognition problems. However, such models typically require very large datasets to train millions of model parameters. Because underwater fish image and video datasets are scarce, non-uniform, and often extremely unbalanced, deep neural networks may be inadequately trained, and undergo a much larger risk of overfitting. In this paper, we propose small convolutional neural networks as a practical engineering solution that helps tackle fish image classification. The concept of “small” refers to the number of parameters of the resulting models: smaller models are lighter to run on low-power devices, and drain fewer resources per execution. This is especially relevant for fish recognition systems that run unattended on offshore platforms, often on embedded hardware. Here, established deep neural network models would require too many computational resources. We show that even networks with little more than 12,000 parameters provide an acceptable working degree of accuracy in the classification task (almost 42% for six fish species), even when trained on small and unbalanced datasets. If the fish images come from videos, we augment the data via a low-complexity object tracking algorithm, increasing the accuracy to almost 49% for six fish species. We tested the networks with images obtained from the deployments of an experimental system in the Mediterranean sea, showing a good level of accuracy given the low quality of the dataset.

Exploiting pretrained CNN models for the development of an EEG-based robust BCI framework

Identifying motor and mental imagery electroencephalography (EEG) signals is imperative to realizing automated, robust brain-computer interface (BCI) systems. In the present study, we proposed a pretrained convolutional neural network (CNN)-based new automated framework feasible for robust BCI systems with small and ample samples of motor and mental imagery EEG training data. The framework is explored by investigating the implications of different limiting factors, such as learning rates and optimizers, processed versus unprocessed scalograms, and features derived from untuned pretrained models in small, medium, and large pretrained CNN models. The experiments were performed on three public datasets obtained from BCI Competition III. The datasets were denoised with multiscale principal component analysis, and time-frequency scalograms were obtained by employing a continuous wavelet transform. The scalograms were fed into several variants of ten pretrained models for feature extraction and identification of different EEG tasks. The experimental results showed that ShuffleNet yielded the maximum average classification accuracy of 99.52% using an RMSProp optimizer with a learning rate of 0.000 1. It was observed that low learning rates converge to more optimal performances compared to high learning rates. Moreover, noisy scalograms and features extracted from untuned networks resulted in slightly lower performance than denoised scalograms and tuned networks, respectively. The overall results suggest that pretrained models are robust when identifying EEG signals because of their ability to preserve the time-frequency structure of EEG signals and promising classification outcomes.

A Fast Indoor Positioning Using a Knowledge-Distilled Convolutional Neural Network (KD-CNN)

Fingerprint-based indoor positioning systems (F-IPS) may provide inexpensive solutions to GPS-denied environments. Most F-IPSs adopt traditional machine learning for position prediction, resulting in low accuracy. Deep neural networks (DNN) were recently employed for F-IPSs to minimize positioning errors. Nevertheless, a DNN-IPS fails to guarantee high accuracy in dynamic environments as it is sensitive to changes in the input data. A convolutional neural network (CNN) is recommended to replace DNN due to its capability to learn the overall topology of fingerprinting images and capture highly abstract features. Due to the convolution process and image representation, CNN-IPS incurs prohibitive storage and computational requirement for implementation on resource-limited devices. This paper incorporates knowledge distillation (KD) into CNN-IPS to distil knowledge from large deep CNNs into small CNNs. The pretrained teacher network uses the soft probability output where the score vector from the trained network is converted into a probability distribution, which is softened by the temperature hyperparameter, leading to a more simplified model. Based on the numerical results, KD-CNN-IPS manifests better localization performance where 79.84% of the positioning errors are within 2 meters while its testing time is only 79.68% of that of the teacher model. Compared to the CNN-IPS, KD-CNN-IPS with precisely the same architecture and size could achieve a performance improvement of 13.65% in terms of the average positioning error.

An Attention Mechanism for Combination of CNN and VAE for Image-Based Malware Classification

Currently, malware is increasing in both number and complexity dramatically. Several techniques and methodologies have been proposed to detect and neutralize malicious software. However, traditional methods based on the signatures or behaviors of malware often require considerable computational time and resources for feature engineering. Recent studies have applied machine learning to the problems of identifying and classifying malware families. Combining many state-of-the-art techniques has become popular but choosing the appropriate combination with high efficiency is still a problem. The classification performance has been significantly improved using complex neural network architectures. However, the more complex the network, the more resources it requires. This paper proposes a novel lightweight architecture by combining small Convolutional Neural Networks and advanced Variational Autoencoder, enhanced by channel and spatial attention mechanisms. We achieve overperformance and sufficient time through various experiments compared to other cutting-edge techniques using unbalanced and balanced Malimg datasets.

Low-Parameter Small Convolutional Neural Network Applied to Functional Medical Imaging of Tc-99m Trodat-1 Brain Single-Photon Emission Computed Tomography for Parkinson’s Disease

Parkinson’s disease (PD), a progressive disease that affects movement, is related to dopaminergic neuron degeneration. Tc-99m Trodat-1 brain (TRODAT) single-photon emission computed tomography (SPECT) aids the functional imaging of dopamine transporters and is used for dopaminergic neuron enumeration. Herein, we employed a convolutional neural network to facilitate PD diagnosis through TRODAT SPECT, which is simpler than models such as VGG16 and ResNet50. We retrospectively collected the data of 3188 patients (age range 20–107 years) who underwent TRODAT SPECT between June 2011 and December 2019. We developed a set of functional imaging multiclassification deep learning algorithms suitable for TRODAT SPECT on the basis of the annotations of medical experts. We then applied our self-proposed model and compared its results with those of four other models, including deep and machine learning models. TRODAT SPECT included three images collected from each patient: one presenting the maximum absorption of the metabolic function of the striatum and two adjacent images. An expert physician determined that our model’s accuracy, precision, recall, and F1-score were 0.98, 0.98, 0.98, and 0.98, respectively. Our TRODAT SPECT model provides an objective, more standardized classification correlating to the severity of PD-related diseases, thereby facilitating clinical diagnosis and preventing observer bias.

Use of Deep Learning to Improve the Computational Complexity of Reconstruction Algorithms in High Energy Physics

The optimization of reconstruction algorithms has become a key aspect in the field of experimental particle physics. Since technology has allowed gradually increasing the complexity of the measurements, the amount of data taken that needs to be interpreted has grown as well. This is the case with the LHCb experiment at CERN, where a major upgrade currently undergoing will considerably increase the data processing rate. This has presented the need to search for specific reconstruction techniques that aim to accelerate one of the most time consuming reconstruction algorithms in LHCb, the electromagnetic calorimeter clustering. Together with the use of deep learning techniques and the understanding of the current reconstruction algorithm, we propose a method that decomposes the reconstruction process into small parts that can be formulated as a cellular automaton. This approach is shown to benefit the generalized learning of small convolutional neural network architectures and also simplify the training dataset. Final results applied to a complete LHCb simulation reconstruction are compatible in terms of efficiency, and execute in nearly constant time with independence on the complexity of the data.

Combating data incompetence in pollen images detection and classification for pollinosis prevention

Automatic pollen images recognition is crucial for pollinosis symptoms prevention and treatment. The problem of pollen recognition can be efficiently solved using deep learning, however neural networks require tens of thousands of images to generalize. At the same time, the existing open pollen images datasets are very small. In this paper, we present a novel open pollen dataset annotated for both detection and classification tasks. Based on our dataset we study learning from a small data using different state-of-the-art approaches. For the detection task we propose to use our new Bayesian RetinaNet network, which models aleatoric uncertainty. We compare it with the baseline RetinaNet and demonstrate that our model allows for higher detection precision. For the classification task we compare the impact of pre-training on the synthetic images from generative adversarial networks (GANs) and metric-based few-shot learning. Namely, we pre-trained our small convolutional neural network and Siamese neural network classifiers on synthetic pollen images generated by two GANs: StyleGAN and Self-attention GAN. The best classifier is the convolutional neural network pre-trained on StyleGAN images. Our best models achieved 96.3% of mean average precision for the detection task and 97.7% of F1 measure for the classification task on 13 pollen plant species. We have implemented the best models in our pollen recognition web service, which is available for palynologists on request.

Are Microcontrollers Ready for Deep Learning-Based Human Activity Recognition?

The last decade has seen exponential growth in the field of deep learning with deep learning on microcontrollers a new frontier for this research area. This paper presents a case study about machine learning on microcontrollers, with a focus on human activity recognition using accelerometer data. We build machine learning classifiers suitable for execution on modern microcontrollers and evaluate their performance. Specifically, we compare Random Forests (RF), a classical machine learning technique, with Convolutional Neural Networks (CNN), in terms of classification accuracy and inference speed. The results show that RF classifiers achieve similar levels of classification accuracy while being several times faster than a small custom CNN model designed for the task. The RF and the custom CNN are also several orders of magnitude faster than state-of-the-art deep learning models. On the one hand, these findings confirm the feasibility of using deep learning on modern microcontrollers. On the other hand, they cast doubt on whether deep learning is the best approach for this application, especially if high inference speed and, thus, low energy consumption is the key objective.

Vascular Auscultation of Carotid Artery: Towards Biometric Identification and Verification of Individuals.

Background: Biometric sensing is a security method for protecting information and property. State-of-the-art biometric traits are behavioral and physiological in nature. However, they are vulnerable to tampering and forgery. Methods: The proposed approach uses blood flow sounds in the carotid artery as a source of biometric information. A handheld sensing device and an associated desktop application were built. Between 80 and 160 carotid recordings of 11 s in length were acquired from seven individuals each. Wavelet-based signal analysis was performed to assess the potential for biometric applications. Results: The acquired signals per individual proved to be consistent within one carotid sound recording and between multiple recordings spaced by several weeks. The averaged continuous wavelet transform spectra for all cardiac cycles of one recording showed specific spectral characteristics in the time-frequency domain, allowing for the discrimination of individuals, which could potentially serve as an individual fingerprint of the carotid sound. This is also supported by the quantitative analysis consisting of a small convolutional neural network, which was able to differentiate between different users with over 95% accuracy. Conclusion: The proposed approach and processing pipeline appeared promising for the discrimination of individuals. The biometrical recognition could clinically be used to obtain and highlight differences from a previously established personalized audio profile and subsequently could provide information on the source of the deviation as well as on its effects on the individual’s health. The limited number of individuals and recordings require a study in a larger population along with an investigation of the long-term spectral stability of carotid sounds to assess its potential as a biometric marker. Nevertheless, the approach opens the perspective for automatic feature extraction and classification.

Facial Expression Recognition Based on Transfer Learning and SVM

The facial expression datasets always have a problem: data with small amount or large amounts of data but also with large noisy. Both problems will affect the facial expression recognition accuracy of the model. A transfer learning method for facial expression recognition is proposed by combining the Convolutional Neural Network (CNN) and Support Vector Machine (SVM). SVM have good performance on small data sets and CNN based on transfer learning have better ability of feature extraction for large noisy data set. This method reduces the training time of model and increase the facial expression recognition accuracy. The experimental results show that the accuracy of the proposed method on the CK+ and FER2013 data sets has reached 99.6% and 68.1%.

Eye tracking based dyslexia detection using a holistic approach

A new detection method for cognitive impairments is presented utilizing an eye tracking signals in a text reading test. This research enhances published articles that extract combination of various features. It does so by processing entire eye-tracking records either in time or frequency whereas applying only basic signal pre-processing. Such signals were classified as a whole by Convolutional Neural Networks (CNN) that hierarchically extract substantial features scatter either in time or frequency and nonlinearly binds them using machine learning to minimize a detection error. In the experiments we used a 100 fold cross validation and a dataset containing signals of 185 subjects (88 subjects with low risk and 97 subjects with high risk of dyslexia). In a series of experiments it was found that magnitude spectrum based representation of time interpolated eye-tracking signals recorded the best results, i.e. an average accuracy of 96.6% was reached in comparison to 95.6% that is the best published result on the same database. These findings suggest that a holistic approach involving small but complex enough CNNs applied to properly pre-process and expressed signals provides even better results than a combination of meticulously selected well-known features.

Using Convolutional Neural Network and a Single Heartbeat for ECG Biometric Recognition.

The electrocardiogram (ECG) signal has become a popular biometric modality due to characteristics that make it suitable for developing reliable authentication systems. However, the long segment of signal required for recognition is still one of the limitations of existing ECG biometric recognition methods and affects its acceptability as a biometric modality. This paper investigates how a short segment of an ECG signal can be effectively used for biometric recognition, using deep-learning techniques. A small convolutional neural network (CNN) is designed to achieve better generalization capability by entropy enhancement of a short segment of a heartbeat signal. Additionally, it investigates how various blind and feature-dependent segments with different lengths affect the performance of the recognition system. Experiments were carried out on two databases for performance evaluation that included single and multisession records. In addition, a comparison was made between the performance of the proposed classifier and four well-known CNN models: GoogLeNet, ResNet, MobileNet and EfficientNet. Using a time–frequency domain representation of a short segment of an ECG signal around the R-peak, the proposed model achieved an accuracy of 99.90% for PTB, 98.20% for the ECG-ID mixed-session, and 94.18% for ECG-ID multisession datasets. Using the preprinted ResNet, we obtained 97.28% accuracy for 0.5-second segments around the R-peaks for ECG-ID multisession datasets, outperforming existing methods. It was found that the time–frequency domain representation of a short segment of an ECG signal can be feasible for biometric recognition by achieving better accuracy and acceptability of this modality.