Medical Image Classification Research Articles

Explainable Artificial Intelligence (XAI) for Deep Learning Based Medical Imaging Classification.

Recently, deep learning has gained significant attention as a noteworthy division of artificial intelligence (AI) due to its high accuracy and versatile applications. However, one of the major challenges of AI is the need for more interpretability, commonly referred to as the black-box problem. In this study, we introduce an explainable AI model for medical image classification to enhance the interpretability of the decision-making process. Our approach is based on segmenting the images to provide a better understanding of how the AI model arrives at its results. We evaluated our model on five datasets, including the COVID-19 and Pneumonia Chest X-ray dataset, Chest X-ray (COVID-19 and Pneumonia), COVID-19 Image Dataset (COVID-19, Viral Pneumonia, Normal), and COVID-19 Radiography Database. We achieved testing and validation accuracy of 90.6% on a relatively small dataset of 6432 images. Our proposed model improved accuracy and reduced time complexity, making it more practical for medical diagnosis. Our approach offers a more interpretable and transparent AI model that can enhance the accuracy and efficiency of medical diagnosis.

Idecomp: imbalance-aware decomposition for class-decomposed classification using conditional GANs

Medical image classification tasks frequently encounter challenges associated with class imbalance, resulting in biased model training and suboptimal classification performance. To address this issue, the combination of class decomposition and transfer learning has proven to be effective in classifying imbalanced medical imaging datasets. Nevertheless, in order to further augment the performance gains achieved through the utilisation of class decomposition within deep learning frameworks, we propose a novel model coined imbalance-Aware Decomposition for Class-Decomposed Classification (iDeComp) model. By incorporating a conditional Generative Adversarial Network (GAN) model, iDeComp is capable of generating additional samples specifically tailored to underrepresented decomposed subclasses. This paper investigates the application of iDeComp using two different medical imaging datasets. iDeComp selects underrepresented samples from the training set of the sublevel classes within each dataset, which are then employed to train separate conditional Deep Convolutional GAN (DCGAN) models and verification models. The conditional DCGAN model is responsible for generating additional samples, while the verification model critically evaluates the appropriateness of the synthesised images. Subsequently, the resulting augmented samples are utilized to train the classification model. To assess the effectiveness of iDeComp, we employ various evaluation metrics including accuracy, precision, recall, and F1 score. The results obtained from our experiments clearly indicate that iDeComp outperforms existing approaches in terms of classifying both imbalanced datasets.

A Bio-Inspired Method For Breast Histopathology Image Classification Using Transfer Learning

Breast cancer is one of the deadly cancer among the female population, and still a developing area of research in the field of medical imaging. The fatality rate is more in patients who are not early diagnosed and are given delayed treatment. Hence researchers are keeping their lot of efforts in developing breast cancer detection systems that could provide accurate diagnosis in the initial stages which are relied on medical imaging. Deep learning is offering key solutions to overcome many image classification tasks. Though deep learning techniques have extended their root to many medical fields even it suffers from the problem of lack of sufficient data. Convolutional Neural Networks are more preferred for medical image classification tasks. In this paper, we propose a transfer learning method that overcomes the issue of insufficient data. We use a familiar pre-trained network VGG-16 (Visual Geometric Group) + with Logistic Regression as a binary classifier. Since hyper-parameters of every CNN has a closer impact on the performance of the entire deep learning model, our method focus on optimizing hyper-parameters using particle swarm optimization which is a bio-inspired algorithm, The proposed model performs classification of Breast Histopathology images into benign and malignant images and produce better results. We use Break His Dataset to test our method and achieve an accuracy of around 96.9%. The experimental results show that this study has improved performance metrics when compared to other transfer learning methods.

Use of Ensemble Learning to Improve Performance of Known Convolutional Neural Networks for Mammography Classification

Convolutional neural networks and deep learning models represent the gold standard in medical image classification. Their innovative architectures have led to notable breakthroughs in image classification and feature extraction performance. However, these advancements often remain underutilized in the medical imaging field due to the scarcity of sufficient labeled data which are needed to leverage these new features fully. While many methodologies exhibit stellar performance on benchmark data sets like DDSM or Minimias, their efficacy drastically decreases when applied to real-world data sets. This study aims to develop a tool to streamline mammogram classification that maintains high reliability across different data sources. We use images from the DDSM data set and a proprietary data set, YERAL, which comprises 943 mammograms from Mexican patients. We evaluate the performance of ensemble learning algorithms combined with prevalent deep learning models such as Alexnet, VGG-16, and Inception. The computational results demonstrate the effectiveness of the proposed methodology, with models achieving 82% accuracy without overtaxing our hardware capabilities, and they also highlight the efficiency of ensemble algorithms in enhancing accuracy across all test cases.

Adversarial Attacks on Medical Image Classification.

Due to the growing number of medical images being produced by diverse radiological imaging techniques, radiography examinations with computer-aided diagnoses could greatly assist clinical applications. However, an imaging facility with just a one-pixel inaccuracy will lead to the inaccurate prediction of medical images. Misclassification may lead to the wrong clinical decision. This scenario is similar to the adversarial attacks on deep learning models. Therefore, one-pixel and multi-pixel level attacks on a Deep Neural Network (DNN) model trained on various medical image datasets are investigated in this study. Common multiclass and multi-label datasets are examined for one-pixel type attacks. Moreover, different experiments are conducted in order to determine how changing the number of pixels in the image may affect the classification performance and robustness of diverse DNN models. The experimental results show that it was difficult for the medical images to survive the pixel attacks, raising the issue of the accuracy of medical image classification and the importance of the model's ability to resist these attacks for a computer-aided diagnosis.

A deep convolutional neural network model for medical data classification from computed tomography images

AbstractMachine learning provides powerful techniques for several applications, including automated disease diagnosis through medical image classification. Recently, many studies reported that deep learning approaches have demonstrated significant performance and accuracy improvements over shallow learning techniques. The deep learning approaches have been used in many problems related to disease diagnoses, such as thyroid diagnosis, diabetic retinopathy detection, foetal localization, and breast cancer detection. Many deep learning methods have been reported in the recent past that uses medical images from various sources, such as healthcare providers and open data initiatives, and reported significant improvement in terms of precision, recall, and accuracy. This paper proposes a framework incorporating deep convolutional neural networks and an enhanced feature extraction technique for classifying medical data. To show the real‐world usability of the proposed approach, it has been used for the classification of COVID‐19 images from computed tomography scans. The experimental results show that the proposed approach outperformed some of the chosen baselines and obtained an accuracy of 98.91%, comparable with already reported accuracies.

Privacy-preserving continual learning methods for medical image classification: a comparative analysis.

The implementation of deep learning models for medical image classification poses significant challenges, including gradual performance degradation and limited adaptability to new diseases. However, frequent retraining of models is unfeasible and raises concerns about healthcare privacy due to the retention of prior patient data. To address these issues, this study investigated privacy-preserving continual learning methods as an alternative solution. We evaluated twelve privacy-preserving non-storage continual learning algorithms based deep learning models for classifying retinal diseases from public optical coherence tomography (OCT) images, in a class-incremental learning scenario. The OCT dataset comprises 108,309 OCT images. Its classes include normal (47.21%), drusen (7.96%), choroidal neovascularization (CNV) (34.35%), and diabetic macular edema (DME) (10.48%). Each class consisted of 250 testing images. For continuous training, the first task involved CNV and normal classes, the second task focused on DME class, and the third task included drusen class. All selected algorithms were further experimented with different training sequence combinations. The final model's average class accuracy was measured. The performance of the joint model obtained through retraining and the original finetune model without continual learning algorithms were compared. Additionally, a publicly available medical dataset for colon cancer detection based on histology slides was selected as a proof of concept, while the CIFAR10 dataset was included as the continual learning benchmark. Among the continual learning algorithms, Brain-inspired-replay (BIR) outperformed the others in the continual learning-based classification of retinal diseases from OCT images, achieving an accuracy of 62.00% (95% confidence interval: 59.36-64.64%), with consistent top performance observed in different training sequences. For colon cancer histology classification, Efficient Feature Transformations (EFT) attained the highest accuracy of 66.82% (95% confidence interval: 64.23-69.42%). In comparison, the joint model achieved accuracies of 90.76% and 89.28%, respectively. The finetune model demonstrated catastrophic forgetting in both datasets. Although the joint retraining model exhibited superior performance, continual learning holds promise in mitigating catastrophic forgetting and facilitating continual model updates while preserving privacy in healthcare deep learning models. Thus, it presents a highly promising solution for the long-term clinical deployment of such models.

Federated adaptive reweighting for medical image classification

Medical data sharing across institutes is crucial to large-scale multi-center studies and the development of real-world AI applications but suffers from serious privacy issues. A promising solution to address this challenge is federated learning, which typically aggregates a global model from heterogeneous data spread across numerous clients without exchanging data. However, the traditional federated learning algorithm (i.e., FedAvg) merely aggregates the locally distributed models according to the amount of data on each client and lacks the consideration of data heterogeneity. In this paper, we propose a novel Federated Adaptive Reweighting (FedAR) algorithm for medical image classification. FedAR employs a flexible re-weighting scheme that can balance adaptively the contributions of the amount of data and the performance of the local model on each client to the weight of that client. Specifically, we allow the amount of local data to contribute more to the weight of each client in the early training stage and let the performance of the local model play a more important role in the late stage. We have evaluated the proposed FedAR algorithm against the locally trained model, globally trained baseline, and two existing federated learning algorithms on the ISIC2018 dataset and Chest X-ray14 dataset under the settings with a variable number of clients. Our results suggest that FedAR is an effective federated learning algorithm that substantially outperforms existing federated learning approaches.

Eff-PCNet: An Efficient Pure CNN Network for Medical Image Classification

With the development of deep learning, convolutional neural networks (CNNs) and Transformer-based methods have become key techniques for medical image classification tasks. However, many current neural network models have problems such as high complexity, a large number of parameters, and large model sizes; such models obtain higher classification accuracy at the expense of lightweight networks. Moreover, such larger-scale models pose a great challenge for practical clinical applications. Meanwhile, Transformer and multi-layer perceptron (MLP) methods have some shortcomings in terms of local modeling capability and high model complexity, and need to be used on larger datasets to show good performance. This makes it difficult to utilize these networks in clinical medicine. Based on this, we propose a lightweight and efficient pure CNN network for medical image classification (Eff-PCNet). On the one hand, we propose a multi-branch multi-scale CNN (M2C) module, which divides the feature map into four parallel branches along the channel dimensions by a certain scale factor and carries out a deep convolution operation using different scale convolution kernels, and this multi-branch multi-scale operation effectively replaces the large kernel convolution. This multi-branch multi-scale operation effectively replaces the large kernel convolution. It reduces the computational cost of the module while fusing the feature information between different channels and thus obtains richer feature information. Finally, the four feature maps are then spliced along the channel dimensions to fuse the multi-scale and multi-dimensional feature information. On the other hand, we introduce the structural reparameterization technique and propose the structural reparameterized CNN (Rep-C) module. Specifically, it utilizes multiple linear operators to generate different feature maps during the training process and fuses all the participants into one through parameter fusion to achieve fast inference while providing a more effective solution for feature reuse. A number of experimental results show that our Eff-PCNet performs better than current methods based on CNN, Transformer, and MLP in the classification of three publicly available medical image datasets. Among them, we achieve 87.4% Acc on the HAM10000 dataset, 91.06% Acc on the SkinCancer dataset, and 97.03% Acc on the Chest-Xray dataset. Meanwhile, our approach achieves a better trade-off between the number of parameters; computation; and other performance metrics as well.

Robust Medical X-Ray Image Classification by Deep Learning with Multi-Versus Optimizer

Classification of medical images plays an indispensable role in medical treatment and training tasks. Much effort and time are required in the extraction and selection of classification features of medical images. Deep Neural Networks (DNNs) are an evolving Machine Learning (ML) method that has proved its ability in various classification tasks. Convolutional Neural Networks (CNNs) present the optimal results for changing image classification tasks. In this regard, this study focused on developing a Multi-versus Optimizer with Deep Learning Enabled Robust Medical X-ray Image Classification (MVODL-RMXIC) method, aiming to identify abnormalities in medical X-ray images. The MVODL-RMXIC model used the Cross Bilateral Filtering (CBF) technique for noise removal, a MixNet feature extractor with an MVO algorithm based on hyperparameter optimization, and Bidirectional Long-Short-Term Memory (BiLSTM) for image classification. The proposed MVODL-RMXIC model was simulated and evaluated, showing its efficiency over other current methods.

Deep-learning based classification of a tumor marker for prognosis on Hodgkin's disease.

Hodgkin's disease is a common malignant disorder in adolescent patients. Although most patients are cured, approximately 10%-15% of patients experience a relapse or have resistant disease. Furthermore, there are no definitive molecular predictors for early identification of patients at high risk of treatment failure to first line therapy. The aim of this study was to evaluate the deep learning-based classifier model of medical image classification to predict clinical outcome that may help in appropriate therapeutic decisions. Eighty-three FFPE biopsy specimens from patients with Hodgkin's disease were stratified according to the patient's qPET scores, stained with picrosirius red dye and digitalized by whole slide image scanning. The resulting whole slide images were cut into tiles and annotated by two classes based on the collagen fibers' degree of coloring with picrosirius red. The neural network (YOLOv4) was then trained with the annotated data. Training was performed with 30 cases. Prognostic power of the weakly stained picrosirius red fibers was evaluated with 53 cases. The same neural network was trained with MMP9 stained tissue slides from the same cases and the quantification results were compared with the variant from the picrosirius red cases. There was a weak monotonically increasing relationship by parametric ANOVA between the qPET groups and the percentages of weakly stained fibers (p = .0185). The qPET-positive cases showed an average of 18% of weakly stained fibers, and the qPET-negative cases 10%-14%. Detection performance showed an AUC of 0.79. Picrosirius red shows distinct associations as a prognostic metric candidate of disease progression in Hodgkin's disease cases using whole slide images but not sufficiently as a prognostic device.

Deep Transfer Learning with Enhanced Feature Fusion for Detection of Abnormalities in X-ray Images.

Medical image classification poses significant challenges in real-world scenarios. One major obstacle is the scarcity of labelled training data, which hampers the performance of image-classification algorithms and generalisation. Gathering sufficient labelled data is often difficult and time-consuming in the medical domain, but deep learning (DL) has shown remarkable performance, although it typically requires a large amount of labelled data to achieve optimal results. Transfer learning (TL) has played a pivotal role in reducing the time, cost, and need for a large number of labelled images. This paper presents a novel TL approach that aims to overcome the limitations and disadvantages of TL that are characteristic of an ImageNet dataset, which belongs to a different domain. Our proposed TL approach involves training DL models on numerous medical images that are similar to the target dataset. These models were then fine-tuned using a small set of annotated medical images to leverage the knowledge gained from the pre-training phase. We specifically focused on medical X-ray imaging scenarios that involve the humerus and wrist from the musculoskeletal radiographs (MURA) dataset. Both of these tasks face significant challenges regarding accurate classification. The models trained with the proposed TL were used to extract features and were subsequently fused to train several machine learning (ML) classifiers. We combined these diverse features to represent various relevant characteristics in a comprehensive way. Through extensive evaluation, our proposed TL and feature-fusion approach using ML classifiers achieved remarkable results. For the classification of the humerus, we achieved an accuracy of 87.85%, an F1-score of 87.63%, and a Cohen's Kappa coefficient of 75.69%. For wrist classification, our approach achieved an accuracy of 85.58%, an F1-score of 82.70%, and a Cohen's Kappa coefficient of 70.46%. The results demonstrated that the models trained using our proposed TL approach outperformed those trained with ImageNet TL. We employed visualisation techniques to further validate these findings, including a gradient-based class activation heat map (Grad-CAM) and locally interpretable model-independent explanations (LIME). These visualisation tools provided additional evidence to support the superior accuracy of models trained with our proposed TL approach compared to those trained with ImageNet TL. Furthermore, our proposed TL approach exhibited greater robustness in various experiments compared to ImageNet TL. Importantly, the proposed TL approach and the feature-fusion technique are not limited to specific tasks. They can be applied to various medical image applications, thus extending their utility and potential impact. To demonstrate the concept of reusability, a computed tomography (CT) case was adopted. The results obtained from the proposed method showed improvements.

Hierarchical Bias Mitigation for Semi-Supervised Medical Image Classification

Semi-supervised learning (SSL) has demonstrated remarkable advances on medical image classification, by harvesting beneficial knowledge from abundant unlabeled samples. The pseudo labeling dominates current SSL approaches, however, it suffers from intrinsic biases within the process. In this paper, we retrospect the pseudo labeling and identify three hierarchical biases: perception bias, selection bias and confirmation bias, at feature extraction, pseudo label selection and momentum optimization stages, respectively. In this regard, we propose a HierArchical BIas miTigation (HABIT) framework to amend these biases, which consists of three customized modules including Mutual Reconciliation Network (MRNet), Recalibrated Feature Compensation (RFC) and Consistency-aware Momentum Heredity (CMH). Firstly, in the feature extraction, MRNet is devised to jointly utilize convolution and permutator-based paths with a mutual information transfer module to exchanges features and reconcile spatial perception bias for better representations. To address pseudo label selection bias, RFC adaptively recalibrates the strong and weak augmented distributions to be a rational discrepancy and augments features for minority categories to achieve the balanced training. Finally, in the momentum optimization stage, in order to reduce the confirmation bias, CMH models the consistency among different sample augmentations into network updating process to improve the dependability of the model. Extensive experiments on three semi-supervised medical image classification datasets demonstrate that HABIT mitigates three biases and achieves state-of-the-art performance. Our codes are available at https://github.com/ CityU-AIM-Group/HABIT.

Classification of Medical Image Notes for Image Labeling by Using MinBERT

The lack of labeled image data poses a serious challenge to the application of artificial intelligence (AI) in medical image diagnosis. Medical image notes contain valuable patient information that could be used to label images for machine learning tasks. However, most image note texts are unstructured with heterogeneity and short-paragraph characters, which fail traditional keyword-based techniques. We utilized a deep learning approach to recover missing labels for medical image notes automatically by using a combination of deep word embedding and deep neural network classifiers. Bidirectional encoder representations from transformers trained on medical image notes corpus (MinBERT) were proposed. We applied the proposed techniques to two typical classification tasks: Medical image type identification and clinical diagnosis identification. The two methods significantly outperformed baseline methods and presented high accuracies of 99.56% and 99.72% in image type identification and of 94.56% and 92.45% in clinical diagnosis identification. Visualization analysis further indicated that word embedding could efficiently capture semantic similarities and regularities across diverse expressions. Results indicated that our proposed framework could accurately recover the missing label information of medical images through the automatic extraction of electronic medical record information. Hence, it could serve as a powerful tool for exploring useful training data in various medical AI applications.

A graph neural network model for the diagnosis of lung adenocarcinoma based on multimodal features and an edge-generation network.

Lung cancer is a global disease with high lethality, with early screening being considerably helpful for improving the 5-year survival rate. Multimodality features in early screening imaging are an important part of the prediction for lung adenocarcinoma, and establishing a model for adenocarcinoma diagnosis based on multimodal features is an obvious clinical need. Through our practice and investigation, we found that graph neural networks (GNNs) are excellent platforms for multimodal feature fusion, and the data can be completed using the edge-generation network. Therefore, we propose a new lung adenocarcinoma multiclassification model based on multimodal features and an edge-generation network. According to a ratio of 80% to 20%, respectively, the dataset of 338 cases was divided into the training set and the test set through 5-fold cross-validation, and the distribution of the 2 sets was the same. First, the regions of interest (ROIs) cropped from computed tomography (CT) images were separately fed into convolutional neural networks (CNNs) and radiomics processing platforms. The results of the 2 parts were then input into a graph embedding representation network to obtain the fused feature vectors. Subsequently, a graph database based on the clinical and semantic features was established, and the data were supplemented by an edge-generation network, with the fused feature vectors being used as the input of the nodes. This enabled us to clearly understand where the information transmission of the GNN takes place and improves the interpretability of the model. Finally, the nodes were classified using GNNs. On our dataset, the proposed method presented in this paper achieved superior results compared to traditional methods and showed some comparability with state-of-the-art methods for lung nodule classification. The results of our method are as follows: accuracy (ACC) =66.26% (±4.46%), area under the curve (AUC) =75.86% (±1.79%), F1-score =64.00% (±3.65%), and Matthews correlation coefficient (MCC) =48.40% (±5.07%). The model with the edge-generating network consistently outperformed the model without it in all aspects. The experiments demonstrate that with appropriate data=construction methods GNNs can outperform traditional image processing methods in the field of CT-based medical image classification. Additionally, our model has higher interpretability, as it employs subjective clinical and semantic features as the data construction approach. This will help doctors better leverage human-computer interactions.

DPAM-PSPNet: ultrasonic image segmentation of thyroid nodule based on dual-path attention mechanism

Objective. Deep learning has demonstrated its versatility in the medical field, particularly in medical image segmentation, image classification, and other forms of automated diagnostics. The clinical diagnosis of thyroid nodules requires radiologists to locate nodules, diagnose conditions based on nodule boundaries, textures and their experience. This task is labor-intensive and tiring; therefore, an automated system for accurate thyroid nodule segmentation is essential. In this study, a model named DPAM-PSPNet was proposed, which automatically segments nodules in thyroid ultrasound images and enables to segment malignant nodules precisely. Approach. In this paper, accurate segmentation of nodule edges is achieved by introducing the dual path attention mechanism (DPAM) in PSPNet. In one channel, it captures global information with a lightweight cross-channel interaction mechanism. In other channel, it focus on nodal margins and surrounding information through the residual bridge network. We also updated the integrated loss function to accommodate the DPAM-PSPNet. Main results. The DPAM-PSPNet was tested against the classical segmentation model. Ablation experiments were designed for the two-path attention mechanism and the new loss function, and generalization experiments were designed on the public dataset. Our experimental results demonstrate that DPAM-PSPNet outperforms other existing methods in various evaluation metrics. In the model comparison experiments, it achieved performance with an mIOU of 0.8675, mPA of 0.9357, mPrecision of 0.9202, and Dice coefficient of 0.9213. Significance. The DPAM-PSPNet model can segment thyroid nodules in ultrasound images with little training data and generate accurate boundary regions for these nodules.

A classification algorithm based on improved meta learning and transfer learning for few‐shot medical images

AbstractAt present, medical image classification algorithm plays an important role in clinical diagnosis. However, due to the scarcity of data labels, small sample size, uneven distribution, and poor domain generalization, many algorithms still have limitations. Therefore, a deep learning training network for disease classification and recognition of multimodal few‐shot medical images are proposed, trying to solve the above problems and limitations. The network is based on the idea of meta‐learning for training. Specifically, the technology of transfer learning and few‐shot learning are used. In the process of building and improving the network structure, the multi‐source domain generalization method, which performs well in the field of person re‐identification, is combined. Finally, the applicability and effectiveness of the model are verified by using Grad‐CAM tool. The experiments show that the accuracy of classification and recognition of the model is better than the advanced model in this field. The concerned areas of model classification are similar or the same as the manually labelled areas. It is of far‐reaching significance to improve the efficiency of future clinical auxiliary diagnosis and patient diversion, as well as to promote the development of the Wise Information Technology of Med in the future.

Deep Convolutional Generative Adversarial Networks for Imbalance Medical Image Classification.

Medical image classification is an essential task in clinical practice and research. It enables medical professionals to be assisted in diagnosing medical conditions accurately and efficiently, leading to improved patient outcomes and survival rates. However, traditional manual interpretation methods for diagnosing medical images have some drawbacks. Firstly, imbalanced medical images often exhibit a significant disparity in the number of samples across different classes, posing challenges in training accurate and robust models that can effectively learn from limited data in the minority class while avoiding biases towards the majority class. Secondly, the limited availability of labelled data will put a further load on the healthcare system, as labelling medical images is a time-consuming and resource-intensive task, often requiring expert knowledge. This paper proposed a generative adversarial network (GAN) with the purpose of improving the limitations associated with the imbalanced distribution of medical images. Based on the experiments conducted, it shows that the proposed model exhibits a high level of accuracy for two-class labelled dataset, with a low performance for the skin cancer dataset due to number of the labelled dataset is more than two

Self-Supervision for Medical Image Classification: State-of-the-Art Performance with ~100 Labeled Training Samples per Class.

Is self-supervised deep learning (DL) for medical image analysis already a serious alternative to the de facto standard of end-to-end trained supervised DL? We tackle this question for medical image classification, with a particular focus on one of the currently most limiting factor of the field: the (non-)availability of labeled data. Based on three common medical imaging modalities (bone marrow microscopy, gastrointestinal endoscopy, dermoscopy) and publicly available data sets, we analyze the performance of self-supervised DL within the self-distillation with no labels (DINO) framework. After learning an image representation without use of image labels, conventional machine learning classifiers are applied. The classifiers are fit using a systematically varied number of labeled data (1-1000 samples per class). Exploiting the learned image representation, we achieve state-of-the-art classification performance for all three imaging modalities and data sets with only a fraction of between 1% and 10% of the available labeled data and about 100 labeled samples per class.

BIDL: a brain-inspired deep learning framework for spatiotemporal processing.

Brain-inspired deep spiking neural network (DSNN) which emulates the function of the biological brain provides an effective approach for event-stream spatiotemporal perception (STP), especially for dynamic vision sensor (DVS) signals. However, there is a lack of generalized learning frameworks that can handle various spatiotemporal modalities beyond event-stream, such as video clips and 3D imaging data. To provide a unified design flow for generalized spatiotemporal processing (STP) and to investigate the capability of lightweight STP processing via brain-inspired neural dynamics, this study introduces a training platform called brain-inspired deep learning (BIDL). This framework constructs deep neural networks, which leverage neural dynamics for processing temporal information and ensures high-accuracy spatial processing via artificial neural network layers. We conducted experiments involving various types of data, including video information processing, DVS information processing, 3D medical imaging classification, and natural language processing. These experiments demonstrate the efficiency of the proposed method. Moreover, as a research framework for researchers in the fields of neuroscience and machine learning, BIDL facilitates the exploration of different neural models and enables global-local co-learning. For easily fitting to neuromorphic chips and GPUs, the framework incorporates several optimizations, including iteration representation, state-aware computational graph, and built-in neural functions. This study presents a user-friendly and efficient DSNN builder for lightweight STP applications and has the potential to drive future advancements in bio-inspired research.