Backbone Architecture Research Articles

Underwater Marine Life Study using Yolo V8

Abstract: Underwater target detection, object classification, and image segmentation play an important role in ocean exploration and marine life studies for which the improvement of relevant technology is of much practical significance. Although existing target detection, image classification, and segmentation algorithms have achieved excellent performance on land they often fail to achieve satisfactory outcomes of detection and classification when in the underwater environment. In this paper, one of the most advanced target detection algorithms, YOLO v8 (You Only Look Once), was first applied in the underwater environment before being improved by combining it with some methods characteristic of the underwater environment. The state-of-the-art backbone and neck architectures and TC-YOLO/SAM were treated as the basic backbone network of YOLO v8, which makes the network suitable for underwater images.

SEMANTIC ENRICHMENT OF 3D POINT CLOUDS USING 2D IMAGE SEGMENTATION

Abstract. 3D point cloud segmentation is computationally intensive due to the lack of inherent structural information and the unstructured nature of the point cloud data, which hinders the identification and connection of neighboring points. Understanding the structure of the point cloud data plays a crucial role in obtaining a meaningful and accurate representation of the underlying 3D environment. In this paper, we propose an algorithm that builds on existing state-of-the-art techniques of 2D image segmentation and point cloud registration to enrich point clouds with semantic information. DeepLab2 with ResNet50 as backbone architecture trained on the COCO dataset is used for indoor scene semantic segmentation into several classes like wall, floor, ceiling, doors, and windows. Semantic information from 2D images is propagated along with other input data, i.e., RGB images, depth images, and sensor information to generate 3D point clouds with semantic information. Iterative Closest Point (ICP) algorithm is used for the pair-wise registration of consecutive point clouds and finally, optimization is applied using the pose graph optimization on the whole set of point clouds to generate the combined point cloud of the whole scene. 3D point cloud of the whole scene contains pseudo-color information which denotes the semantic class to which each point belongs. The proposed methodology use an off-the-shelf 2D semantic segmentation deep learning model to semantically segment 3D point clouds collected using handheld mobile LiDAR sensor. We demonstrate a comparison of the accuracy achieved compared to a manually segmented point cloud on an in-house dataset as well as a 2D3DS benchmark dataset.

LibFewShot: A Comprehensive Library for Few-Shot Learning.

Few-shot learning, especially few-shot image classification, has received increasing attention and witnessed significant advances in recent years. Some recent studies implicitly show that many generic techniques or "tricks", such as data augmentation, pre-training, knowledge distillation, and self-supervision, may greatly boost the performance of a few-shot learning method. Moreover, different works may employ different software platforms, backbone architectures and input image sizes, making fair comparisons difficult and practitioners struggle with reproducibility. To address these situations, we propose a comprehensive library for few-shot learning (LibFewShot) by re-implementing eighteen state-of-the-art few-shot learning methods in a unified framework with the same single codebase in PyTorch. Furthermore, based on LibFewShot, we provide comprehensive evaluations on multiple benchmarks with various backbone architectures to evaluate common pitfalls and effects of different training tricks. In addition, with respect to the recent doubts on the necessity of meta- or episodic-training mechanism, our evaluation results confirm that such a mechanism is still necessary especially when combined with pre-training. We hope our work can not only lower the barriers for beginners to enter the area of few-shot learning but also elucidate the effects of nontrivial tricks to facilitate intrinsic research on few-shot learning. The source code is available from https://github.com/RL-VIG/LibFewShot.

Anomaly detection via gating highway connection for retinal fundus images

Since the labels for medical images are challenging to collect in real scenarios, especially for rare diseases, fully supervised methods cannot achieve robust performance for clinical anomaly detection. Recent research tried to tackle this problem by training the anomaly detection framework using only normal data. Reconstruction-based methods, e.g., auto-encoder, achieved impressive performances in the anomaly detection task. However, most existing methods adopted the straightforward backbone architecture (i.e., encoder-and-decoder) for image reconstruction. The design of a skip connection, which can directly transfer information between the encoder and decoder, is rarely used. Since the existing U-Net has demonstrated the effectiveness of skip connections for image reconstruction tasks, in this paper, we first use the dynamic gating strategy to achieve the usage of skip connections in existing reconstruction-based anomaly detection methods and then propose a novel gating highway connection module to adaptively integrate skip connections into the framework and boost its anomaly detection performance, namely GatingAno. Furthermore, we formulate an auxiliary task, namely histograms of oriented gradients (HOG) prediction, to encourage the framework to exploit contextual information from fundus images in a self-driven manner, which increases the robustness of feature representation extracted from the healthy samples. Last but not least, to improve the model generalization for anomalous data, we introduce an adversarial strategy for the training of our multi-task framework. Experimental results on the publicly available datasets, i.e., IDRiD and ADAM, validate the superiority of our method for detecting abnormalities in retinal fundus images. The source code is available at https://github.com/WentianZhang-ML/GatingAno.

Mobile Implementation of Retinal Image Analysis for Efficient Vessel, Optic Disc, and Lesion Detection

Smartphone-based mobile fundus photography is gaining popularity due to the rise of handheld fundus lenses, allowing a portable solution for a mobile-based computer-assisted diagnostic system (CADS). With such a system, professionals can monitor and diagnose numerous retinal diseases, including diabetic retinopathy (DR), glaucoma, age-related macular degeneration, etc. on their smartphone devices. In this study, we proposed a unified CADS tool for smartphone devices that can detect and identify six crucial retinal features utilizing both a filtering approach and a deep learning (DL) approach. These features are retinal blood vessels (RBV), optic discs (OD), hemorrhages (HM), microaneurysm (MA), hard exudates (HE), and soft exudates (SE). Traditional filtering is applied for RBV segmentation using B-COSFIRE and Frangi filter, whereas vessel inpainting and automatic canny edge-based Hough transform are used to localize OD center and radius. The DR lesions (HM, MA, HE, OD segmentation, and SE) are detected using a transfer learning-based Resnet50 backbone and multiclass DL U-net architecture. RBV segmentation achieved 94.94% and 94.44% accuracy in the DRIVE and STARE datasets. OD localization achieved an accuracy of 99.60% in the MESSIDOR dataset. Lastly, the IDRiD dataset is used to train and validate the DR lesions with an overall accuracy of 99.7%, F1-score of 77.4, and mean IoU of 59.2. The smartphone application can perform all the segmentation tasks at once in an average of 30 seconds. Given the availability, it is possible to improve the accuracy of the DL method further by training with more mobile fundus datasets.

Utilizing deep learning via the 3D U-net neural network for the delineation of brain stroke lesions in MRI image

The segmentation of acute stroke lesions plays a vital role in healthcare by assisting doctors in making prompt and well-informed treatment choices. Although Magnetic Resonance Imaging (MRI) is a time-intensive procedure, it produces high-fidelity images widely regarded as the most reliable diagnostic tool available. Employing deep learning techniques for automated stroke lesion segmentation can offer valuable insights into the precise location and extent of affected tissue, enabling medical professionals to effectively evaluate treatment risks and make informed assessments. In this research, a deep learning approach is introduced for segmenting acute and sub-acute stroke lesions from MRI images. To enhance feature learning through brain hemisphere symmetry, pre-processing techniques are applied to the data. To tackle the class imbalance challenge, we employed a strategy of using small patches with balanced sampling during training, along with a dynamically weighted loss function that incorporates f1-score and IOU-score (Intersection over Union). Furthermore, the 3D U-Net architecture is used to generate predictions for complete patches, employing a high degree of overlap between patches to minimize the requirement for subsequent post-processing steps. The 3D U-Net model, utilizing ResnetV2 as the pre-trained encoder for IOU-score and Seresnext101 for f1-score, stands as the leading state-of-the-art (SOTA) model for segmentation tasks. However, recent research has introduced a novel model that surpasses these metrics and demonstrates superior performance compared to other backbone architectures. The f1-score and IOU-score were computed for various backbones, with Seresnext101 achieving the highest f1-score and ResnetV2 performing the highest IOU-score. These calculations were conducted using a threshold value of 0.5. This research proposes a valuable model based on transfer learning for the classification of brain diseases in MRI scans. The achieved f1-score using the recommended classifiers demonstrates the effectiveness of the approach employed in this study. The findings indicate that Seresnext101 attains the highest f1-score of 0.94226, while ResnetV2 achieves the best IOU-score of 0.88342, making it the preferred architecture for segmentation methods. Furthermore, the study presents experimental results of the 3D U-Net model applied to brain stroke lesion segmentation, suggesting prospects for researchers interested in segmenting brain strokes and enhancing 3D U-Net models.

EDKSANet: An Efficient Dual-Kernel Split Attention Neural Network for the Classification of Tibetan Medicinal Materials

Tibetan medicine has received wide acclaim for its unique diagnosis and treatment methods. The identification of Tibetan medicinal materials, which are a vital component of Tibetan medicine, is a key research area in this field. However, traditional deep learning-based visual neural networks face significant challenges in efficiently and accurately identifying Tibetan medicinal materials due to their large number, complex morphology, and the scarcity of public visual datasets. To address this issue, we constructed a computer vision dataset with 300 Tibetan medicinal materials and proposed a lightweight and efficient cross-dimensional attention mechanism, the Dual-Kernel Split Attention (DKSA) module, which can adaptively share parameters of the kernel in both spatial and channel dimensions. Based on the DKSA module, we achieve efficient unification of convolution and self-attention under the CNN architecture and develop a new lightweight backbone architecture, EDKSANet, to provide enhanced performance for various computer vision tasks. As compared to RedNet, the top-1 accuracy is improved by 1.2% on an ImageNet dataset, and a larger margin of +1.5 box AP for object detection and an improvement of +1.3 mask AP for instance segmentation on MS-COCO dataset are obtained. Moreover, EDKSANet achieved excellent classification performance on the Tibetan medicinal materials dataset, with an accuracy of up to 96.85%.

An automatic network structure search via channel pruning for accelerating human activity inference on mobile devices

During recent years, deep convolutional neural networks (CNNs) have become increasingly popular in sensor-based human activity recognition (HAR), due to powerful feature extraction capacity. However, the required high computational cost has hindered their practical deployment in real-world applications. To address this issue, this paper presents a novel approach named ABCSearch to prune channels through an artificial bee colony (ABC) algorithm, which is a meta-heuristic optimization algorithm inspired by the foraging behavior of honeybees. Unlike traditional pruning strategies that prioritize channel importance, the proposed method seeks to identify the optimal number of channels at each layer, i.e., the optimal pruning structure. To lessen human interference, we transform the search problem of the optimal pruning structure into a simple optimization problem and solve it automatically through the ABC algorithm, which can significantly decrease both model size and computational cost without compromising performance. Specifically, we validate the effectiveness of ABCSearch for a standard five-layer CNN backbone architecture, which achieves 97.05%, 98.45%, 91.71%, 78.89% classification accuracies on four HAR benchmarks, i.e., UCI-HAR, WISDM, PAMAP2, UniMiB-SHAR. Our approach can yield 87.44%, 89.41%, 89.06%, 89.59% FLOPs reductions, and 89.54%, 88.70%, 88.38%, 90.45% less memory consumptions, as well as 4.4×, 5.0×, 5.8×, 6.1× practical speedups on real hardware devices, while maintaining comparable accuracy.

Analysis of printed document identification based on Deep Learning

In this study, we investigate the effectiveness of ResNet, a deep neural network architecture, for a deep learning approach to address the problem of printed document identification. ResNet is known for its ability to handle the vanishing gradient problem and learn highly representative features. Multiple variations of ResNet have been applied, including ResNet50, ResNet101, and ResNet152, which provide the backbone architecture of our classification model and are trained on a comprehensive dataset of microscopic printed images containing some microscopic printing patterns from various source printers. We also incorporate Mix-up augmentation, a technique that generates virtual training samples by interpolating pairs of images and labels, to further enhance the performance and generalization capability of the model. The experimental results showed that ResNet101 and ResNet152 variants outperformed in accurately distinguishing printer sources based on microscopic printed patterns. We developed a mobile app to test the feasibility of our findings in practice. In conclusion, this study aims to lay the groundwork for creating a sufficiently pre-trained model with accurate performance of identification that can be deployed on mobile devices to detect the printed sources of documents.

On the Importance of Attention and Augmentations for Hypothesis Transfer in Domain Adaptation and Generalization.

Unsupervised domain adaptation (UDA) aims to mitigate the performance drop due to the distribution shift between the training and testing datasets. UDA methods have achieved performance gains for models trained on a source domain with labeled data to a target domain with only unlabeled data. The standard feature extraction method in domain adaptation has been convolutional neural networks (CNNs). Recently, attention-based transformer models have emerged as effective alternatives for computer vision tasks. In this paper, we benchmark three attention-based architectures, specifically vision transformer (ViT), shifted window transformer (SWIN), and dual attention vision transformer (DAViT), against convolutional architectures ResNet, HRNet and attention-based ConvNext, to assess the performance of different backbones for domain generalization and adaptation. We incorporate these backbone architectures as feature extractors in the source hypothesis transfer (SHOT) framework for UDA. SHOT leverages the knowledge learned in the source domain to align the image features of unlabeled target data in the absence of source domain data, using self-supervised deep feature clustering and self-training. We analyze the generalization and adaptation performance of these models on standard UDA datasets and aerial UDA datasets. In addition, we modernize the training procedure commonly seen in UDA tasks by adding image augmentation techniques to help models generate richer features. Our results show that ConvNext and SWIN offer the best performance, indicating that the attention mechanism is very beneficial for domain generalization and adaptation with both transformer and convolutional architectures. Our ablation study shows that our modernized training recipe, within the SHOT framework, significantly boosts performance on aerial datasets.

Automated classification of “cluttered” construction housekeeping images through supervised and self-supervised feature representation learning

Construction housekeeping is crucial for safety, but frequent manual inspections are difficult to maintain. A computer vision approach to automatically monitor housekeeping can overcome these issues. However, it requires labelling large number of “cluttered” construction housekeeping images that are difficult to label, even by experts. Thus, this paper presents an alternative approach that evaluates the use of self-supervised learning feature extraction to classify “cluttered” construction housekeeping images. The most suitable (84% accuracy) backbone architecture for supervised classification of housekeeping images was found to be Swin-transformer. In addition, the experiments show that self-supervised learning approach can perform better (1–4% improvement in prediction accuracy, precision, and recall) than the supervised learning approach in a non-transfer learning context and when the number of training images is reduced.

Pseudo labelling workflow, margin losses, hard triplet mining, and PENViT backbone for explainable age and biological gender estimation using dental panoramic radiographs

This study aimed to estimate human age and gender from panoramic radiographs using various deep learning techniques while using explainability to have a novel hybrid unsupervised model explain the decision-making process. The classification task involved training neural networks and vision transformers on 706 panoramic radiographs using different loss functions and backbone architectures namely ArcFace, a triplet network named TriplePENViT, and the subsequently developed model called PENViT. Pseudo labeling techniques were applied to train the models using unlabeled data. FullGrad Explainable AI was used to gain insights into the decision-making process of the developed PENViT model. The ViT Large 32 model achieved a validation accuracy of 68.21% without ArcFace, demonstrating its effectiveness in the classification task. The PENViT model outperformed other backbones, achieving the same validation accuracy without ArcFace and an improved accuracy of 70.54% with ArcFace. The TriplePENViT model achieved a validation accuracy of 67.44% using hard triplet mining techniques. Pseudo labeling techniques yielded poor performance, with a validation accuracy of 64.34%. Validation accuracy without ArcFace was established at 67.44% for Age and 84.49% for gender. The unsupervised model considered developing tooth buds, tooth proximity and mandibular shape for estimating age within deciduous and mixed dentitions. For ages 20–29, it factored permanent dentition, alveolar bone density, root apices, and third molars. Above 30, it notes occlusal deformity resulting from missing dentition and the temporomandibular joint complex as predictors for age estimation from panoramic radiographs.Graphical abstract

Graph Neural Networks for source localization using Ships-of-Opportunity spectrograms

Traditional underwater source localization methods have primarily relied on optimization techniques, matched-field processing, beamforming, and, more recently, deep learning approaches. However, these methods often fail to exploit the spatial correlation of data due to their representation in a regular domain. Nowadays, data collection commonly occurs in complex domains, like sensor networks, where signals and features are best represented as graphs based on feature similarity metrics. In a graph representation setting, each sensor or feature corresponds to a node in the graph, accompanied by a feature vector that may or may not vary with time. As edges define spatial relationships, spatio-temporal information is considered simultaneously during the learning process. This work proposes a novel graph learning module for source localization using Ships-of-Opportunity (SOO) spectrograms, which represent mid-frequency acoustic broadband signals (360-1100Hz) collected during the 2017 Seabed Characterization Experiment (SBCEX17). The proposed approach employs a semi-supervisedlearning algorithm on a graph constructed through a k-nearest neighbor (k-NN) algorithm, incorporating features extracted from the spectrograms using a backbone architecture. The efficacy of the proposed approach is demonstrated through model evaluation on both synthetic and measured data, validating the generalization power of the architecture. [Work supported by ONR under Grant No. N00014-21-1-2760.]

ADNet++: A few-shot learning framework for multi-class medical image volume segmentation with uncertainty-guided feature refinement.

A major barrier to applying deep segmentation models in the medical domain is their typical data-hungry nature, requiring experts to collect and label large amounts of data for training. As a reaction, prototypical few-shot segmentation (FSS) models have recently gained traction as data-efficient alternatives. Nevertheless, despite the recent progress of these models, they still have some essential shortcomings that must be addressed. In this work, we focus on three of these shortcomings: (i) the lack of uncertainty estimation, (ii) the lack of a guiding mechanism to help locate edges and encourage spatial consistency in the segmentation maps, and (iii) the models' inability to do one-step multi-class segmentation. Without modifying or requiring a specific backbone architecture, we propose a modified prototype extraction module that facilitates the computation of uncertainty maps in prototypical FSS models, and show that the resulting maps are useful indicators of the model uncertainty. To improve the segmentation around boundaries and to encourage spatial consistency, we propose a novel feature refinement module that leverages structural information in the input space to help guide the segmentation in the feature space. Furthermore, we demonstrate how uncertainty maps can be used to automatically guide this feature refinement. Finally, to avoid ambiguous voxel predictions that occur when images are segmented class-by-class, we propose a procedure to perform one-step multi-class FSS. The efficiency of our proposed methodology is evaluated on two representative datasets for abdominal organ segmentation (CHAOS dataset and BTCV dataset) and one dataset for cardiac segmentation (MS-CMRSeg dataset). The results show that our proposed methodology significantly (one-sided Wilcoxon signed rank test, p<0.05) improves the baseline, increasing the overall dice score with +5.2, +5.1, and +2.8 percentage points for the CHAOS dataset, the BTCV dataset, and the MS-CMRSeg dataset, respectively.

Coping with change: Learning invariant and minimum sufficient representations for fine-grained visual categorization

Fine-grained visual categorization (FGVC) is a challenging task due to similar visual appearances between various species. Previous studies always implicitly assume that the training and test data have the same underlying distributions, and that features extracted by modern backbone architectures remain discriminative and generalize well to unseen test data. However, we empirically justify that these conditions are not always true on benchmark datasets. To this end, we combine the merits of invariant risk minimization (IRM) and information bottleneck (IB) principle to learn invariant and minimum sufficient (IMS) representations for FGVC, such that the overall model can always discover the most succinct and consistent fine-grained features. We apply the matrix-based Rényi’s α-order entropy to simplify and stabilize the training of IB; we also design a “soft” environment partition scheme to make IRM applicable to FGVC task. To the best of our knowledge, we are the first to address the problem of FGVC from a generalization perspective and develop a new information-theoretic solution accordingly. Extensive experiments demonstrate the consistent performance gain offered by our IMS. Code is available at: https://github.com/SYe-hub/IMS.

CMNet: a novel model and design rationale based on comparison studies and synergy of CNN and MetaFormer

Convolutional- and Transformer-based backbone architecture are two dominant, widely accepted, models in computer vision. Nevertheless, it is still a challenge, thus a focus of research, to decide which backbone architecture performs better, and under which circumstances. In this paper, we conduct an in-depth investigation into the differences of the macroscopic backbone design of the CNN and Transformer models with the ultimate purpose of developing new models to combine the strengths of both types of architectures for effective image classification. Specifically, we first analyze the model structures of both models and identified four main differences, then we design four sets of ablation experiments using the ImageNet-1K dataset with an image classification problem as an example to study the impacts of these four differences on model performance. Based on the experimental results, we derive four observations as rules of thumb for designing a vision model backbone architecture. Informed by the experiment findings, we then conceive a novel model called CMNet which marries the experiment-proved best design practices of CNN and Transformer architectures. Finally, we carry out extensive experiments on CMNet using the same dataset against baseline classifiers. Initial results prove CMNet achieves the highest top-1 accuracy of 80.08% on the ImageNet-1K validation set, this is a very competitive value compared to previous classical models with similar computational complexity. Details of the implementation, algorithms and codes, are publicly available on Github: https://github.com/Arwin-Yu/CMNet.

Exploring the impact of backbone architecture on the thermal decomposition of silicon-containing arylacetylene resins

Silicon-containing arylacetylene (PSA) resins are promising heat-resistant polymers. However, little is known about their decomposition mechanisms. A molecular dynamics (MD) study based on reactive force field (ReaxFF) was employed to investigate the thermal decomposition mechanism of two representative PSA resins containing benzene or naphthalene in the backbone. The effect of heating rate, temperature, and crosslinking degree on decomposition behavior was examined, and the temperatures of three decomposition stages were disclosed. The predicted 5% decomposition temperatures (Td5) and decomposition products are consistent with experimental findings. The ReaxFF MD simulation reveals that the Td5 is dominated by the secondary decomposition and final decomposition, in which the high molecular weight and the stable CC bond of naphthyl play a dominant role in determining high-temperature resistance of the naphthalene-containing PSA resin. We also disclose the decomposition pathways of PSA-type resin. The results gained from the present work can be a helpful guideline for designing resins with excellent heat resistance.

Brake light detection of vehicles using differential evolution based neural architecture search

Recognition of brake light status in vehicles is crucial for anticipating speed changes and preventing rear-end collisions for autonomous driving systems. Existing literature presents two types of methods for brake light detection: hand-crafted feature-based methods and deep learning-based methods. However, hand-crafted methods often struggle to capture brake light characteristics accurately in real-life conditions. In contrast, deep learning-based systems can adapt to diverse brake light variations across different vehicle types and environments. Nevertheless, manually designing Deep Neural Network (DNN) models requires expertise and is prone to errors. To address this limitation, we propose a novel approach that leverages Neural Architecture Search (NAS) to automatically generate optimal DNN architectures for object detection tasks, specifically for brake light detection. In contrast to the existing NAS approaches that focus on classification models, our technique explores NAS for object detection tasks. We employ a modified Differential Evolution algorithm, incorporating evaluation correction-based selection for mutation and species protection-based selection to identify the optimal DNN backbone architecture with optimal training parameters. The proposed approach achieved mean accuracy of 89.73 %, and 88.90 % on four-wheeler datasets CaltechGraz and UC Merced Vehicle Rear Signal datasets, respectively, and it has achieved 97.97 % on the proposed two-wheeler NITW-MBS dataset. The proposed approach’s generalization capability and practical applicability are ascertained through cross-dataset evaluation and experiments on real-world traffic video.

Dermo-Seg: ResNet-UNet Architecture and Hybrid Loss Function for Detection of Differential Patterns to Diagnose Pigmented Skin Lesions.

Convolutional neural network (CNN) models have been extensively applied to skin lesions segmentation due to their information discrimination capabilities. However, CNNs' struggle to capture the connection between long-range contexts when extracting deep semantic features from lesion images, resulting in a semantic gap that causes segmentation distortion in skin lesions. Therefore, detecting the presence of differential structures such as pigment networks, globules, streaks, negative networks, and milia-like cysts becomes difficult. To resolve these issues, we have proposed an approach based on semantic-based segmentation (Dermo-Seg) to detect differential structures of lesions using a UNet model with a transfer-learning-based ResNet-50 architecture and a hybrid loss function. The Dermo-Seg model uses ResNet-50 backbone architecture as an encoder in the UNet model. We have applied a combination of focal Tversky loss and IOU loss functions to handle the dataset's highly imbalanced class ratio. The obtained results prove that the intended model performs well compared to the existing models. The dataset was acquired from various sources, such as ISIC18, ISBI17, and HAM10000, to evaluate the Dermo-Seg model. We have dealt with the data imbalance present within each class at the pixel level using our hybrid loss function. The proposed model achieves a mean IOU score of 0.53 for streaks, 0.67 for pigment networks, 0.66 for globules, 0.58 for negative networks, and 0.53 for milia-like-cysts. Overall, the Dermo-Seg model is efficient in detecting different skin lesion structures and achieved 96.4% on the IOU index. Our Dermo-Seg system improves the IOU index compared to the most recent network.

Fully Automated Skull Stripping from Brain Magnetic Resonance Images Using Mask RCNN-Based Deep Learning Neural Networks.

This research comprises experiments with a deep learning framework for fully automating the skull stripping from brain magnetic resonance (MR) images. Conventional techniques for segmentation have progressed to the extent of Convolutional Neural Networks (CNN). We proposed and experimented with a contemporary variant of the deep learning framework based on mask region convolutional neural network (Mask-RCNN) for all anatomical orientations of brain MR images. We trained the system from scratch to build a model for classification, detection, and segmentation. It is validated by images taken from three different datasets: BrainWeb; NAMIC, and a local hospital. We opted for purposive sampling to select 2000 images of T1 modality from data volumes followed by a multi-stage random sampling technique to segregate the dataset into three batches for training (75%), validation (15%), and testing (10%) respectively. We utilized a robust backbone architecture, namely ResNet-101 and Functional Pyramid Network (FPN), to achieve optimal performance with higher accuracy. We subjected the same data to two traditional methods, namely Brain Extraction Tools (BET) and Brain Surface Extraction (BSE), to compare their performance results. Our proposed method had higher mean average precision (mAP) = 93% and content validity index (CVI) = 0.95%, which were better than comparable methods. We contributed by training Mask-RCNN from scratch for generating reusable learning weights known as transfer learning. We contributed to methodological novelty by applying a pragmatic research lens, and used a mixed method triangulation technique to validate results on all anatomical modalities of brain MR images. Our proposed method improved the accuracy and precision of skull stripping by fully automating it and reducing its processing time and operational cost and reliance on technicians. This research study has also provided grounds for extending the work to the scale of explainable artificial intelligence (XAI).