Classification Backbone Research Articles

CEDNet: A cascade encoder–decoder network for dense prediction

The prevailing methods for dense prediction tasks typically utilize a heavy classification backbone to extract multi-scale features and then fuse these features using a lightweight module. However, these methods allocate most computational resources to the classification backbone, which delays the multi-scale feature fusion and potentially leads to inadequate feature fusion. Although some methods perform feature fusion from early stages, they either fail to fully leverage high-level features to guide low-level feature learning or have complex structures, resulting in sub-optimal performance. We propose a streamlined cascade encoder–decoder network, named CEDNet, tailored for dense prediction tasks. All stages in CEDNet share the same encoder–decoder structure and perform multi-scale feature fusion within each decoder, thereby enhancing the effectiveness of multi-scale feature fusion. We explored three well-known encoder–decoder structures: Hourglass, UNet, and FPN, all of which yielded promising results. Experiments on various dense prediction tasks demonstrated the effectiveness of our method.11Code: https://github.com/zhanggang001/CEDNet.

Multi-View Fusion Network-Based Gesture Recognition Using sEMG Data.

sEMG(surface electromyography) signals have been widely used in rehabilitation medicine in the past decades because of their non-invasive, convenient and informative features, especially in human action recognition, which has developed rapidly. However, the research on sparse EMG in multi-view fusion has made less progress compared to high-density EMG signals, and for the problem of how to enrich sparse EMG feature information, a method that can effectively reduce the information loss of feature signals in the channel dimension is needed. In this article, a novel IMSE (Inception-MaxPooling-Squeeze- Excitation) network module is proposed to reduce the loss of feature information during deep learning. Then, multiple feature encoders are constructed to enrich the information of sparse sEMG feature maps based on the multi-core parallel processing method in multi-view fusion networks, while SwT (Swin Transformer) is used as the classification backbone network. By comparing the feature fusion effects of different decision layers of the multi-view fusion network, it is experimentally obtained that the fusion of decision layers can better improve the classification performance of the network. In NinaPro DB1, the proposed network achieves 93.96% average accuracy in gesture action classification with the feature maps obtained in 300ms time window, and the maximum variation range of action recognition rate of individuals is less than 11.2%. The results show that the proposed framework of multi-view learning plays a good role in reducing individuality differences and augmenting channel feature information, which provides a certain reference for non-dense biosignal pattern recognition.

Weakly supervised learning for multi-class medical image segmentation via feature decomposition

Weakly supervised learning with image-level labels, releasing deep learning from highly labor-intensive pixel-wise annotation, has gained great attention for medical image segmentation. However, existing weakly supervised methods are mainly designed for single-class segmentation while leaving multi-class medical image segmentation rarely-explored. Different from natural images, label symbiosis, together with location adjacency, are much more common in medical images, making it more challenging for multi-class segmentation. In this paper, we propose a novel weakly supervised learning method for multi-class medical image segmentation with image-level labels. In terms of the multi-class classification backbone, a multi-level classification network encoding multi-scale features is proposed to produce binary predictions, together with the corresponding CAMs, of each class separately. To address the above issues (i.e., label symbiosis and location adjacency), a feature decomposition module based on semantic affinity is first proposed to learn both class-independent and class-dependent features by maximizing the inter-class feature distance. Through a cross-guidance loss to jointly utilize the above features, label symbiosis is largely alleviated. In terms of location adjacency, a mutually exclusive loss is constructed to minimize the overlap among regions corresponding to different classes. Experimental results on three datasets demonstrate the superior performance of the proposed weakly-supervised framework for both single-class and multi-class medical image segmentation. We believe the analysis in this paper would shed new light on future work for multi-class medical image segmentation. The source code of this paper is publicly available at https://github.com/HustAlexander/MCWSS.

Transductive meta-learning with enhanced feature ensemble for few-shot semantic segmentation

This paper addresses few-shot semantic segmentation and proposes a novel transductive end-to-end method that overcomes three key problems affecting performance. First, we present a novel ensemble of visual features learned from pretrained classification and semantic segmentation networks with the same architecture. Our approach leverages the varying discriminative power of these networks, resulting in rich and diverse visual features that are more informative than a pretrained classification backbone that is not optimized for dense pixel-wise classification tasks used in most state-of-the-art methods. Secondly, the pretrained semantic segmentation network serves as a base class extractor, which effectively mitigates false positives that occur during inference time and are caused by base objects other than the object of interest. Thirdly, a two-step segmentation approach using transductive meta-learning is presented to address the episodes with poor similarity between the support and query images. The proposed transductive meta-learning method addresses the prediction by first learning the relationship between labeled and unlabeled data points with matching support foreground to query features (intra-class similarity) and then applying this knowledge to predict on the unlabeled query image (intra-object similarity), which simultaneously learns propagation and false positive suppression. To evaluate our method, we performed experiments on benchmark datasets, and the results demonstrate significant improvement with minimal trainable parameters of 2.98M. Specifically, using Resnet-101, we achieve state-of-the-art performance for both 1-shot and 5-shot Pascal-5i\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$5^{i}$$\\end{document}, as well as for 1-shot and 5-shot COCO-20i\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$20^{i}$$\\end{document}.

Transfer learning for galaxy feature detection: Finding giant star-forming clumps in low-redshift galaxies using Faster Region-based Convolutional Neural Network

Abstract Giant star-forming clumps (GSFCs) are areas of intensive star-formation that are commonly observed in high-redshift (z ≳ 1) galaxies but their formation and role in galaxy evolution remain unclear. Observations of low-redshift clumpy galaxy analogues are rare but the availability of wide-field galaxy survey data makes the detection of large clumpy galaxy samples much more feasible. Deep Learning (DL), and in particular Convolutional Neural Networks (CNNs), have been successfully applied to image classification tasks in astrophysical data analysis. However, one application of DL that remains relatively unexplored is that of automatically identifying and localizing specific objects or features in astrophysical imaging data. In this paper, we demonstrate the use of DL-based object detection models to localize GSFCs in astrophysical imaging data. We apply the Faster Region-based Convolutional Neural Network object detection framework (FRCNN) to identify GSFCs in low-redshift (z ≲ 0.3) galaxies. Unlike other studies, we train different FRCNN models on observational data that was collected by the Sloan Digital Sky Survey and labelled by volunteers from the citizen science project ‘Galaxy Zoo: Clump Scout’. The FRCNN model relies on a CNN component as a ‘backbone’ feature extractor. We show that CNNs, that have been pre-trained for image classification using astrophysical images, outperform those that have been pre-trained on terrestrial images. In particular, we compare a domain-specific CNN – ‘Zoobot’ – with a generic classification backbone and find that Zoobot achieves higher detection performance. Our final model is capable of producing GSFC detections with a completeness and purity of ≥0.8 while only being trained on ∼5000 galaxy images.

Ships Detection on Aerial Imagery using Transfer Learning and Selective Search

The traffic in the water area such as harbor and sea strait is highly important to be monitored because it helps to minimize unwanted ships accident. As a result, we proposed an automatic detection method to localize ships contained in sattelite image. We examine several deep learning method as the classification backbone, namely MobileNetV2, DenseNet121, VGG16, and ResNet50. Afterwards, we employed the trained model for detecting the ships. To make the detection faster, inspite of using a sliding window, we use selective search to sample the object candidates from the given scene. The experiments was done using Shipsnet dataset and tested on aerial images. We also conducted a cross domain evaluation where the images were taken using Google Earth. The results indicate that MobileNetV2 has the best performance on classification and detection tasks. The MobileNetV2 is also able to detect the ships on cross-domain scenarios.

Deep learning diagnostic performance and visual insights in differentiating benign and malignant thyroid nodules on ultrasound images.

This study aims to construct and evaluate a deep learning model, utilizing ultrasound images, to accurately differentiate benign and malignant thyroid nodules. The objective includes visualizing the model's process for interpretability and comparing its diagnostic precision with a cohort of 80 radiologists. We employed ResNet as the classification backbone for thyroid nodule prediction. The model was trained using 2096 ultrasound images of 655 distinct thyroid nodules. For performance evaluation, an independent test set comprising 100 cases of thyroid nodules was curated. In addition, to demonstrate the superiority of the artificial intelligence (AI) model over radiologists, a Turing test was conducted with 80 radiologists of varying clinical experience. This was meant to assess which group of radiologists' conclusions were in closer alignment with AI predictions. Furthermore, to highlight the interpretability of the AI model, gradient-weighted class activation mapping (Grad-CAM) was employed to visualize the model's areas of focus during its prediction process. In this cohort, AI diagnostics demonstrated a sensitivity of 81.67%, a specificity of 60%, and an overall diagnostic accuracy of 73%. In comparison, the panel of radiologists on average exhibited a diagnostic accuracy of 62.9%. The AI's diagnostic process was significantly faster than that of the radiologists. The generated heat-maps highlighted the model's focus on areas characterized by calcification, solid echo and higher echo intensity, suggesting these areas might be indicative of malignant thyroid nodules. Our study supports the notion that deep learning can be a valuable diagnostic tool with comparable accuracy to experienced senior radiologists in the diagnosis of malignant thyroid nodules. The interpretability of the AI model's process suggests that it could be clinically meaningful. Further studies are necessary to improve diagnostic accuracy and support auxiliary diagnoses in primary care settings.

Automated Preselection of Regions of Interest By Deep Learning Facilitates Rapid Whole Slide Image Analysis of Bone Marrow Smears

The analysis of bone marrow smears (BMS) serves as a critical diagnostic tool in hematology, offering valuable insights into the cellular composition and distinctive morphology of bone marrow cells. It enables the accurate diagnosis, classification, and monitoring of various hematologic diseases. Currently, both clinical routine diagnostics and scientific research, especially for the implementation of digital technologies, rely heavily on the manual selection of regions of interest (ROI) suitable for cell-level evaluation. Current guidelines recommend the manual evaluation of up to 500 cells per sample making the process time-consuming, cost-ineffective, subjective, and prone to human error. Consequently, significant inter-observer variability and inconsistencies in diagnosis often arise. Automated analysis of BMS using whole slide images (WSI) offers a promising approach to alleviate financial and time-related burdens while promoting standardization and streamlining of routine diagnostics. We have developed a machine learning approach utilizing a fully convolutional neural network (CNN) specifically designed for semantic segmentation based on the algorithm by Long et al. (IEEE Computer Society, 2015). The model is capable of learning deep semantic representations in its classification backbone combined with shape features in a shallow layer to produce detailed segmentations. The classification backbone consists of an adapted residual neural network with 50 layers (Resnet50) by He et al. (arXiv:1512.03385, 2015). To train the model, we curated a diverse dataset consisting of 60 different WSI BMS, each manually and independently annotated by at least two experts providing a robust ground truth (figure 1). Data augmentation was performed via horizontal flipping, random resizing, and random cropping. A train-test split of 80:20 was implemented with 5-fold internal cross-validation. The model's accuracy was evaluated as mean intersection-over-union ratio yielding high congruency between ground truth and model prediction (figure 2). The final model uses downscaled images effectively reducing data storage size and data transfer time decreasing fully automated ROI prediction time to <300 ms for one WSI. The model allows for in-depth examination at the cellular level, harnessing the potential of thousands of relevant cells present in a WSI, in contrast to the limited range of 200 to 500 cells typically examined in conventional clinical routine diagnostics. Thereby, the model solves an essential bottleneck in computer vision in microscopy minimizing human manual labor and standardizing BMS processing results.

Linear listing order and hierarchical classification: history, conflict, and use

Taxonomic criteria alone are not sufficient to determine a linear sequence for the arrangement of collection specimens according to a preferred classification or the linear sequence according to which taxa are best discussed in articles or books. The choice of methodology to obtain a linear sequence of taxa in agreement with a hierarchical classification has been little studied and remains controversial. In this article, I offer an historical background, before examining properties, use and limits of possible listing criteria. The result of a linearization effort depends on arbitrary choices with respect to two aspects of the hierarchical classification we intend to linearize. One is the order to be followed in listing the immediately subordinate members of a given taxon, the other is the choice of the sets of taxa to be linearized according to tradition, alphabetic order or other criterion. The example presented here, related to the “orders” of Hexapoda, demonstrates the need to specify very clearly the extent and composition of the uncollapsed classification backbone retained in the linearization procedure.

Predicting the efficacy of non-steroidal anti-inflammatory drugs in migraine using deep learning and three-dimensional T1-weighted images

Deep learning (DL) models based on individual images could contribute to tailored therapies and personalized treatment strategies. We aimed to construct a DL model using individual 3D structural images for predicting the efficacy of non-steroidal anti-inflammatory drugs (NSAIDs) in migraine. A 3D convolutional neural network model was constructed, with ResNet18 as the classification backbone, to link structural images to predict the efficacy of NSAIDs. In total, 111 patients were included and allocated to the training and testing sets in a 4:1 ratio. The prediction accuracies of the ResNet34, ResNet50, ResNeXt50, DenseNet121, and 3D ResNet18 models were 0.65, 0.74, 0.65, 0.70, and 0.78, respectively. This model, based on individual 3D structural images, demonstrated better predictive performance in comparison to conventional models. Our study highlights the feasibility of the DL algorithm based on brain structural images and suggests that it can be applied to predict the efficacy of NSAIDs in migraine treatment.

Weakly supervised semantic segmentation via self-supervised destruction learning

Currently, weakly supervised semantic segmentation approaches adopt the Class Activation Map (CAM) to generate the initial attention maps from the standard classification backbone network, with only image-level class labels as training supervision. In this paper, we propose a novel “destruction learning” method via self-supervised manner, producing the CAM attention maps better covering the whole object rather than only the most discriminative regions as previous approaches. Region destruction mechanism is proposed to deliberately “destruct” the global structure in both Mid-Level and Low-Level feature learning following jigsaw puzzle operation, for better local feature extraction of the classification network. Specifically, a Mid-Level Distribution-and-Collection Module is proposed to firstly independently process local patches in parallel and then aggregate them, achieving enhanced local part sensitivity. For low-level “destruction learning”, a Low-Level Destruction Module is proposed to partition the original image into patches and shuffle them randomly to obtain a destructed image, which enforces the network to find weak information from the scattered discriminative small parts. The two destruction modules can be conveniently embedded into arbitrary backbone networks and trained in an end-to-end manner, without any extra annotations. Our proposed method achieves outstanding performance on the PASCAL VOC 2012 dataset (e.g., with 69.1% mIoU on the testing set), which surpasses existing state-of-the-art weakly supervised semantic segmentation methods.

Online intervention siamese tracking

Online target update excels in helping a visual tracking algorithm adapt to variations in target appearance during inference, and thus is preferred by recent advances. However, this technique typically operates in a heuristic manner by updating numerous network parameters with large amounts of online-collected data. This poses a distinct challenge: can the online target update be effectively executed without using the aforementioned approach? To this end, we propose a novel target feature update scheme to reduce the need for tedious data collection and computation-intensive parameter updates. This scheme operates on the principle of causal intervention and is just as effective as default parameter updates in visual tracking. Besides, we explore a novel video-specific target label to capture the context of a specific target in video frames for feature discrimination. This makes target features better fit for appearance changes. Such schemes together with the off-shelf pre-trained classification backbone form a novel online intervention siamese tracker (OIS). When equipped with an unsupervised pre-trained backbone, OIS outperforms current state-of-the-art unsupervised trackers on the OTB and VOT. When exploiting a supervised trained backbone, it competes with typical supervised trackers trained on massive offline training and online tracking data.

UIU-Net: U-Net in U-Net for Infrared Small Object Detection.

Learning-based infrared small object detection methods currently rely heavily on the classification backbone network. This tends to result in tiny object loss and feature distinguishability limitations as the network depth increases. Furthermore, small objects in infrared images are frequently emerged bright and dark, posing severe demands for obtaining precise object contrast information. For this reason, we in this paper propose a simple and effective "U-Net in U-Net" framework, UIU-Net for short, and detect small objects in infrared images. As the name suggests, UIU-Net embeds a tiny U-Net into a larger U-Net backbone, enabling the multi-level and multi-scale representation learning of objects. Moreover, UIU-Net can be trained from scratch, and the learned features can enhance global and local contrast information effectively. More specifically, the UIU-Net model is divided into two modules: the resolution-maintenance deep supervision (RM-DS) module and the interactive-cross attention (IC-A) module. RM-DS integrates Residual U-blocks into a deep supervision network to generate deep multi-scale resolution-maintenance features while learning global context information. Further, IC-A encodes the local context information between the low-level details and high-level semantic features. Extensive experiments conducted on two infrared single-frame image datasets, i.e., SIRST and Synthetic datasets, show the effectiveness and superiority of the proposed UIU-Net in comparison with several state-of-the-art infrared small object detection methods. The proposed UIU-Net also produces powerful generalization performance for video sequence infrared small object datasets, e.g., ATR ground/air video sequence dataset. The codes of this work are available openly at https://github.com/danfenghong/IEEE_TIP_UIU-Net.

Three-Dimensional Face Recognition Using Solid Harmonic Wavelet Scattering and Homotopy Dictionary Learning.

Data representation has been one of the core topics in 3D graphics and pattern recognition in high-dimensional data. Although the high-resolution geometrical information of a physical object can be well preserved in the form of metrical data, e.g., point clouds/triangular meshes, from a regular data (e.g., image/audio) processing perspective, they also bring excessive noise in the course of feature abstraction and regression. For 3D face recognition, preceding attempts focus on treating the scan samples as signals laying on an underlying discrete surface (mesh) or morphable (statistic) models and by embedding auxiliary information, e.g., texture onto the regularized local planar structure to obtain a superior expressive performance to registration-based methods, but environmental variations such as posture/illumination will dissatisfy the integrity or uniform sampling condition, which holistic models generally rely on. In this paper, a geometric deep learning framework for face recognition is proposed, which merely requires the consumption of raw spatial coordinates. The non-uniformity and non-grid geometric transformations in the course of point cloud face scanning are mitigated by modeling each identity as a stochastic process. Individual face scans are considered realizations, yielding underlying inherent distributions under the appropriate assumption of ergodicity. To accomplish 3D facial recognition, we propose a windowed solid harmonic scattering transform on point cloud face scans to extract the invariant coefficients so that unrelated variations can be encoded into certain components of the scattering domain. With these constructions, a sparse learning network as the semi-supervised classification backbone network can work on reducing intraclass variability. Our framework obtained superior performance to current competing methods; without excluding any fragmentary or severely deformed samples, the rank-1 recognition rate (RR1) achieved was 99.84% on the Face Recognition Grand Challenge (FRGC) v2.0 dataset and 99.90% on the Bosphorus dataset.

PDBL: Improving Histopathological Tissue Classification With Plug-and-Play Pyramidal Deep-Broad Learning.

Histopathological tissue classification is a simpler way to achieve semantic segmentation for the whole slide images, which can alleviate the requirement of pixel-level dense annotations. Existing works mostly leverage the popular CNN classification backbones in computer vision to achieve histopathological tissue classification. In this paper, we propose a super lightweight plug-and-play module, named Pyramidal Deep-Broad Learning (PDBL), for any well-trained classification backbone to improve the classification performance without a re-training burden. For each patch, we construct a multi-resolution image pyramid to obtain the pyramidal contextual information. For each level in the pyramid, we extract the multi-scale deep-broad features by our proposed Deep-Broad block (DB-block). We equip PDBL in three popular classification backbones, ShuffLeNetV2, EfficientNetb0, and ResNet50 to evaluate the effectiveness and efficiency of our proposed module on two datasets (Kather Multiclass Dataset and the LC25000 Dataset). Experimental results demonstrate the proposed PDBL can steadily improve the tissue-level classification performance for any CNN backbones, especially for the lightweight models when given a small among of training samples (less than 10%). It greatly saves the computational resources and annotation efforts. The source code is available at: https://github.com/linjiatai/PDBL.

A Computer Vision Model to Identify the Incorrect Use of Face Masks for COVID-19 Awareness

Face mask detection has become a great challenge in computer vision, demanding the coalition of technology with COVID-19 awareness. Researchers have proposed deep learning models to detect the use of face masks. However, the incorrect use of a face mask can be as harmful as not wearing any protection at all. In this paper, we propose a compound convolutional neural network (CNN) architecture based on two computer vision tasks: object localization to discover faces in images/videos, followed by an image classification CNN to categorize the faces and show if someone is using a face mask correctly, incorrectly, or not at all. The first CNN is built upon RetinaFace, a model to detect faces in images, whereas the second CNN uses a ResNet-18 architecture as a classification backbone. Our model enables an accurate identification of people who are not correctly following the COVID-19 healthcare recommendations on face mask use. To enable further global use of our technology, we have released both the dataset used to train the classification model and our proposed computer vision pipeline to the public, and optimized it for embedded systems deployment.

The Bangkok Urbanscapes Dataset for Semantic Urban Scene Understanding Using Enhanced Encoder-Decoder With Atrous Depthwise Separable A1 Convolutional Neural Networks

Semantic segmentation is one of the computer vision tasks which is widely researched at present. It plays an essential role to adapt and apply for real-world use-cases, including the application with autonomous driving systems. To further study self-driving cars in Thailand, we provide both the proposed methods and the proposed dataset in this paper. In the proposed method, we contribute Deeplab-V3-A1 with Xception, which is an extension of DeepLab-V3&#x002B; architecture. Our proposed method as DeepLab-V3-A1 with Xception is enhanced by the different number of <inline-formula> <tex-math notation="LaTeX">$1\times1$ </tex-math></inline-formula> convolution layers on the decoder side and refining the image classification backbone with modification of the Xception model. The experiment was conducted on four datasets: the proposed dataset and three public datasets i.e., the CamVid, the cityscapes, and IDD datasets, respectively. The results show that our proposed strategy as DeepLab-V3-A1 with Xception performs comparably to the baseline methods for all corpora including measurement units such as mean IoU, F1 score, Precision, and Recall. In addition, we benchmark DeepLab-V3-A1 with Xception on the validation set of the cityscapes dataset with a mean IoU of 78.86&#x0025;. For our proposed dataset, we first contribute the Bangkok Urbanscapes dataset, the urban scenes in Southeast Asia. This dataset contains the pair of input images and annotated labels for 701 images. Our dataset consists of various driving environments in Bangkok, as shown for eleven semantic classes (Road, Building, Tree, Car, Footpath, Motorcycle, Pole, Person, Trash, Crosswalk, and Misc). We hope that our architecture and our dataset would help self-driving autonomous developers improve systems for driving in many cities with unique traffic and driving conditions similar to Bangkok and elsewhere in Thailand. Our implementation codes and dataset are available at <uri>https://kaopanboonyuen.github.io/bkkurbanscapes</uri>.

Context-aware Attentional Pooling (CAP) for Fine-grained Visual Classification

Deep convolutional neural networks (CNNs) have shown a strong ability in mining discriminative object pose and parts information for image recognition. For fine-grained recognition, context-aware rich feature representation of object/scene plays a key role since it exhibits a significant variance in the same subcategory and subtle variance among different subcategories. Finding the subtle variance that fully characterizes the object/scene is not straightforward. To address this, we propose a novel context-aware attentional pooling (CAP) that effectively captures subtle changes via sub-pixel gradients, and learns to attend informative integral regions and their importance in discriminating different subcategories without requiring the bounding-box and/or distinguishable part annotations. We also introduce a novel feature encoding by considering the intrinsic consistency between the informativeness of the integral regions and their spatial structures to capture the semantic correlation among them. Our approach is simple yet extremely effective and can be easily applied on top of a standard classification backbone network. We evaluate our approach using six state-of-the-art (SotA) backbone networks and eight benchmark datasets. Our method significantly outperforms the SotA approaches on six datasets and is very competitive with the remaining two.

Video action detection by learning graph-based spatio-temporal interactions

Action Detection is a complex task that aims to detect and classify human actions in video clips. Typically, it has been addressed by processing fine-grained features extracted from a video classification backbone. Recently, thanks to the robustness of object and people detectors, a deeper focus has been added on relationship modeling. Following this line, we propose a graph-based framework to learn high-level interactions between people and objects, in both space and time. In our formulation, spatio-temporal relationships are learned through self-attention on a multi-layer graph structure which can connect entities from consecutive clips, thus considering long-range spatial and temporal dependencies. The proposed module is backbone independent by design and does not require end-to-end training. Extensive experiments are conducted on the AVA dataset, where our model demonstrates state-of-the-art results and consistent improvements over baselines built with different backbones. Code is publicly available at https://github.com/aimagelab/STAGE_action_detection.

Local Enhancement and Bidirectional Feature Refinement Network for Single-Shot Detector

Benefit from multi-scale feature pyramid methods, recently single-stage object detectors have achieved promising accuracy and fast inference speed. However, the majority of existing feature pyramid detection techniques only simply describe complex contextual relationships from different scales. Not only are there no effective modules that adaptively extend appropriate semantic information from deeper layers, but the finer spatial localization cues from lower layers are often ignored. In this paper, we present a Local Enhancement and Bidirectional Feature Refinement Network (LFBFR), which includes two optimization methods to achieve remarkable improvements in detection accuracy. Firstly, to make the backbone more suitable for detection task, we modify the pre-trained classification backbone to mitigate the loss of details in small objects due to consecutive decrease of the image resolution. Then we propose a Bidirectional Feature Refinement Pyramid, which can effectively utilize the inter-channel relationship of higher-level features and fine appearance cues from lower-level features by using the attention residual refinement module and the feature reuse module. Ultimately, to assess the performance of the proposed LFBFR, we design a powerful end-to-end single-stage detector called LFBFR-SSD by embedding it into the framework of SSD. Extensive experiments on the PASCAL VOC and MS COCO verify that our LFBFR-SSD outperforms a lot of state-of-the-art detectors while maintaining a real-time speed.