Scale Variation Of Objects Research Articles

MRSNet: Multi-Resolution Scale Feature Fusion-Based Universal Density Counting Network.

This study focuses on the problem of dense object counting. In dense scenes, variations in object scales and uneven distributions greatly hinder counting accuracy. The current methods, whether CNNs with fixed convolutional kernel sizes or Transformers with fixed attention sizes, struggle to handle such variability effectively. Lower-resolution features are more sensitive to larger objects closer to the camera, while higher-resolution features are more efficient for smaller objects further away. Thus, preserving features that carry the most relevant information at each scale is crucial for improving counting precision. Motivated by this, we propose a multi-resolution scale feature fusion-based universal density counting network (MRSNet). It utilizes independent modules to process high- and low-resolution features, adaptively adjusts receptive field sizes, and incorporates dynamic sparse attention mechanisms to optimize feature information at each resolution, by integrating optimal features across multiple scales into density maps for counting evaluation. Our proposed network effectively mitigates issues caused by large variations in object scales, thereby enhancing counting accuracy. Furthermore, extensive quantitative analyses on six public datasets demonstrate the algorithm's strong generalization ability in handling diverse object scale variations.

Enhanced multi-scale networks for semantic segmentation

Multi-scale representation provides an effective answer to the scale variation of objects and entities in semantic segmentation. The ability to capture long-range pixel dependency facilitates semantic segmentation. In addition, semantic segmentation necessitates the effective use of pixel-to-pixel similarity in the channel direction to enhance pixel areas. By reviewing the characteristics of earlier successful segmentation models, we discover a number of crucial elements that enhance segmentation model performance, including a robust encoder structure, multi-scale interactions, attention mechanisms, and a robust decoder structure. The attention mechanism of the asymmetric non-local neural network (ANNet) is merged with multi-scale pyramidal modules to accelerate model segmentation while maintaining high accuracy. However, ANNet does not account for the similarity between pixels in the feature map channel direction, making the segmentation accuracy unsatisfactory. As a result, we propose EMSNet, a straightforward convolutional network architecture for semantic segmentation that consists of Integration of enhanced regional module (IERM) and Multi-scale convolution module (MSCM). The IERM module generates weights using four or five-stage feature maps, then fuses the input features with the weights and uses more computation. The similarity of the channel direction feature graphs is also calculated using ANNet’s auxiliary loss function. The MSCM module can more accurately describe the interactions between various channels, capture the interdependencies between feature pixels, and capture the multi-scale context. Experiments prove that we perform well in tests using the benchmark dataset. On Cityscapes test data, we get 82.2% segmentation accuracy. The mIoU in the ADE20k and Pascal VOC datasets are, respectively, 45.58% and 85.46%.

Towards object tracking for quadruped robots

With the development of quadruped robot technology, object tracking for quadruped robots has become an important research topic where violent camera shaking caused by the robot’s movement makes this task very challenging. In this letter, a quadruped robot object tracking dataset (QROD-111), including 111 video sequences, is first established, which was collected through our quadruped robot platform. A tracking algorithm based on Siamese network is then proposed where an alignment module is introduced to alleviate tracking difficulties caused by the quadruped robot movement. Moreover, a scale adaptation subnetwork is designed to alleviate the impact of the object scale variation during the whole tracking process. Experimental results demonstrate that our algorithm can achieve advanced performance for quadruped robot object tracking.

Scale-Aware Siamese Object Tracking for Vision-Based UAM Approaching

In many industrial applications of unmanned aerial manipulator (UAM), visual approaching the object is crucial to subsequent manipulating. In comparison with the widely-studied manipulating, the key to efficient vision-based UAM approaching, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i.e.</i> , UAM object tracking, is still limited. Since traditional model-based UAM tracking is costly and cannot track arbitrary objects, an intuitive solution is to introduce state-of-the-art model-free Siamese trackers from the visual tracking field. Although Siamese tracking is most suitable for the onboard embedded processors, severe object scale variation in UAM tracking brings formidable challenges. To address these problems, this work proposes a novel model-free scale-aware Siamese tracker (SiamSA). Specifically, a scale attention network is proposed to emphasize scale awareness in feature processing. A scale-aware anchor proposal network is designed to achieve anchor proposing. Besides, two novel UAM tracking benchmarks are first recorded. Comprehensive experiments on benchmarks validate the effectiveness of SiamSA. Furthermore, real-world tests also confirm practicality for industrial UAM approaching tasks with high efficiency and robustness.

A Lightweight YOLOv5-Based Model with Feature Fusion and Dilation Convolution for Image Segmentation

Image segmentation has played an essential role in computer vision. The target detection model represented by YOLOv5 is widely used in image segmentation. However, YOLOv5 has performance bottlenecks such as object scale variation, object occlusion, computational volume, and speed when processing complex images. To solve these problems, an enhanced algorithm based on YOLOv5 is proposed. MobileViT is used as the backbone network of the YOLOv5 algorithm, and feature fusion and dilated convolution are added to the model. This method is validated on the COCO and PASCAL-VOC datasets. Experimental results show that it significantly reduces the processing time and achieves high segmentation quality with an accuracy of 95.32% on COCO and 96.02% on PASCAL-VOC. The improved model is 116 M, 52 M, and 76 M, smaller than U-Net, SegNet, and Mask R-CNN, respectively. This paper provides a new idea and method with which to solve the problems in the field of image segmentation, and the method has strong practicality and generalization value.

Learning Event-Relevant Factors for Video Anomaly Detection

Most video anomaly detection methods discriminate events that deviate from normal patterns as anomalies. However, these methods are prone to interferences from event-irrelevant factors, such as background textures and object scale variations, incurring an increased false detection rate. In this paper, we propose to explicitly learn event-relevant factors to eliminate the interferences from event-irrelevant factors on anomaly predictions. To this end, we introduce a causal generative model to separate the event-relevant factors and event-irrelevant ones in videos, and learn the prototypes of event-relevant factors in a memory augmentation module. We design a causal objective function to optimize the causal generative model and develop a counterfactual learning strategy to guide anomaly predictions, which increases the influence of the event-relevant factors. The extensive experiments show the effectiveness of our method for video anomaly detection.

PolarFormer: Multi-Camera 3D Object Detection with Polar Transformer

3D object detection in autonomous driving aims to reason “what” and “where” the objects of interest present in a 3D world. Following the conventional wisdom of previous 2D object detection, existing methods often adopt the canonical Cartesian coordinate system with perpendicular axis. However, we conjugate that this does not fit the nature of the ego car’s perspective, as each onboard camera perceives the world in shape of wedge intrinsic to the imaging geometry with radical (non perpendicular) axis. Hence, in this paper we advocate the exploitation of the Polar coordinate system and propose a new Polar Transformer (PolarFormer) for more accurate 3D object detection in the bird’s-eye-view (BEV) taking as input only multi-camera 2D images. Specifically, we design a cross-attention based Polar detection head without restriction to the shape of input structure to deal with irregular Polar grids. For tackling the unconstrained object scale variations along Polar’s distance dimension, we further introduce a multi-scale Polar representation learning strategy. As a result, our model can make best use of the Polar representation rasterized via attending to the corresponding image observation in a sequence-to-sequence fashion subject to the geometric constraints. Thorough experiments on the nuScenes dataset demonstrate that our PolarFormer outperforms significantly state-of-the-art 3D object detection alternatives.

Adaptive Discriminative Regions Learning Network for Remote Sensing Scene Classification.

As an auxiliary means of remote sensing (RS) intelligent interpretation, remote sensing scene classification (RSSC) attracts considerable attention and its performance has been improved significantly by the popular deep convolutional neural networks (DCNNs). However, there are still several challenges that hinder the practical applications of RSSC, such as complex composition of land cover, scale-variation of objects, and redundant and noisy areas for scene classification. In order to mitigate the impact of these issues, we propose an adaptive discriminative regions learning network for RSSC, referred as ADRL-Net briefly, which locates discriminative regions effectively for boosting the performance of RSSC by utilizing a novel self-supervision mechanism. Our proposed ADRL-Net consists of three main modules, including a discriminative region generator, a region discriminator, and a region scorer. Specifically, the discriminative region generator first generates some candidate regions which could be informative for RSSC. Then, the region discriminator evaluates the regions generated by region generator and provides feedback for the generator to update the informative regions. Finally, the region scorer makes prediction scores for the whole image by using the discriminative regions. In such a manner, the three modules of ADRL-Net can cooperate with each other and focus on the most informative regions of an image and reduce the interference of redundant regions for final classification, which is robust to the complex scene composition, object scales, and irrelevant information. In order to validate the efficacy of the proposed network, we conduct experiments on four widely used benchmark datasets, and the experimental results demonstrate that ADRL-Net consistently outperforms other state-of-the-art RSSC methods.

Anchor-Free Object Detection with Scale-Aware Networks for Autonomous Driving

Current anchor-free object detectors do not rely on anchors and obtain comparable accuracy with anchor-based detectors. However, anchor-free object detectors that adopt a single-level feature map and lack a feature pyramid network (FPN) prior information about an object’s scale; thus, they insufficiently adapt to large object scale variation, especially for autonomous driving in complex road scenes. To address this problem, we propose a divide-and-conquer solution and attempt to introduce some prior information about object scale variation into the model when maintaining a streamlined network structure. Specifically, for small-scale objects, we add some dense layer jump connections between the shallow high-resolution feature layers and the deep high-semantic feature layers. For large-scale objects, dilated convolution is used as an ingredient to cover the features of large-scale objects. Based on this, a scale adaptation module is proposed. In this module, different dilated convolution expansion rates are utilized to change the network’s receptive field size, which can adapt to changes from small-scale to large-scale. The experimental results show that the proposed model has better detection performance with different object scales than existing detectors.

EASNet: a multiscale attention semantic segmentation network combined with asymmetric convolution

The huge scale variation of objects in scene images limits the performance of deep neural networks based on single-scale image input. To tackle the scale variation issue, we propose an improved multiscale attention network based on enhancing asymmetric convolution and strip pooling (EASNet), which uses three different inference scales. First, complementary weight masks are employed from branches of coarse scale to fine scale, to enhance predictions across scales. Second, the strip pooling module is adopted as strip pooling attention module (SPAM) in the coarser scales to produce the weight masks. It will make maximum use of the coarser scale, improving both the accuracy of boundary segmentation and the integrity of body segmentation for large targets or homogeneous objects but distributed discretely. Besides, enhancing asymmetric convolution block is designed to obtain richer and more robust feature information before SPAM. The effectiveness of EASNet experiments is validated using benchmarks (PASCAL VOC 2012 Aug and Cityscapes). The experimental results demonstrate that our method outperforms or is comparable to state-of-the-art methods by the widely used criteria mean intersection over union.

Semantics reused context feature pyramid network for object detection in remote sensing images

Object detection plays an important role in the field of remote sensing (RS) images analysis. The advancement of object detection task for RS images is extremely challenging due to object scale variation and complex background. Almost all detection frameworks use the neck network to fuse the feature maps extracted by the backbone to obtain better features, among which feature pyramid network (FPN) is the most widely used. Although traditional FPN has shown great potential in multi-scale object detection based on deep learning, it has unsatisfactory detection accuracy for small objects and confusing objects in RS images because of the lack of rich semantic and contextual information. We propose an architecture, called semantics reused context FPN that is portable to any FPN-based detectors to boost the detection performance in RS images without parameters increasing significantly. It includes two blocks, namely, context feature enhanced block, which uses dense connection and a learnable branch structure to extract rich context features with multiple receptive fields, and semantic feature reused block, which enhances semantic information of shallow feature maps by reusing later-layer features. Comprehensive evaluations on three benchmark datasets of geospatial object detection demonstrate that our method is superior to the existing methods.

Deformable Part Region Learning for Object Detection

In a convolutional object detector, the detection accuracy can be degraded often due to the low feature discriminability caused by geometric variation or transformation of an object. In this paper, we propose a deformable part region learning in order to allow decomposed part regions to be deformable according to geometric transformation of an object. To this end, we introduce trainable geometric parameters for the location of each part model. Because the ground truth of the part models is not available, we design classification and mask losses for part models, and learn the geometric parameters by minimizing an integral loss including those part losses. As a result, we can train a deformable part region network without extra super-vision and make each part model deformable according to object scale variation. Furthermore, for improving cascade object detection and instance segmentation, we present a Cascade deformable part region architecture which can refine whole and part detections iteratively in the cascade manner. Without bells and whistles, our implementation of a Cascade deformable part region detector achieves better detection and segmentation mAPs on COCO and VOC datasets, compared to the recent cascade and other state-of-the-art detectors.

Autonomous health assessment of civil infrastructure using deep learning and smart devices

Damage detection via drones is fundamental in infrastructure health assessment. However, object scale variation due to drones' swift movement and sparse scenes make damage detection challenging. This paper describes a multi-task framework, EnsembleDetNet, for damage detection and multi-label scene classification by leveraging object detectors and classifiers based on ensemble learning which induces diversity and strength-correlation. Further, a novel attention module that significantly improves EnsembleDetNet by about 5% is proposed via explicit ensembling of parallel and sequential channel and spatial attention maps. Extensive experiments with a public dataset and an onsite verification utilizing a micro drone indicate that EsembleDetNet outperforms state-of-the-art detectors and classifiers under variant evaluation metrics. EnsembleDetNet has the potential to become a new paradigm in infrastructure health assessment.

Object detectors involving a NAS-gate convolutional module and capsule attention module

Several state-of-the-art object detectors have demonstrated outstanding performances by optimizing feature representation through modification of the backbone architecture and exploitation of a feature pyramid. To determine the effectiveness of this approach, we explore the modification of object detectors’ backbone and feature pyramid by utilizing Neural Architecture Search (NAS) and Capsule Network. We introduce two modules, namely, NAS-gate convolutional module and Capsule Attention module. The NAS-gate convolutional module optimizes standard convolution in a backbone network based on differentiable architecture search cooperation with multiple convolution conditions to overcome object scale variation problems. The Capsule Attention module exploits the strong spatial relationship encoding ability of the capsule network to generate a spatial attention mask, which emphasizes important features and suppresses unnecessary features in the feature pyramid, in order to optimize the feature representation and localization capability of the detectors. Experimental results indicate that the NAS-gate convolutional module can alleviate the object scale variation problem and the Capsule Attention network can help to avoid inaccurate localization. Next, we introduce NASGC-CapANet, which incorporates the two modules, i.e., a NAS-gate convolutional module and capsule attention module. Results of comparisons against state-of-the-art object detectors on the MS COCO val-2017 dataset demonstrate that NASGC-CapANet-based Faster R-CNN significantly outperforms the baseline Faster R-CNN with a ResNet-50 backbone and a ResNet-101 backbone by mAPs of 2.7% and 2.0%, respectively. Furthermore, the NASGC-CapANet-based Cascade R-CNN achieves a box mAP of 43.8% on the MS COCO test-dev dataset.

CrossNet: Detecting Objects as Crosses

With the use of deep learning, object detection has achieved great breakthroughs. However, existing object detection methods still can not cope with challenging environments, such as dense objects, small objects, and object scale variations. To address these issues, this paper proposes a novel keypoint-based detection framework, called CrossNet, which significantly improves detection performance with minimal costs. In our approach, an object is modeled as a cross that consists of a center keypoint and a specific size, which eliminates the need of hand-craft anchor design. The proposed CrossNet outputs three types of maps: the center map, size map, and offset map, where both center map and offset map are to predict the center keypoints of objects and the size map is to estimate the sizes (width and height) of objects. Specifically, we first design a cascaded center prediction method that introduces a coarse-to-fine idea to improve center prediction. Furthermore, since center prediction considered as a classification task is easier than size regression relatively, we design a center-attention size regression module that uses the detection results of centers to assist the size prediction. In addition, a slightly modified hourglass network is designed to enhance the quality of feature maps for center and size prediction. Extensive experiments are conducted to demonstrate the effectiveness of CrossNet on the challenging PASCAL VOC, COCO, KITTI, and WiderFace datasets. Empirical studies show that CrossNet achieves competitive results with top-ranked one-stage and two-stage detectors while being time-efficient.

Few-Shot Object Detection via Context-Aware Aggregation for Remote Sensing Images

Few-shot object detection methods have made prodigious progress in recent years. However, these methods are designed for optical images at a single scale, which leads to significantly degraded detection performance due to object scale variation of remote sensing images. In this letter, we propose a few-shot object detection method for the problem of scale variation in remote sensing images. More specifically, our model contains two main components: a context-aware pixel aggregation (CPA) that allows the model to adapt to objects at different scales through different scale convolution and a context-aware feature aggregation (CFA) that enhances context awareness to obtain more semantic information through a graph convolution network (GCN). Experiments on the DIOR dataset demonstrate that our model can achieve a satisfying detection performance on remote sensing images, and our model performs significantly better than the state-of-the-art model.

Dual-det : a fast detector for oriented object detection in aerial images

ABSTRACT Fast and accurate object detection in aerial images remains a challenging task. Usually, to better describe an object, oriented bounding boxes (OBBs) can better fit objects. Due to high background complexity and large object scale variation, single-angle anchor-based two-stage detectors are widely adopted, which offer better accuracy. However, the single-angle prediction has a small error tolerance for objects with a large aspect ratio, and the hyperparameters of the anchor-based network are difficult to adjust, and the number of hyperparameters is extremely large. Furthermore, the two-stage detection inference speed is slow, and it is difficult to achieve real-time detection. In this paper, we propose Dual-Det, a keypoint-based oriented object detector. We firstly propose a dual-angle with a short-side and ratio regression strategy (DASR), which uses the object centre and the length and angles of two diagonals to represent an object. A short side guided (SSG) loss is further added to guide the direction of the diagonal regression box. To improve the detection performance for dense and tiny objects, a lightweight supervised pixel attention learner is finally proposed. The experiment results show that Dual-Det achieves 90.23 mAP at 46FPS on HRSC2016, 90.83 mAP at 46FPS on UCAS-AOD and 72.00 mAP at 0.018 s per image in the inference phase on DOTA. The code will be open source on https://github.com/gqy4166000/ijrs_dasr.

Human gaze-aware attentive object detection for ambient intelligence

Understanding human behavior and the surrounding environment is essential for realizing ambient intelligence (AmI), for which eye gaze and object information are reliable cues. In this study, the authors propose a novel human gaze-aware attentive object detection framework as an elemental technology for AmI. The proposed framework detects users’ attentive objects and shows more precise and robust performance against object-scale variations. A novel Adaptive-3D-Region-of-Interest (Ada-3D-RoI) scheme is designed as a front-end module, and scalable detection network structures are proposed to maximize cost-efficiency. The experiments show that the detection rate is improved up to 97.6% on small objects (14.1% on average), and it is selectively tunable with a tradeoff between accuracy and computational complexity. In addition, the qualitative results demonstrate that the proposed framework detects a user’s single object-of-interest only, even when the target object is occluded or extremely small. Complementary matters for follow-up study are presented as suggestions to extend the results of the proposed framework to further practical AmI applications. This study will help develop advanced AmI applications that demand a higher-level understanding of scene context and human behavior such as human–robot symbiosis, remote-/autonomous control, and augmented/mixed reality.

BIDIRECTIONAL MULTI-SCALE ATTENTION NETWORKS FOR SEMANTIC SEGMENTATION OF OBLIQUE UAV IMAGERY

Abstract. Semantic segmentation for aerial platforms has been one of the fundamental scene understanding task for the earth observation. Most of the semantic segmentation research focused on scenes captured in nadir view, in which objects have relatively smaller scale variation compared with scenes captured in oblique view. The huge scale variation of objects in oblique images limits the performance of deep neural networks (DNN) that process images in a single scale fashion. In order to tackle the scale variation issue, in this paper, we propose the novel bidirectional multi-scale attention networks, which fuse features from multiple scales bidirectionally for more adaptive and effective feature extraction. The experiments are conducted on the UAVid2020 dataset and have shown the effectiveness of our method. Our model achieved the state-of-the-art (SOTA) result with a mean intersection over union (mIoU) score of 70.80%.

Multi-Level and Multi-Scale Feature Aggregation Network for Semantic Segmentation in Vehicle-Mounted Scenes.

The main challenges of semantic segmentation in vehicle-mounted scenes are object scale variation and trading off model accuracy and efficiency. Lightweight backbone networks for semantic segmentation usually extract single-scale features layer-by-layer only by using a fixed receptive field. Most modern real-time semantic segmentation networks heavily compromise spatial details when encoding semantics, and sacrifice accuracy for speed. Many improving strategies adopt dilated convolution and add a sub-network, in which either intensive computation or redundant parameters are brought. We propose a multi-level and multi-scale feature aggregation network (MMFANet). A spatial pyramid module is designed by cascading dilated convolutions with different receptive fields to extract multi-scale features layer-by-layer. Subseqently, a lightweight backbone network is built by reducing the feature channel capacity of the module. To improve the accuracy of our network, we design two additional modules to separately capture spatial details and high-level semantics from the backbone network without significantly increasing the computation cost. Comprehensive experimental results show that our model achieves 79.3% MIoU on the Cityscapes test dataset at a speed of 58.5 FPS, and it is more accurate than SwiftNet (75.5% MIoU). Furthermore, the number of parameters of our model is at least 53.38% less than that of other state-of-the-art models.