Scale-aware Network Research Articles

ESA-Net: An efficient scale-aware network for small crop pest detection

Pest detection aims to locate and classify the pests that may be present in the image, which plays a crucial role in the early warning of pests linked to the agriculture industry. However, crop pests are generally very small and uneven in scale, resulting in poor detection results. To address this challenge, we propose the Efficient Scale-Aware Network (ESA-Net), which comprises three core components: a High-Level Semantic Feature Extraction Module (HSFEM), a Low-Level Feature Enhancement Module (LFEM), and a Dynamic Scale-Aware Head (DSH). To avoid network optimization confusion, we use the HSFEM to extract high-level features that build the feature pyramid networks. Then, the LFEM is designed to further optimize the integrated feature map of low-level features, providing more fine-gained information for feature extraction and improving the detector's ability to differentiate between pests with similar appearances. Additionally, the proposed DSH can improve the detection performance of small objects by adaptively selecting the appropriate detection receptive field in accordance with various object scales. Experiment results on LMPD2020 dataset (68.8% mAP) and APHIDc dataset (75.3% mAP) demonstrate that ESA-Net achieves competitive results compared to the state-of-the-art methods.

Object Counting in Remote Sensing via Triple Attention and Scale-Aware Network

Object counting is a fundamental task in remote sensing analysis. Nevertheless, it has been barely studied compared with object counting in natural images due to the challenging factors, e.g., background clutter and scale variation. This paper proposes a triple attention and scale-aware network (TASNet). Specifically, a triple view attention (TVA) module is adopted to remedy the background clutter, which executes three-dimension attention operations on the input tensor. In this case, it can capture the interaction dependencies between three dimensions to distinguish the object region. Meanwhile, a pyramid feature aggregation (PFA) module is employed to relieve the scale variation. The PFA module is built in a four-branch architecture, and each branch has a similar structure composed of dilated convolution layers to enlarge the receptive field. Furthermore, a scale transmit connection is introduced to enable the lower branch to acquire the upper branch’s scale, increasing the output’s scale diversity. Experimental results on remote sensing datasets prove that the proposed model can address the issues of background clutter and scale variation. Moreover, it outperforms the state-of-the-art (SOTA) competitors subjectively and objectively.

Scale-aware network with modality-awareness for RGB-D indoor semantic segmentation

This paper focuses on indoor semantic segmentation based on RGB-D images. Semantic segmentation is a pixel-level classification task that has made steady progress based on fully convolutional networks (FCNs). However, we find there is still room for improvements in the following three aspects. The first is related to multi-scale feature extraction. Recent state-of-the-art works forcibly concatenate multi-scale feature representations extracted by spatial pyramid pooling, dilated convolution or other architectures, regardless of the spatial extent for each pixel. The second is regarding RGB-D modal fusion. Most successful methods treat RGB and depth as two separate modalities and force them to be joined together regardless of their different contributions to the final prediction. The final aspect is about the modeling ability of extracted features. Due to the “local grid” defined by the receptive field, the learned feature representation lacks the ability to model spatial dependencies. In addition to these modules, we design a depth estimation module to encourage the RGB network to extract more effective features. To solve the above challenges, we propose four modules to address them: scale-aware module, modality-aware module, attention module and depth estimation module. Extensive experiments on the NYU-Depth v2 and SUN RGB-D datasets demonstrate that our method is effective against RGB-D indoor semantic segmentation.

Scale-Aware Network with Scale Equivariance

The convolutional neural network (CNN) has achieved good performance in object classification due to its inherent translation equivariance, but its scale equivariance is poor. A Scale-Aware Network (SA Net) with scale equivariance is proposed to estimate the scale during classification. The SA Net only learns samples of one scale in the training stage; in the testing stage, the unknown-scale testing samples are up-sampled and down-sampled, and a group of image copies with different scales are generated to form the image pyramid. The up-sampling adopts interpolation, and the down-sampling adopts interpolation combined with wavelet transform to avoid spectrum aliasing. The generated test samples with different scales are sent to the Siamese network with weight sharing for inferencing. According to the position of the maximum value of the classification-score matrix, the testing samples can be classified and the scale can be estimated simultaneously. The results on the MNIST and FMNIST datasets show that the SA Net has better performance than the existing methods. When the scale is larger than 4, the SA Net has higher classification accuracy than other methods. In the scale-estimation experiment, the SA Net can achieve low relative RMSE on any scale. The SA Net has potential for effective use in remote sensing, optical image recognition and medical diagnosis in cytohistology.

Multiregion Scale-Aware Network for Building Extraction From High-Resolution Remote Sensing Images

Building extraction is an essential task due to its relevance to urban planning and automatic surveying mapping activities. Despite the existing convolutional neural network-based methods that have achieved remarkable progress on building extraction from remote sensing images, the accurate extraction of buildings with extremely large variations of scales and layouts is still challenging. In this study, a novel multiregion scale-aware network is proposed to address these issues. The network consists of two key components. First, a multiregion attention module is proposed to capture long-range context dependencies and exploit different regions’ attention information, alleviating the interference of cluttered backgrounds and variations in building layouts. With the multiscale features generated by a backbone network as input, a graph-based scale-aware structure is designed to model and reason the interactions between different scale features to enable a better understanding of multiscale features. Extensive experiments conducted on three datasets demonstrate that the proposed method achieves superior performance compared with the other state-of-the-art methods.

ES-Net: Efficient Scale-Aware Network for Tiny Defect Detection

Defect detection is to locate and classify the possible defects in an image, which plays a key role in the quality inspection link in the manufacturing process of industrial products. Defects in industrial products are generally very small and extremely uneven in scale, resulting in poor detection results. Therefore, we propose an efficient scale-aware network (ES-Net) to improve the effect of defect detection. By addressing the information loss of tiny targets and the mismatch between the receptive field of detection head and the scale of targets, ES-Net improves the overall defect detection effect, especially for tiny defects. Considering that existing works directly use an integrated feature to enhance features at all levels, it may cause confusion in the direction of network optimization. Therefore, we propose the aggregated feature guidance module (AFGM), which first performs efficient cascading fusion of multi-level features to filter cross-layer conflicts. Then the split and aggregation enhancement (SAE) module is designed to further optimize the integrated feature map, and the result is used to guide the shallow features. Moreover, we also introduce the multi-receptive field fusion (MFF) module to generate multi-receptive field information to supplement the shallow features after dimensionality reduction. The efficient stair pyramid (ESP) is a further improvement of feature pyramid network (FPN)-based network. In particular, we propose the dynamic scale-aware head (DSH) in shallow detection layer, which can adaptively select the best detection receptive field according to different scales of targets, thereby improving the detection performance of tiny targets. Extensive experimental results on Aliyun Tianchi fabric dataset (76.2% mAP), NEU-DET (79.1% mAP), and printed circuit board (PCB) defect dataset of Peking University (97.5% mAP) demonstrate the proposed ES-Net achieves competitive results compared to the state-of-the-art (SOTA) methods. Moreover, the high efficiency of ES-Net makes it more applicable in scenarios with high real-time requirements.

Feature Split–Merge–Enhancement Network for Remote Sensing Object Detection

Recently, multicategory object detection in high-resolution remote sensing images is still a challenge. First, objects with significant scale differences exist in one scene simultaneously, so it is generally difficult for the detectors to balance the detection performance of large and small objects. Second, because of the complex background and the objects&#x2019; densely distributed characteristics in the remote sensing images, the extracted features usually have noise and blurred boundaries, which interfere with the detection performance of the object detectors. With this observation, we propose an end-to-end scale-aware network called feature split&#x2013;merge&#x2013;enhancement network (SME-Net) for remote sensing object detection, composed of the feature split-and-merge (FSM) module, the offset-error rectification (OER) module, and the object saliency enhancement (OSE) strategy. FSM eliminates salient information of large objects to highlight the features of small objects in the shallow feature maps. It also transmits the effective detailed features of large objects to the deep feature maps, alleviating feature confusion between multiscale objects. OER corrects the inconsistency of the features spatial layout among the multilayer feature maps by the proposed offset loss, so as to achieve supervised elimination and transmission in FSM. OSE enhances the features of interests and suppresses the background information by the proposed membership function, thus preventing false detection and missed detection caused by noise and blurred boundaries. The effectiveness of the proposed algorithm has been verified on multiple datasets. Our code is available at: <uri>https://github.com/Momuli/SMENet.git</uri>

Real-time crowd counting via lightweight scale-aware network

Accurate crowd counting in congested scenes is still challenging, and its inference speed is also essential in intelligent edge devices. To tackle this issue, we propose a novel lightweight scale-aware network (LSANet) for high-accuracy real-time crowd counting, which consists of four structures: local extractor (LE), scale feature extraction module (SFEM), scale feature fusion module (SFFM), and density map regressor (DMR). Specifically, we devise the SFEM and SFFM carefully to reinforce the learning capability of scale representation, which significantly improves the counting accuracy. Besides, the multi-level intersection-over-union (MIoU) loss function is proposed to reduce background false recognition and estimation errors. Extensive experiments on four mainstream datasets demonstrate that our method achieves the optimal trade-off between effectiveness and efficiency compared with previous lightweight methods.

Learning scale awareness in keypoint extraction and description

To recover relative camera motion accurately and robustly, establishing a set of point-to-point correspondences in the pixel space is an essential yet challenging task in computer vision. Even though multi-scale design philosophy has been used with significant success in computer vision tasks, such as object detection and semantic segmentation, learning-based image matching has not been fully exploited. In this work, we explore a scale awareness learning approach in finding pixel-level correspondences based on the intuition that keypoints need to be extracted and described on an appropriate scale. With that insight, we propose a novel scale-aware network and then develop a new fusion scheme that derives high-consistency response maps and high-precision descriptions. We also revise the Second Order Similarity Regularization (SOSR) to make it more effective for the end-to-end image matching network, which leads to significant improvement in local feature descriptions. Experimental results run on multiple datasets demonstrate that our approach performs better than state-of-the-art methods under multiple criteria.

Adaptive Anchor for Fast Object Detection in Aerial Image

Object detection in aerial images is an important task for many applications such as reconnaissance, surveillance, search, and rescue. At present, convolution neural network-based aerial image object detection algorithms mainly focus on rotation invariance and scale invariance, but ignore an important characteristic of the aerial image that the image captured height is an important prior knowledge. At the same captured height, the target has a clear scale range. In this letter, a scale-aware network is proposed to determine the scale of predefined anchors, which can effectively reduce the scale search range, reduce the risk of overfitting, and improve the detection accuracy and speed in aerial images. Experiments on the VisDrone data set show that the proposed method can not only improve the detection speed by 18% but also improve the average accuracy by 1.6%.