Anchor-free Network Research Articles

One-Stage Anchor-Free Online Multiple Target Tracking With Deformable Local Attention and Task-Aware Prediction.

The tracking-by-detection paradigm currently dominates multiple target tracking algorithms. It usually includes three tasks: target detection, appearance feature embedding, and data association. Carrying out these three tasks successively usually leads to lower tracking efficiency. In this paper, we propose a one-stage anchor-free multiple task learning framework which carries out target detection and appearance feature embedding in parallel to substantially increase the tracking speed. This framework simultaneously predicts a target detection and produces a feature embedding for each location, by sharing a pyramid of feature maps. We propose a deformable local attention module which utilizes the correlations between features at different locations within a target to obtain more discriminative features. We further propose a task-aware prediction module which utilizes deformable convolutions to select the most suitable locations for the different tasks. At the selected locations, classification of samples into foreground or background, appearance feature embedding, and target box regression are carried out. Two effective training strategies, regression range overlapping and sample reweighting, are proposed to reduce missed detections in dense scenes. Ambiguous samples whose identities are difficult to determine are effectively dealt with to obtain more accurate feature embedding of target appearance. An appearance-enhanced non-maximum suppression is proposed to reduce over-suppression of true targets in crowded scenes. Based on the one-stage anchor-free network with the deformable local attention module and the task-aware prediction module, we implement a new online multiple target tracker. Experimental results show that our tracker achieves a very fast speed while maintaining a high tracking accuracy.

Dual-branch information extraction and local attention anchor-free network for defect detection

In the production process, the presence of surface defects seriously affects the quality of industrial products. Existing defect detectors are not suitable for surface with scattered distribution and complex texture of defects. In this study, a dual-branch information extraction and local attention anchor-free network for defect detection (DLA-FCOS), which is based on the fully convolutional one-stage network, is proposed to accurately locate and detect surface defects of industrial products. Firstly, a dual-branch feature extraction network (DFENeT) is proposed and used to improve the extraction ability of complex defects. Then, a local feature enhancement module is proposed, and a residual connection is established to enrich local semantic information. Meanwhile, the self-attention mechanism is introduced to form local attentional residual feature pyramid networks (LA-RFPN) to eliminate the influences of feature misalignments. The mean average accuracy (mAP) and frames per second (FPS) of the proposed DLA-FCOS on the cut layer of the tobacco packet defect dataset (CLTP-DD) are 96.8% and 20.7, respectively, which meets the requirements for accurate and real-time defect detection. Meanwhile, the average accuracy of the proposed DLA-FCOS on the NEU-DET and GC10-DET datasets is 78.4% and 67.7%, respectively. The results demonstrate that the DLA-FCOS has good feasibility and high generalization capability to perform defect detection tasks of industrial products.

Ship Target Detection in Optical Remote Sensing Images Based on E2YOLOX-VFL

In this research, E2YOLOX-VFL is proposed as a novel approach to address the challenges of optical image multi-scale ship detection and recognition in complex maritime and land backgrounds. Firstly, the typical anchor-free network YOLOX is utilized as the baseline network for ship detection. Secondly, the Efficient Channel Attention module is incorporated into the YOLOX Backbone network to enhance the model’s capability to extract information from objects of different scales, such as large, medium, and small, thus improving ship detection performance in complex backgrounds. Thirdly, we propose the Efficient Force-IoU (EFIoU) Loss function as a replacement for the Intersection over Union (IoU) Loss, addressing the issue whereby IoU Loss only considers the intersection and union between the ground truth boxes and the predicted boxes, without taking into account the size and position of targets. This also considers the disadvantageous effects of low-quality samples, resulting in inaccuracies in measuring target similarity, and improves the regression performance of the algorithm. Fourthly, the confidence loss function is improved. Specifically, Varifocal Loss is employed instead of CE Loss, effectively handling the positive and negative sample imbalance, challenging samples, and class imbalance, enhancing the overall detection performance of the model. Then, we propose Balanced Gaussian NMS (BG-NMS) to solve the problem of missed detection caused by the occlusion of dense targets. Finally, the E2YOLOX-VFL algorithm is tested on the HRSC2016 dataset, achieving a 9.28% improvement in mAP compared to the baseline YOLOX algorithm. Moreover, the detection performance using BG-NMS is also analyzed, and the experimental results validate the effectiveness of the E2YOLOX-VFL algorithm.

A Compact and High-Efficiency Anchor-Free Network Based on Contour Key Points for SAR Ship Detection

A Compact and High-Efficiency Anchor-Free Network Based on Contour Key Points for SAR Ship Detection

Optimized anchor-free network for dense rotating object detection in remote sensing images

Optimized anchor-free network for dense rotating object detection in remote sensing images

DsP-YOLO: An anchor-free network with DsPAN for small object detection of multiscale defects

Industrial defect detection is of great significance to ensure the quality of industrial products. The surface defects of industrial products are characterized by multiple scales, multiple types, abundant small objects, and complex background interference. In particular, small object detection of multiscale defects under complex background interference poses significant challenges for defect detection tasks. How to improve the algorithm’s ability to detect industrial defects, especially in enhancing the detection capabilities of small-sized defects, while ensuring that the inference speed is not overly affected is a long-term prominent challenge. Aiming at achieving accurate and fast detection of industrial defects, this paper proposes an anchor-free network with DsPAN for small object detection. The core of this method is to propose a new idea i.e., feature enhancement in the feature fusion network for the feature information of small-size objects. Firstly, anchor-free YOLOv8 is adopted as the basic framework for detection to eliminate the affections of hyperparameters related to anchors, as well as to improve the detection capability of multi-scale and small-size defects. Secondly, considering the PAN path (top layer of neural networks for feature fusion) is more task-specific, we advocate that the underlying feature map of the PAN path is more vulnerable to small object detection. Hence, we in-depth study the PAN path and point out that the standard PAN will encounter several drawbacks caused by losing local details and position information in deep layer. As an alternative, we propose a lightweight and detail-sensitive PAN (DsPAN) for small object detection of multiscale defects by designing an attention mechanism embedded feature transformation module(LCBHAM) and optimizing the lightweight implementation. Our proposed DsPAN is very easy to be incorporated in YOLO series framework. The proposed method is evaluated on three public datasets, NEU-DET, PCB-DET, and GC10-DET. The mAP of the method is 80.4%, 95.8%, and 76.3%, which are 3.6%, 2.1%, and 3.9% higher than that of YOLOv8 and significantly higher than the state-of-the-art (SOTA) detection methods. Also, the method achieves the second-highest inference speed among the thirteen models tested. The results indicate that DsP-YOLO is expected to be used for real-time defect detection in industry.

CFRLA-Net: A Context-Aware Feature Representation Learning Anchor-Free Network for Pedestrian Detection

High resolution and strong semantic representation are both vital for feature extraction networks of pedestrian detection. The existing high-resolution network (HRNet) has presented a promising performance for pedestrian detection. However, we observed that it still has some significant shortcomings for heavily occluded and small-scale pedestrians. In this paper, we propose to address the shortcomings by extracting semantic and spatial context from HRNet. Specifically, we propose a Context-aware Feature Representation Learning Module (CFRL-Module), which combines a Multi-scale Feature Context Extraction Parallel Block for Convolution and Self-attention (CEPCA-Block) with two parallel paths and an Equivalent FFN (EFFN) Block. The core CEPCA-Block adopts a parallel design to integrate convolution and multi-head self-attention (MHSA) with low parameter computational cost, which can obtain the deep semantic context by convolution path and precise context by MHSA path. Furthermore, to overcome the inefficiency of global MHSA in high-resolution pedestrian detection, we propose a novel local window MHSA, which can significantly reduce memory consumption but barely affect the detection performance. Cascading the proposed CFRL-Module with the anchor-free detection head constitutes our Context-aware Feature Representation Learning Anchor-Free Network (CFRLA-Net). The proposed CFRLA-Net can catch a high-level understanding of the heavily occluded and small-scale pedestrian instances based on HRNet, which can effectively solve the limitation of the insufficient feature extraction ability of HRNet for the hard samples. Experimental results show that CFRLA-Net achieves state-of-the-art performance on CityPersons, Caltech, and CrowdHuman benchmarks.

Fabric defect detection based on anchor-free network

Fabrics play a pivotal role in human life and production, and surface defects can directly affect the quality and value of fabrics. Many methods for fabric defect detection have been proposed, but tiny defects are still difficult to be detected effectively, and the accuracy of defect localization and classification is low. To address these issues, a modified YOLOX network called YOLOX-CATD is proposed, which was supplemented with a coordinate attention module (CAM) and tiny defect detection layer (TDDL) for fast and efficient detection of fabric defects, especially tiny defects. Firstly, the anchor-free network is used as the detection framework to avoid the influence of hyperparameters of the setting anchor. Secondly, a CAM is proposed to enhance the representation of the object of interest in the input feature map and suppress the background regions. Finally, a TDDL is added to introduce high-resolution features to improve the localization accuracy of tiny defects. The experimental results on the Aliyun Tianchi Fabric dataset and NEU-DET demonstrate the superiority and generalization of the modified model. The mean average precision (mAP) of YOLOX-CATD on the fabric defect dataset is improved by 5.67% compared to the original YOLOX, and the detection speed can reach 35–36 frames per second (FPS). This proves that YOLOX-CATD can obtain excellent fabric defect detection performance and meet the urgent need for real-time detection in industrial applications.

In-situ enhanced anchor-free deep CNN framework for a high-speed human-machine interaction

Human-Robot Interaction (HRI) constitutes a demanding research field that integrates artificial intelligence, informatics, robotics, engineering, and human-machine interaction. In the present era, there is an increased focus on natural user interfaces, with particular attention to gestural modalities. This leads to the coordination of modern robotic systems, where real-time integration and interpretation of manual gestures play a vital role in facilitating effective manipulation. Hand gesture recognition holds a crucial role in the field of machine vision, perhaps greatly exacerbated with varying lighting conditions and backgrounds. In this paper, first an Enhanced Anchor-free Network (EAF-Net) is proposed to perform hand gesture recognition in real-time. The EAF-Net is deep single-stage Convolutional Neural Network (CNN) based architecture. The Custom Precise Prediction Function (CPPF) is utilized for a continuous recognition of a specific gesture from a video feed. Then, the 6-axis collaborative Robot is manipulated using the predicted gesture. The EAF-Net model utilizes the Enriched Hourglass as a backbone feature extraction network. The proposed EAF-Net model undergoes training and assessment using the MITI HD-II dataset. The performance of the model is also analyzed for the standard benchmark datasets (NUSHP-II and Senz-3D). The evaluation of the EAF-Net model’s performance spans a range of IoU values from 0.5 to 0.95. The EAF-Net model led to higher precision values (AP0.5) as 99.22%, recall values (AR0.5) as 98.20%, and F1-Score0.5 as 98.64% on the MITI HD-II dataset. The EAF-Net model consumed a prediction time of 259 ms on NVIDIA Jetson Nano Processor and 14 ms on NVIDIA Titan X GPU. It aided in implementing a portable high-speed 6-axis Robot control framework to pick and place operation in 3D space. The robot would be able to imitate human hand activities such as hand rotation, hand tilting, picking and arranging items, and so on in the near future. Further reduction in prediction time of the model on the portable processors (NVIDIA Jetson Nano, Raspberry-Pi) could enhance the speed of the Human Machine Interaction process.

ADFireNet: An Anchor-Free Smoke and Fire Detection Network Based on Deformable Convolution.

In this paper, we propose an anchor-free smoke and fire detection network, ADFireNet, based on deformable convolution. The proposed ADFireNet network is composed of three parts: The backbone network is responsible for feature extraction of input images, which is composed of ResNet added to deformable convolution. The neck network, which is responsible for multi-scale detection, is composed of the feature pyramid network. The head network outputs results and adopts pseudo intersection over union combined with anchor-free network structure. The head network consists of two full convolutional subnetworks: the first is the classification sub-network, which outputs a classification confidence score, and the second is the regression sub-network, which predicts the parameters of bounding boxes. The deformable convolution (DCN) added to the backbone network enhances the shape feature extraction capability for fire and smoke, and the pseudo intersection over union (pseudo-IoU) added to the head network solves the label assignment problem that exists in anchor-free object detection networks. The proposed ADFireNet is evaluated using the fire smoke dataset. The experimental results show that ADFireNet has higher accuracy and faster detection speeds compared with other methods. Ablation studies have demonstrated the effectiveness of DCN and pseudo IoU.

An Anchor-Free Network for Increasing Attention to Small Objects in High Resolution Remote Sensing Images

Aimed at the problems of small object detection in high resolution remote sensing images, such as difficult detection, diverse scales, and dense distribution, this study proposes a new method, DCE_YOLOX, which is more focused on small objects. The method uses depthwise separable deconvolution for upsampling, which can effectively recover lost feature information and combines dilated convolution and CoTNet to extract local contextual features, which can make full use of the hidden semantic information. At the same time, EcaNet is added to the enhanced feature extraction network of the baseline model to make the model more focused on information-rich features; secondly, the network input resolution is optimized, which can avoid the impact of image scaling to a certain extent and improve the accuracy of small object detection. Finally, CSL is used to calculate the angular loss to achieve the rotated object detection of remote sensing images. The proposed method in this study achieves 83.9% accuracy and 76.7% accuracy for horizontal object detection and rotationally invariant object detection, respectively, in the DOTA remote sensing dataset; it even achieves 96% accuracy for rotationally invariant object detection in the HRSC2016 dataset. It can be concluded that our algorithm has a better focus on small objects, while it has an equally good focus on other objects and is well suited for applications in remote sensing, and it has certain reference significance for realizing the detection of small objects in remote sensing images.

An Efficient RGB-D Hand Gesture Detection Framework for Dexterous Robot Hand-Arm Teleoperation System

Aiming at the problems of accurate and fast hand gesture detection and teleoperation mapping in the hand-based visual teleoperation of dexterous robots, an efficient hand gesture detection framework based on deep learning is proposed in this article. It can achieve an accurate and fast hand gesture detection and teleoperation of dexterous robots based on an anchor-free network architecture by using an RGB-D camera. First, an RGB-D early-fusion method based on the HSV space is proposed, effectively reducing background interference and enhancing hand information. Second, a hand gesture classification network (HandClasNet) is proposed to realize hand detection and localization by detecting the center and corner points of hands, and a HandClasNet is proposed to realize gesture recognition by using a parallel EfficientNet structure. Then, a dexterous robot hand-arm teleoperation system based on the hand gesture detection framework is designed to realize the hand-based teleoperation of a dexterous robot. Our method achieves high accuracy with fast speed on public and custom hand datasets and outperforms some state-of-the-art methods. In addition, the application of the proposed method in the hand-based teleoperation system can control the grasping of various objects by a dexterous hand-arm system in real time and accurately, which verifies the efficiency of our method.

Free$\rm ^{3}$Net: Gliding Free, Orientation Free, and Anchor Free Network for Oriented Object Detection

Object detection for aerial images has achieved remarkable progress in recent years. Nevertheless, most exiting studies do not differentiate oriented object detection from horizontal detection. Certain schemes ignore the ambiguity of oriented object representation and leverage label assignment designed for horizontal object detection directly. Consequently, it leads to unstable training and causes performance degradation, because high-quality samples surrounding the oriented bounding boxes can not be leveraged effectively. To address this problem, we propose a gliding Free, orientation Free, and anchor Free Network (Free <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\rm ^{3}$</tex-math></inline-formula> Net) with high-efficiency for oriented object detection. Specifically, we propose an unambiguous oriented object representation scheme, named FreeGliding, by gliding the projection points of samples on each edge of horizontal bounding boxes. It makes the detection largely free from representation ambiguity and multi-task dependency. To overcome the restrictions of label assignment, we put forward a novel Loss-aware Outer Sample Selection (LOSS) scheme, which takes into consideration spatial information and localization capability to retain high-quality samples surrounding the objects. Moreover, we introduce an Oriented Feature Fusion (OFF) scheme to tackle feature alignment by adjusting the receptive field and fusing oriented features dynamically. Experimental results on two large-scale remote sensing datasets HRSC2016 and DOTA demonstrate that Free <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\rm ^{3}$</tex-math></inline-formula> Net outperforms the state-of-the-art schemes with a large margin. We hope our work can inspire rethinking the design of anchor-free detectors, and serve as a strong baseline for oriented object detection.

Microarmature Solder Surface Detection: An Adaptive Central Region Sample Selection Anchor Free Framework

As a core procedure in an increasingly automated industrial environment, defect detection is important for producing micro armatures. However, there are still some difficulties in using general deep learning detection methods, the spatial feature loss problem for small targets of solder surface defects and the difficulty in tuning the reference of sample selection strategy. Therefore, this paper proposes an adaptive label assignment detection framework. The detector uses an anchor free object detection network as the backbone. A feature compensation method is proposed to complement the spatial information for solving the feature disappearance problem of small objects. For addressing the label matching tuning problem of anchor-free network, an adaptive central region sample selection strategy is proposed at the base of the feature selective anchor-free (FSAF) module. To further improve the performance of the detection head, the dynamic head and soft non-maximum suppression (Soft-NMS) are introduced. The detection framework in this paper is evaluated on the Micro-Armature Solder Surface Defect Detection (MASS-DET) dataset, and the experimental results show that our framework can achieve 87.7 % mAP50 and 65.5% mAR, making it superior to common object detection methods. Moreover, the method presented in this paper can be applied in most industrial environments to improve the accuracy and efficiency of defect detection of non-standard micro workpiece. The dataset of this paper is open source and available on https://github.com/scuzw/Micro-Armature-Solder-Surface-Defect-Detection.

Anchor-free Proposal Generation Network for Efficient Object Detection

Deep learning object detection methods are usually based on anchor-free or anchor-based scheme for extracting object proposals and one-stage or two-stage structure for producing final predictions. As each scheme or structure has its own strength and weakness, combining their strength in a unified framework is an interesting research topic. However, this topic has not attracted much attention in recent years. This paper presents a two-stage object detection method that utilizes an anchor-free scheme for generating object proposals in the initial stage. For proposal generation, this paper employs an efficient anchor-free network for predicting object corners and assigns object proposals based on detected corners. For object prediction, an efficient detection network is designed to enhance both detection accuracy and speed. The detection network includes a lightweight binary classification subnetwork for removing most false positive object candidates and a light-head detection subnetwork for generating final predictions. Experimental results on the MS-COCO dataset demonstrate that the proposed method outperforms both anchor-free and two-stage object detection baselines in terms of detection performance.

Bridge coating inspection based on two-stage automatic method and collision-tolerant unmanned aerial system

Regular inspection of antirust coating is a vital approach to prevent steel bridges from corrosion. However, most existing coating inspection methods rely on inefficient manual examination with non-destructive testing (NDT) instruments. This study proposes a collision-tolerant unmanned aerial system (UAS) and the corresponding two-stage examination approach for bridge coating inspection. The contributions are as follows: (1) The UAS for coating inspection is designed with unique benefits including collision tolerance, non-GPS positioning, and NDT-based contact inspection. (2) Aiming at the problem that the GPS-based positioning method that existing UASs used is invalid in the close-range inspection for steel bridges because the GPS signal is blocked near steel bridges, a pseudo GPS navigation method using ultra-wideband (UWB) beacon is adopted to provide positioning data for the UAS. (3) To detect surface coating defects and coating thickness information in the inspection, a two-stage coating inspection method combining vision-based detection and contact-based measurement is proposed, in which surface defects of the coating are detected using a lightweight anchor-free network in real-time and then coating thickness around the defected area is measured with NDT method based on magnetic eddy current. The proposed method can detect the surface defects including coating spalling, fading, and hollowing in stage 1, and measure the coating thickness in stage 2. Experiments on an in-service bridge show the measured coating thickness error between the proposed method and manual inspection is less than 5%, which verifies the practicability and high precision of the proposed method.

OAF-Net: An Occlusion-Aware Anchor-Free Network for Pedestrian Detection in a Crowd

Although pedestrian detection has achieved promising performance with the development of deep learning techniques, it remains a great challenge to detect heavily occluded pedestrians in crowd scenes. Therefore, to make the anchor-free network pay more attention to learning the hard examples of occluded pedestrians, we propose a simple but effective Occlusion-aware Anchor-Free Network (namely OAF-Net) for pedestrian detection in crowd scenes. Specifically, we first design a novel occlusion-aware detection head, which includes three separate center prediction branches combining with the scale and offset prediction branches. In the detection head of OAF-Net, occluded pedestrian instances are assigned to the most suitable center prediction branch according to the occlusion level of human body. To optimize the center prediction, we accordingly propose a novel weighted Focal Loss where pedestrian instances are assigned with different weights according to their visibility ratios, so that the occluded pedestrians are up-weighted during the training process. Our OAF-Net is able to model different occlusion levels of pedestrian instances effectively, and can be optimized towards catching a high-level understanding of the hard training samples of occluded pedestrians. Experiments on the challenging CityPersons, Caltech, and CrowdHuman benchmarks sufficiently validate the efficacy of our OAF-Net for pedestrian detection in crowd scenes.

Breast lesion detection using an anchor-free network from ultrasound images with segmentation-based enhancement

The survival rate of breast cancer patients is closely related to the pathological stage of cancer. The earlier the pathological stage, the higher the survival rate. Breast ultrasound is a commonly used breast cancer screening or diagnosis method, with simple operation, no ionizing radiation, and real-time imaging. However, ultrasound also has the disadvantages of high noise, strong artifacts, low contrast between tissue structures, which affect the effective screening of breast cancer. Therefore, we propose a deep learning based breast ultrasound detection system to assist doctors in the diagnosis of breast cancer. The system implements the automatic localization of breast cancer lesions and the diagnosis of benign and malignant lesions. The method consists of two steps: 1. Contrast enhancement of breast ultrasound images using segmentation-based enhancement methods. 2. An anchor-free network was used to detect and classify breast lesions. Our proposed method achieves a mean average precision (mAP) of 0.902 on the datasets used in our experiment. In detecting benign and malignant tumors, precision is 0.917 and 0.888, and recall is 0.980 and 0.963, respectively. Our proposed method outperforms other image enhancement methods and an anchor-based detection method. We propose a breast ultrasound image detection system for breast cancer detection. The system can locate and diagnose benign and malignant breast lesions. The test results on single dataset and mixed dataset show that the proposed method has good performance.

MA-CharNet: Multi-angle fusion character recognition network.

Irregular text recognition of natural scene is a challenging task due to large span of character angles and morphological diversity of a word. Recent work first rectifies curved word region, and then employ sequence algorithm to complete the recognition task. However, this strategy largely depends on rectification quality of the text region, and cannot be applied to large difference between tilt angles of character. In this work, a novel anchor-free network structure of rotating character detection is proposed, which includes multiple sub-angle domain branch networks, and the corresponding branch network can be selected adaptively according to character tilt angle. Meanwhile, a curvature Adaptive Text linking method is proposed to connect the discrete strings detected on the two-dimensional plane into words according to people’s habits. We achieved state-of-the-art performance on two irregular texts (TotalText, CTW1500), outperforming state-of-the-art by 2.4% and 2.7%, respectively. The experimental results demonstrate the effectiveness of the proposed algorithm.

An Anchor-Free 3D Object Detection Approach Based on Hierarchical Pillars

Object detection in the 2D domain is well developed owing to the wide application of CMOS image sensors and the great success of deep learning technologies in recent years. However, under circumstances such as autonomous driving, the variation of weather conditions and light conditions makes it impossible to perform reliable detection using regular 2D image sensors. 3D data generated by a Lidar or Radar is more robust to such environments, hence serving as an essential complement to 2D data in such scenarios. Well-established anchor-based detectors in the 2D domain suffer from time-consuming anchor configuration and cannot be exploited directly to process 3D data. This paper proposes an anchor-free network that encodes the raw point cloud into a hierarchical pillar representation to locate objects. Without predefined anchors and NMS postprocessing, our method directly predicts the center points and box properties to accomplish the detection task efficiently. In addition, a PCA-based initialization for the convolutional kernel is proposed to accelerate the training process. Experiments are implemented on the KITTI benchmark, and our method can achieve competitive performance with other anchor-based methods. Comprehensive ablation studies further verify the validity and rationality of each part of the proposed method.