Ground-truth Bounding Boxes Research Articles

An Automated Diagnosis Method for Lung Cancer Target Detection and Subtype Classification-Based CT Scans.

When dealing with small targets in lung cancer detection, the YOLO V8 algorithm may encounter false positives and misses. To address this issue, this study proposes an enhanced YOLO V8 detection model. The model integrates a large separable kernel attention mechanism into the C2f module to expand the information retrieval range, strengthens the extraction of lung cancer features in the Backbone section, and achieves effective interaction between multi-scale features in the Neck section, thereby enhancing feature representation and robustness. Additionally, depth-wise convolution and Coordinate Attention mechanisms are embedded in the Fast Spatial Pyramid Pooling module to reduce feature loss and improve detection accuracy. This study introduces a Minimum Point Distance-based IOU loss to enhance correlation between predicted and ground truth bounding boxes, improving adaptability and accuracy in small target detection. Experimental validation demonstrates that the improved network outperforms other mainstream detection networks in terms of average precision values and surpasses other classification networks in terms of accuracy. These findings validate the outstanding performance of the enhanced model in the localization and recognition aspects of lung cancer auxiliary diagnosis.

Boosting Performance on 3D Object Detection with a Plug-in Discrimination Module

Around-view multi-camera 3D object detection in BEV (Bird’s-Eye-View) space has been a research focus over the past few years. As a typical supervised training task, many researchers promote this area with different task-specific key designs, such as exploiting temporal information and correspondence of perspective image plane and BEV space. Most of these works follow the DETR detection framework, yet the nature of learnable queries in DETR, the encodings of objects’ center and bounding box information, have not been discussed in previous studies. In this paper, we take advantage of this prior and further extend it to 3D detection tasks. In 3D object detection, the ground-truth bounding boxes are hardly overlapping. Therefore, the queries should be more diverse under this hypothesis. To achieve this goal, we propose a Plug-in Discrimination Module (PDM) to discriminate learnable queries from all the other queries with a discrimination loss to ensure the diversity of queries. The PDM is a simple train-time-only module. It contains a query projection head to project all the object queries into a common latent space. In the latent space, the discrimination loss is conducted on all the queries. Experimental results show that this design can directly improve the 3D detector’s performance without modifying the detector’s architecture and adding extra inference costs. The NDS improvement on the nuScenes dataset is up to a maximum of 1.62% in the 8th training epoch and remains an average 0.64% improvement in the following epochs, compared with the baseline model.

RG-CAT: Detection pipeline and catalogue of radio galaxies in the EMU pilot survey

Abstract We present source detection and catalogue construction pipelines to build the first catalogue of radio galaxies from the 270 $\rm deg^2$ pilot survey of the Evolutionary Map of the Universe (EMU-PS) conducted with the Australian Square Kilometre Array Pathfinder (ASKAP) telescope. The detection pipeline uses Gal-DINO computer vision networks (Gupta et al. 2024, PASA, 41, e001) to predict the categories of radio morphology and bounding boxes for radio sources, as well as their potential infrared host positions. The Gal-DINO network is trained and evaluated on approximately 5 000 visually inspected radio galaxies and their infrared hosts, encompassing both compact and extended radio morphologies. We find that the Intersection over Union (IoU) for the predicted and ground-truth bounding boxes is larger than 0.5 for 99% of the radio sources, and 98% of predicted host positions are within $3^{\prime \prime}$ of the ground-truth infrared host in the evaluation set. The catalogue construction pipeline uses the predictions of the trained network on the radio and infrared image cutouts based on the catalogue of radio components identified using the Selavy source finder algorithm. Confidence scores of the predictions are then used to prioritise Selavy components with higher scores and incorporate them first into the catalogue. This results in identifications for a total of 211 625 radio sources, with 201 211 classified as compact and unresolved. The remaining 10 414 are categorised as extended radio morphologies, including 582 FR-I, 5 602 FR-II, 1 494 FR-x (uncertain whether FR-I or FR-II), 2 375 R (single-peak resolved) radio galaxies, and 361 with peculiar and other rare morphologies. Each source in the catalogue includes a confidence score. We cross-match the radio sources in the catalogue with the infrared and optical catalogues, finding infrared cross-matches for 73% and photometric redshifts for 36% of the radio galaxies. The EMU-PS catalogue and the detection pipelines presented here will be used towards constructing catalogues for the main EMU survey covering the full southern sky.

Fuzzy Attention U-Net Architecture Based Localization and YOLOv5 Detection for Fetal Cardiac Ultrasound Images

The importance of early identification of congenital heart disease is highlighted by the fact that it accounts for around 28% of all congenital abnormalities this is the primary reason why fetus die. The necessity of having a thorough understanding of normal cardiac architecture has been highlighted by the quick advancements in fetal heart imaging techniques that have occurred in recent years. Without this information, it is challenging, even impossible, to distinguish the many manifestations of congenital heart illness. This research suggests an immediate fetal cardiac identification technique employing US pictures with the You Only Look Once v5 (YOLOv5) framework and localization using Fuzzy Attention U-Net (FAU-Net) framework in order to enhance the interpretation of the anatomy of the fetal heart through Ultrasound (US) for precise and instantaneous diagnoses. Localization is accomplished using a FU-Net architecture, which limits the training set to image-level plane labels. This is a crucial component of the proposed study since, for big datasets, it would take too much time to produce bounding box annotations, which are not always captured. The FAU-Net design has been optimized for best performance. The YOLOv5 framework is built to function in real-time and deliver the best results for object identification. With the aid of appropriate fine-tuning, it may function effectively to automatically identify tiny fetal cardiac objects in a fast phase. Depending on how much of a detection overlapped with ground-truth bounding boxes, it was determined if it was a genuine positive or a false positive. This research primarily aids medical professionals in the diagnosis of fetal cardiac anatomy. The efficiency of the outcomes is evaluated using metrics including accuracy, memory, average precision (AP), mean average precision (mAP), and F-measure.

Development and validation of an artificial intelligence algorithm for detecting vocal cords in video laryngoscopy.

Airway procedures in life-threatening situations are vital for saving lives. Video laryngoscopy (VL) is commonly performed during endotracheal intubation (ETI) in the emergency department. Artificial intelligence (AI) is widely used in the medical field, particularly to detect anatomical structures. This study aimed to develop an AI algorithm that detects vocal cords from VL images acquired during emergent situations. This retrospective study used VL images acquired in the emergency department to facilitate the ETI. The vocal cord image was labeled with a ground-truth bounding box. The dataset was divided into training and validation datasets. The algorithm was developed from a training dataset using the YOLOv4 model. The performance of the algorithm was evaluated using a test set. The test set was further divided into specific environments during the ETI for clinical subgroup analysis. In total, 20,161 images from 84 patients were used in this study. A total of 10,287, 5766, and 4108 images were used for the model training, validation, and test sets, respectively. The developed algorithm achieved F1 score 0.906, sensitivity 0.963, and specificity 0.842 in the validation set. The performance in the test set was F1 score 0.808, sensitivity 0.823, and specificity 0.804. We developed and validated an AI algorithm to detect vocal cords in VL. This algorithm demonstrated a high performance. The algorithm can be used to determine the vocal cord to ensure safe ETI.

HLA-HOD: Joint High-Low Adaptation for Object Detection in Hazy Weather Conditions

Object detection remains challenging in hazy weather conditions due to the poor visibility of captured images. There are currently two types of detectors capable of adapting to varying weather conditions: (i) low-level adaptation methods that combine one detector with an additional dehazing network and (ii) high-level adaptation methods that explore various kinds of domain adaptation knowledge. However, neither of these approaches can achieve desirable performance due to their inherent limitations. We raise an intriguing question—if combining both low-level adaptation and high-level adaptation, can improve the generalization ability of a detector in hazy weather conditions? To answer it, we propose a Joint High-Low Adaptation Object Detection paradigm (HLA-HOD) in hazy weather conditions. By combining both low-level adaptation and high-level adaptation, HLA-HOD achieves superior performance on hazy images without requiring ground-truth bounding boxes or clean images. Extensive experiments demonstrate that our method outperforms state-of-the-art low-level and high-level adaptation methods by a large margin both quantitatively and qualitatively.

DeepWILD: Wildlife Identification, Localisation and estimation on camera trap videos using Deep learning

Videos and images from camera traps are more and more used by ecologists to estimate the population of species on a territory. It is a laborious work since experts have to analyse massive data sets manually. This takes also a lot of time to filter these videos when many of them do not contain animals or are with human presence. Fortunately, deep learning algorithms for object detection can help ecologists to identify multiple relevant species on their data and to estimate their population. In this study, we propose to go even further by using object detection model to detect, classify and count species on camera traps videos. To this end, we developed a 3-step process: (i) At the first stage, after splitting videos into images, we annotate images by associating bounding boxes to each label thanks to MegaDetector algorithm; (ii) then, we extend MegaDetector based on Faster R-CNN architecture with backbone Inception-ResNet-v2 in order to not only detect the 13 relevant classes but also to classify them; (iii) finally, we design a method to count individuals based on the maximum number of bounding boxes detected. This final stage of counting is evaluated in two different contexts: first including only detection results (i.e. comparing our predictions against the right number of individuals, no matter their true class), then an evolved version including both detection and classification results (i.e. comparing our predictions against the right number in the right class). The results obtained during the evaluation of our model on the test data set are: (i) 73,92% mAP for classification, (ii) 96,88% mAP for detection with a ratio Intersection-Over-Union (IoU) of 0.5 (overlapping ratio between groundtruth bounding box and the detected one), and (iii) 89,24% mAP for detection at IoU = 0.75. Highly represented classes, like humans, have highest values of mAP around 81% whereas less represented classes in the train data set, such as dogs, have lowest values of mAP around 66%. Regarding the proposed counting method, we predicted a count either exact or ± 1 unit for 87% with detection results and for 48% with detection and classification results of our test data set. Our model is also able to detect empty videos. To the best of our knowledge, this is the first study in France about the use of object detection model on a French national park to locate, identify and estimate the population of species from camera trap videos.

Real-Time Computer Vision for Tree Stem Detection and Tracking

Object detection and tracking are tasks that humans can perform effortlessly in most environments. Humans can readily recognize individual trees in forests and maintain unique identifiers during occlusion. For computers, on the other hand, this is a complex problem that decades of research have been dedicated to solving. This paper presents a computer vision approach to object detection and tracking tasks in forested environments. We use a state-of-the-art neural network-based detection algorithm to fit bounding boxes around individual tree stems and a simple, efficient, and deterministic multiple object tracking algorithm to maintain unique identities for stems through video frames. We trained the neural network object detector on approximately 3000 ground-truth bounding boxes of ponderosa pine trees. We show that tree stem detection can achieve an average precision of 87% using a Jaccard overlap index of 0.5. We also demonstrate the robustness of the tracking algorithm in occlusion and enter–exit–re-enter scenarios. The presented algorithms can perform object detection and tracking at 49 frames per second on a consumer-grade graphics processing unit.

Quantifying the Effects of Ground Truth Annotation Quality on Object Detection and Instance Segmentation Performance

Fully-supervised object detection and instance segmentation models have accomplished notable results on large-scale computer vision benchmark datasets. However, fully-supervised machine learning algorithms’ performances are immensely dependent on the quality of the training data. Preparing computer vision datasets for object detection and instance segmentation is a labor-intensive task requiring each instance in an image to be annotated. In practice, this often results in the quality of bounding box and polygon mask annotations being suboptimal. This paper quantifies empirically the ground truth annotation quality and COCO’s mean average precision (mAP) performance by introducing two separate noise measures, uniform and radial, into the ground truth bounding box and polygon mask annotations for the COCO and Cityscapes datasets. Mask-RCNN models are trained on various levels of noise measures to investigate the performance of each level of noise. The results showed degradation of mAP as the level of both noise measures increased. For object detection and instance segmentation respectively, using the highest level of noise measure resulted in a mAP degradation of 0.185 & 0.208 for uniform noise with reductions of 0.118 & 0.064 for radial noise on the COCO dataset. As for the Cityscapes datasets, reductions of mAP performance of 0.147 & 0.142 for uniform noise and 0.101 & 0.033 for radial noise were recorded. Furthermore, a decrease in average precision is seen across all classes, with the exception of the class motorcycle. The reductions between classes vary, indicating the effects of annotation uncertainty are class-dependent.

Dynamic Soft Label Assignment for Arbitrary-Oriented Ship Detection

Ship detection with several military and civilian applications has drawn considerable attention in recent years. In remote sensing images, ships have the characteristics of arbitrary orientation. Based on the characteristics, many arbitrary-oriented ship detectors have been proposed. Most of these detectors preset many horizontal or rotated anchors and determine the positive and negative samples based on the intersection over union (IoU) between the anchor and ground-truth bounding box, in what is called the label assignment process. However, IoU performance is limited as it can only reflect the quality of the anchor to a certain extent. In addition, the manually fixed IoU threshold to separate the positive and negative limits the flexibility of the method, as different ships may have different optimal thresholds. Moreover, the equally weighted training samples cause a misalignment between the classification and regression heads. Therefore, we propose a dynamic soft label assignment method for arbitrary-oriented ship detection. First, we design a novel anchor quality score function that takes into account both prior and prediction information of the anchor and enables the model to participate in the label assignment process. Second, we propose a dynamic anchor quality score threshold instead of a fixed IoU threshold for dividing positive and negative samples. Third, in contrast to assigning equal weights, we propose a soft label assignment strategy to weigh the training samples in the loss function. The proposed method offers superior detection performances for arbitrary-oriented ships with only one horizontal preset anchor. Experimental results on HRSC2016, FGSD, and ShipRSImageNet datasets demonstrate the effectiveness of our proposed dynamic soft label assignment for arbitrary-oriented ship detection.

YOLO-F: YOLO for Flame Detection

Flame detection is of great significance in a fire prevention system. YOLOv4 has poor real-time performance on flame detection caused by the complex structure and high parameter size. To address this problem, a novel flame detection framework, YOLO for flame (YOLO-F), is proposed in this paper. The backbone of YOLOv4 is simplified from the original 53 convolutional layers to 34 convolutional layers to reduce the number of parameters by simplifying the structure of the CSPBlock. Based on the FPN, an effective and light-weight feature pyramid architecture, namely FPNs-SE, is then proposed and the neck part of YOLOv4 is replaced by FPNs-SE to enhance the feature extraction ability of different scales. In addition, the CIoU loss in the YOLOv4 ignores the similarity measure of the area between the predicted bounding box and the ground-truth bounding box. An effective loss named ACIoU is proposed in this paper in order to handle the above issue and further improve the detection accuracy. The proposed methods are tested on FLAME dataset and network crawled dataset, respectively. The mAP, recall, and precision of YOLO-F are higher by 2.01%, 4.0%, 2.0% on average than those of YOLOv4. With input size of [Formula: see text] and on a single GTX 1660, the operating speed of our method can reach 24.53[Formula: see text]fps, which is improved by 38.04% compared with YOLOv4. The experimental results show that our method is more robust to the small flame and flame-like objects and can achieve the best balance of detection speed and accuracy. The code is made available at https://github.com/Windxy/YOLO-F .

AMA-Det: Enhancing Shared Head of One-Stage Object Detection With Adaptation, Merging, and Alignment

Feature adaptation(FA) and result alignment(RA) are critical issues in one-stage object detection, since FA amends feature misalignment problem by adapting sampling points to semantically significant locations, and RA corrects result misalignment problem by estimating the localization quality. For FA, the aligned and consistent prediction of sampling points is important. Previous studies directly inherit cascaded “proposal-refine” philosophy from multi-stage detectors and predict sampling points by approximating their minimal external rectangle to ground-truth bounding boxes. This manner generates poorly aligned and consistent sampling points and induces irrelevant features aggregated. Moreover, their sampling points for classification are conventionally generated through the localization branch, without feeding the corresponding features of the classification branch. For RA, previous studies have verified the superiority of utilizing the predicted edge distribution to estimate localization quality; however, their directly regressed distribution is incompatible with the cascaded regression framework. To solve these problems, we firstly propose a focused feature adaptation method by softening the supervision of the proposal points. This method can predict sampling points focused above the assigned objects with excellent alignment and consistency. Subsequently, inner-branch and cross-branch merging were investigated to promote feature sharing from the classification branch. Finally, cascaded distribution-guided result alignment is advanced and verified to predict accurate localization quality. After integrating our proposed adaptation, merging, and alignment, we created AMA-Det with an enhanced shared head, which impressively reaches 43.9 mAP with ResNet50 as the backbone. AMA-Det also achieves a 54.5 mAP by multi-scale testing on MSCOCO test-dev and outperforms all existing CNN-based one-stage counterparts.

Weakly Supervised Real-Time Object Detection Based on Salient Map Extraction and the Improved YOLOv5 Model

In order to improve the accuracy and processing speed of object detection in weakly supervised learning environment, a weakly supervised real-time object detection method based on saliency map extraction and improved YOLOv5 is proposed. For the case where only image-level annotations are available, class-specific saliency maps are generated from the backpropagation process using a VGG-16-based classification network. After obtaining the position information of the target in the image, the pseudobounding box of the target is generated, and the pseudobounding box is used as the ground-truth bounding box to optimize the real-time target detection network. An improved YOLOv5 model is proposed to transfer clear target features to deeper network layers by designing a jump connection operation, thereby solving the problem of feature ambiguity. At the same time, the convolutional attention mechanism module is introduced to solve the problem that the recognition accuracy is affected by invalid features. Experiments on the PASCAL VOC 2007+2012 datasets show that when only image-level annotations are available in the training data, the proposed method can effectively improve the processing speed and maintain a good target detection accuracy, realizing real-time object detection under weakly supervised conditions.

Automatic worm detection to solve overlapping problems using a convolutional neural network

The nematode Caenorhabditis elegans is a powerful experimental model to investigate vital functions of higher organisms. We recently established a novel method, named "pond assay for the sensory systems (PASS)”, that dramatically improves both the evaluation accuracy of sensory response of worms and the efficiency of experiments. This method uses many worms in numbers that are impractical to count manually. Although several automated detection systems have been introduced, detection of overlapped worms remains difficult. To overcome this problem, we developed an automated worm detection system based on a deep neural network (DNN). Our DNN was based on a “YOLOv4″ one-stage detector with one-class classification (OCC) and multi-class classification (MCC). The OCC defined a single class for worms, while the MCC defined four classes for the number of overlapped worms. For the training data, a total of 2000 model sub-images were prepared by manually drawing square worm bounding boxes from 150 images. To make simulated images, a total of 10–80 model images for each class were randomly selected and randomly placed on a simulated microscope field. A total of 19,000 training datasets and 1000 validation datasets with a ground-truth bounding-box were prepared. We evaluated detection accuracy using 150 images, which were different from the training data. Evaluation metrics were detection error, precision, recall, and average precision (AP). Precision values were 0.91 for both OCC and MCC. However, the recall value for MCC (= 0.93) was higher than that for OCC (= 0.79). The number of detection errors for OCC increased with increasing the ground truth; however, that for MCC was independent of the ground truth. AP values were 0.78 and 0.90 for the OCC and the MCC, respectively. Our worm detection system with MCC provided better detection accuracy for large numbers of worms with overlapping positions than that with the OCC.

Hierarchical complementary learning for weakly supervised object localization

Weakly supervised object localization (WSOL) is a challenging problem that aims to localize objects without ground-truth bounding boxes. A common approach is to train the model that generates a class activation map (CAM) to localize the discriminative features of the object. Unfortunately, the limitation of this method is that they detect just a part of the object and not the whole object. To solve this problem, previous works have removed some parts of the image (Zhang et al., 2018; Zhang et al., 2018; Singh and Lee, 2017; Choe and Shim, 2019) to force the model to detect the full object extent. However, these methods require one or many hyper-parameters to erase the appropriate pixels on the image, which could involve a loss of information. In this paper, we propose a Hierarchical Complementary Learning Network method (HCLNet) that helps the CNN to perform better on classification and localization. HCLNet uses a complementary CAM to generate multiple maps that detect different parts of the object. Unlike previous works, this method does not need any extra hyper-parameters, as well as does not introduce a big loss of information. In order to fuse these different maps, two different fusion strategies known as the addition strategy and the l1-norm strategy have been used. These strategies allow to detect the whole object while excluding the background. Extensive experiments show that HCLNet obtains better performance than state-of-the-art methods.

SKNet: Detecting Rotated Ships as Keypoints in Optical Remote Sensing Images

Detecting rotated ships is difficult in optical remote sensing images due to the challenges of complex scenes. Existing advanced rotated ship detectors are typically anchor-based algorithms that require plenty of predefined anchors. However, the use of anchors brings three critical problems: 1) a large number of anchors bring a huge amount of calculation; 2) the attributes (e.g., size and aspect ratios) of anchors are designed via <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">ad hoc</i> heuristics; and 3) only a tiny fraction of anchors that overlap with ground-truth bounding boxes of ships tightly can be considered as positive samples, which causes an extreme imbalance between positive and negative samples. As a result, the detection accuracy will be influenced seriously when the design of anchors is not suitable. To address the above problems, this article proposes a novel anchor-free rotated ship detection framework, called SKNet, which detects rotated ships as keypoints in optical remote sensing images. In SKNet, a ship target is modeled as its center keypoint and morphological sizes, including the width, height, and rotation angle. Accordingly, we design two customized modules: orthogonal pooling and soft-rotate-nonmaximum suppression (NMS), where the former is to improve the prediction accuracy of the center keypoint and the morphological size, and the latter is to effectively remove redundant rotated ship detection results. Extensive experiments are conducted to demonstrate the effectiveness of SKNet on three optical remote sensing image data sets: HRSC2016, DOTA-ship, and HPDM-OSOD, which is collected by ourselves and published in this article. Empirical studies show that SKNet achieves state-of-the-art detection performance while being time-efficient. Overall, SKNet achieves the best speed–accuracy tradeoff.

Deep Graph Metric Learning for Weakly Supervised Person Re-Identification.

In conventional person re-identification (re-id), the images used for model training in the training probe set and training gallery set are all assumed to be instance-level samples that are manually labeled from raw surveillance video (likely with the assistance of detection) in a frame-by-frame manner. This labeling across multiple non-overlapping camera views from raw video surveillance is expensive and time consuming. To overcome these issues, we consider a weakly supervised person re-id modeling that aims to find the raw video clips where a given target person appears. In our weakly supervised setting, during training, given a sample of a person captured in one camera view, our weakly supervised approach aims to train a re-id model without further instance-level labeling for this person in another camera view. The weak setting refers to matching a target person with an untrimmed gallery video where we only know that the identity appears in the video without the requirement of annotating the identity in any frame of the video during the training procedure. The weakly supervised person re-id is challenging since it not only suffers from the difficulties occurring in conventional person re-id (e.g., visual ambiguity and appearance variations caused by occlusions, pose variations, background clutter, etc.), but more importantly, is also challenged by weakly supervised information because the instance-level labels and the ground-truth locations for person instances (i.e., the ground-truth bounding boxes of person instances) are absent. To solve the weakly supervised person re-id problem, we develop deep graph metric learning (DGML). On the one hand, DGML measures the consistency between intra-video spatial graphs of consecutive frames, where the spatial graph captures neighborhood relationship about the detected person instances in each frame. On the other hand, DGML distinguishes the inter-video spatial graphs captured from different camera views at different sites simultaneously. To further explicitly embed weak supervision into the DGML and solve the weakly supervised person re-id problem, we introduce weakly supervised regularization (WSR), which utilizes multiple weak video-level labels to learn discriminative features by means of a weak identity loss and a cross-video alignment loss. We conduct extensive experiments to demonstrate the feasibility of the weakly supervised person re-id approach and its special cases (e.g., its bag-to-bag extension) and show that the proposed DGML is effective.

Explainable Models with Consistent Interpretations

Given the widespread deployment of black box deep neural networks in computer vision applications, the interpretability aspect of these black box systems has recently gained traction. Various methods have been proposed to explain the results of such deep neural networks. However, some recent works have shown that such explanation methods are biased and do not produce consistent interpretations. Hence, rather than introducing a novel explanation method, we learn models that are encouraged to be interpretable given an explanation method. We use Grad-CAM as the explanation algorithm and encourage the network to learn consistent interpretations along with maximizing the log-likelihood of the correct class. We show that our method outperforms the baseline on the pointing game evaluation on ImageNet and MS-COCO datasets respectively. We also introduce new evaluation metrics that penalize the saliency map if it lies outside the ground truth bounding box or segmentation mask, and show that our method outperforms the baseline on these metrics as well. Moreover, our model trained with interpretation consistency generalizes to other explanation algorithms on all the evaluation metrics. The code and models are publicly available.

An anchor-free object detector with novel corner matching method

Recently, remarkable object detection performance has been made by keypoint-based detectors with different keypoint matching strategies. However, the factors considered in most of keypoint matching strategies are not comprehensive, which greatly affects the detection results. In this paper, based on CornerNet we propose a novel anchor-free detector with a brand new keypoint matching method. In our approach, instead of using embedding of each corner to achieve keypoint matching like CornerNet, we predict a matching degree score for each predicted bounding box formed by corners. In order to train model to get accurate matching degree prediction results, we fully consider the location and geometric information of targets, calculating the distance between centers of predicted and ground truth bounding boxes as well as IoU value to decide which pair of top-left and bottom-right corners can form the right bounding box. Besides that, to relieve the problem that categorical predictions associated with corners are not always reliable, we set an additional classification score for each predicted bounding boxes to further promote final detection results. We apply our model on MS-COCO test−dev set for evaluation and the result shows that our method achieves an AP of 46.3 % at single-scale and 47.9% at multi-scale, which are competitive with other state-of-the-art models such as CentripetalNet and Corner Proposal Net (CPN).

Classify and Localize Threat Items in X-Ray Imagery With Multiple Attention Mechanism and High-Resolution and High-Semantic Features

No doubt incorporating deep learning in security public transportation is an attractive and challenging research direction. In this article, we incorporate multiple attention mechanisms with high-resolution and high-semantic features into neural networks to further classify and localize threat items automatically. We propose our efficient model located in different network layers. To be specific, our model is involved in three modules called the spatial attention module (SAM), the channel attention module (CAM), and the high-resolution and high-semantic module (HRHSM). For shallow and deep layers, we adapt SAM and CAM to mine the long-range spatial and channel information of the network, respectively. With regard to the last few layers with different sizes, we leverage HRHSM to fuse their features so that the output contains high-resolution and high-semantic features. Due to the generalizability, our model can be broadly applied to many commonly used networks. Experimental results on the SIXray dataset that is consisted of six classes of threat item in X-ray images, CIFAR-10, and CIFAR-100 datasets demonstrate that our modules can improve the performance of networks in image classification and object localization without ground-truth bounding boxes.