Modern Object Detection Research Articles

YOLO-CIANNA: Galaxy detection with deep learning in radio data

Context. The upcoming Square Kilometer Array (SKA) will set a new standard regarding data volume generated by an astronomical instrument, which is likely to challenge widely adopted data-analysis tools that scale inadequately with the data size. Aims. The aim of this study is to develop a new source detection and characterization method for massive radio astronomical datasets based on modern deep-learning object detection techniques. For this, we seek to identify the specific strengths and weaknesses of this type of approach when applied to astronomical data. Methods. We introduce YOLO-CIANNA, a highly customized deep-learning object detector designed specifically for astronomical datasets. In this paper, we present the method and describe all the elements introduced to address the specific challenges of radio astronomical images. We then demonstrate the capabilities of this method by applying it to simulated 2D continuum images from the SKA observatory Science Data Challenge 1 (SDC1) dataset. Results. Using the SDC1 metric, we improve the challenge-winning score by +139% and the score of the only other post-challenge participation by +61%. Our catalog has a detection purity of 94% while detecting 40–60% more sources than previous top-score results, and exhibits strong characterization accuracy. The trained model can also be forced to reach 99% purity in post-process and still detect 10–30% more sources than the other top-score methods. It is also computationally efficient, with a peak prediction speed of 500 images of 512×512 pixels per second on a single GPU. Conclusions. YOLO-CIANNA achieves state-of-the-art detection and characterization results on the simulated SDC1 dataset and is expected to transfer well to observational data from SKA precursors.

Pseudo-unknown uncertainty learning for open set object detection

Despite the significant strides made by modern object detectors in the closed-set scenarios, open-set object detection (OSOD) remains a formidable challenge. This is particularly evident in misclassifying objects from unknown categories into pre-existing known classes or ignored background classes. A novel approach called PUDet (Pseudo-unknown Uncertainty Detector) based on Evidential Deep Learning (EDL) is proposed, incorporating two modules: the Class-wise Contrastive Learning Network (CCL) and the Uncertainty-Aware Labeling Network (UAL). For CCL, the module leverages class-wise contrastive learning to encourage intra-class compactness and inter-class separation, thereby reducing the overlap between known and unknown classes. Simultaneously, it establishes compact boundaries for known classes and generates pseudo-unknown candidates to facilitate UAL for better learning pseudo-unknown uncertainty. For UAL, the Weight-Impact EDL (WI-EDL) approach is introduced to enhance uncertainty in edge samples by collecting categorical evidence and weight impact. Subsequently, UAL refines uncertainty via localization quality calibration, facilitating the mining of pseudo-unknown samples from foreground and background proposals to construct compact boundaries between known and unknown categories. In comparison to the state of the arts, the proposed PUDet showcases a substantial improvement, achieving a reduction in Absolute Open-Set Errors by 13%–16% across six OSOD benchmarks.

Transfer Learning for Abnormal Object Detection

In today's world, smart surveillance plays an important role in protecting security and creating a safe living environment. For abnormal objects in the smart surveillance system, this is an important issue, requiring attention and timely response from managers and supervisors. To address this issue, the paper uses transfer learning techniques on modern object detection models to detect abnormal objects such as guns, knives, etc. in public places. We experimented with the transfer learning method on the DETR model with a small dataset, and the model results showed a fairly fast convergence speed. Through this method, we hope to help reduce the burden of public security monitoring and warning work for managers, while technicians can use transfer learning techniques that are deployed in practice.

High Throughput FPGA-Based Object Detection via Algorithm-Hardware Co-Design

Object detection and classification is a key task in many computer vision applications such as smart surveillance and autonomous vehicles. Recent advances in deep learning have significantly improved the quality of results achieved by these systems, making them more accurate and reliable in complex environments. Modern object detection systems make use of lightweight convolutional neural networks (CNNs) for feature extraction, coupled with single-shot multi-box detectors (SSDs) that generate bounding boxes around the identified objects along with their classification confidence scores. Subsequently, a non-maximum suppression (NMS) module removes any redundant detection boxes from the final output. Typical NMS algorithms must wait for all box predictions to be generated by the SSD-based feature extractor before processing them. This sequential dependency between box predictions and NMS results in a significant latency overhead and degrades the overall system throughput, even if a high-performance CNN accelerator is used for the SSD feature extraction component. In this paper, we present a novel pipelined NMS algorithm that eliminates this sequential dependency and associated NMS latency overhead. We then use our novel NMS algorithm to implement an end-to-end fully pipelined FPGA system for low-latency SSD-MobileNet-V1 object detection. Our system, implemented on an Intel Stratix 10 FPGA, runs at 400 MHz and achieves a throughput of 2,167 frames per second with an end-to-end batch-1 latency of 2.13 ms. Our system achieves 5.3× higher throughput and 5× lower latency compared to the best prior FPGA-based solution with comparable accuracy.

A Comparative Analysis of Modern Object Detection Algorithms: YOLO vs. SSD vs. Faster R-CNN

In recent years, object detection has become a crucial component in various computer vision applications, including autonomous driving, surveillance, and image recognition. This study provides a comprehensive comparative analysis of three prominent object detection algorithms: You Only Look Once (YOLO), Single Shot MultiBox Detector (SSD), and Faster Region-Based Convolutional Neural Networks (Faster R-CNN). The background of this research lies in the growing need for efficient and accurate object detection methods that can operate in real-time. YOLO is known for its speed, SSD for its balance between speed and accuracy, and Faster R-CNN for its high detection accuracy, albeit at a slower pace. The methodology involves implementing these algorithms on a standardized dataset and evaluating their performance based on various metrics, including detection accuracy, processing speed, and computational resource requirements. Each algorithm is tested under similar conditions to ensure a fair comparison. The results indicate that while YOLO excels in real-time applications due to its high speed, SSD offers a middle ground with respectable accuracy and speed, making it suitable for applications requiring a balance of both. Faster R-CNN demonstrates superior accuracy, making it ideal for scenarios where detection precision is paramount, despite its slower performance. This comparative analysis highlights the strengths and weaknesses of each algorithm, providing valuable insights for researchers and practitioners in selecting the appropriate object detection method for their specific needs.

YOLO: A Competitive Analysis of Modern Object Detection Algorithms for Road Defects Detection Using Drone Images

Efficient identification of road defects is a critical concern for road safety and infrastructure upkeep. This research employs drone-captured imagery and advanced object detection algorithms to expedite defect recognition, with a specific focus on determining the optimal algorithm for prompt and precise detection. The importance of timely road defect detection, crucial for mitigating potential hazards, remains central. A comprehensive comparative analysis of contemporary object detection algorithms, encompassing YOLOv5s, YOLOv5m, YOLOv5l, YOLOv5x, and YOLOv7. The results of this study highlight YOLOv7 as the most efficient, with a notable mAP of 68.3%, closely followed by YOLOv5l (66.8%), YOLOv5m (66.3%), YOLOv5x (66%), and YOLOv5s (63%). The integration of drone-derived imagery, capturing distinct gradients, significantly enhances defect detection accuracy. Beyond road safety, this study offers valuable insights to computer vision and machine learning practitioners. By bridging technological innovation with practical implementation, it holds potential to advance road safety and transportation infrastructure quality and the use of revolutionary drone technology.

One-Shot Replay: Boosting Incremental Object Detection via Retrospecting One Object

Modern object detectors are ill-equipped to incrementally learn new emerging object classes over time due to the well-known phenomenon of catastrophic forgetting. Due to data privacy or limited storage, few or no images of the old data can be stored for replay. In this paper, we design a novel One-Shot Replay (OSR) method for incremental object detection, which is an augmentation-based method. Rather than storing original images, only one object-level sample for each old class is stored to reduce memory usage significantly, and we find that copy-paste is a harmonious way to replay for incremental object detection. In the incremental learning procedure, diverse augmented samples with co-occurrence of old and new objects to existing training data are generated. To introduce more variants for objects of old classes, we propose two augmentation modules. The object augmentation module aims to enhance the ability of the detector to perceive potential unknown objects. The feature augmentation module explores the relations between old and new classes and augments the feature space via analogy. Extensive experimental results on VOC2007 and COCO demonstrate that OSR can outperform the state-of-the-art incremental object detection methods without using extra wild data.

Language-Assisted 3D Feature Learning for Semantic Scene Understanding

Learning descriptive 3D features is crucial for understanding 3D scenes with diverse objects and complex structures. However, it is usually unknown whether important geometric attributes and scene context obtain enough emphasis in an end-to-end trained 3D scene understanding network. To guide 3D feature learning toward important geometric attributes and scene context, we explore the help of textual scene descriptions. Given some free-form descriptions paired with 3D scenes, we extract the knowledge regarding the object relationships and object attributes. We then inject the knowledge to 3D feature learning through three classification-based auxiliary tasks. This language-assisted training can be combined with modern object detection and instance segmentation methods to promote 3D semantic scene understanding, especially in a label-deficient regime. Moreover, the 3D feature learned with language assistance is better aligned with the language features, which can benefit various 3D-language multimodal tasks. Experiments on several benchmarks of 3D-only and 3D-language tasks demonstrate the effectiveness of our language-assisted 3D feature learning. Code is available at https://github.com/Asterisci/Language-Assisted-3D.

Towards fully synthetic training of 3D indoor object detectors: Ablation study

Object detection is a fundamental function of many intelligent autonomous systems such as service robots or autonomous vehicles. State-of-the-art 3D object detection methods are deep learning methods which require large amounts of annotated data for successful training. Acquiring and labeling real-world data is an expensive and time-consuming task. An alternative is to train object detectors using synthetically generated images. Since modern object detectors trained on synthetic data still do not achieve similar results to those trained on real data, generating realistic synthetic data is still an open issue. We address the problem of object detection in indoor environments based on 3D point clouds acquired by 3D cameras. We provide a detailed insight into the importance of five factors involved in generating synthetic data: camera noise, presence of background objects, positioning of objects, scene context, and object sizes. To investigate the importance of these factors, we have developed a fully modular method for generating realistic synthetic single-view point clouds for training object detectors. Our method can generate large amounts of customizable data in a short time. An interesting finding is that pre-training with our data and fine-tuning with real data improves the performance of 3D object detectors, enabling one of them to achieve state-of-the-art results on one of the benchmarks without the use of RGB data.

Transformer-Based Visual Grounding with Cross-Modality Interaction

This article tackles the challenging yet important task of Visual Grounding (VG), which aims to localize a visual region in the given image referred by a natural language query. Existing efforts on the VG task are twofold: (1) two-stage methods first extract region proposals and then rank them according to their similarities with the referring expression, which usually leads to suboptimal results due to the quality of region proposals; (2) one-stage methods usually predict all the possible coordinates of the target region online by leveraging modern object detection architectures, which pay little attention to cross-modality correlations and have limited generalization ability. To better address the task, we present an effective transformer-based end-to-end visual grounding approach, which focuses on capturing the cross-modality correlations between the referring expression and visual regions for accurately reasoning the location of the target region. Specifically, our model consists of a feature encoder, a cross-modality interactor, and a modality-agnostic decoder. The feature encoder is employed to capture the intra-modality correlation, which models the linguistic context in query and the spatial dependency in image respectively. The cross-modality interactor endows the model with the capability of highlighting the localization-relevant visual and textual cues by mutual verification of vision and language, which plays a key role in our model. The decoder learns a consolidated token representation enriched by multi-modal contexts and further directly predicts the box coordinates. Extensive experiments on five public benchmark datasets with quantitative and qualitative analysis clearly demonstrate the effectiveness and rationale of our proposed method.

To make yourself invisible with Adversarial Semantic Contours

Modern object detectors are vulnerable to adversarial examples, which may bring risks to real-world applications. The sparse attack is an important task which, compared with the popular adversarial perturbation on the whole image, needs to select the potential pixels that is generally regularized by an $\ell_0$-norm constraint, and simultaneously optimize the corresponding texture. The non-differentiability of $\ell_0$ norm brings challenges and many works on attacking object detection adopted manually-designed patterns to address them, which are meaningless and independent of objects, and therefore lead to relatively poor attack performance. In this paper, we propose Adversarial Semantic Contour (ASC), an MAP estimate of a Bayesian formulation of sparse attack with a deceived prior of object contour. The object contour prior effectively reduces the search space of pixel selection and improves the attack by introducing more semantic bias. Extensive experiments demonstrate that ASC can corrupt the prediction of 9 modern detectors with different architectures (\e.g., one-stage, two-stage and Transformer) by modifying fewer than 5\% of the pixels of the object area in COCO in white-box scenario and around 10\% of those in black-box scenario. We further extend the attack to datasets for autonomous driving systems to verify the effectiveness. We conclude with cautions about contour being the common weakness of object detectors with various architecture and the care needed in applying them in safety-sensitive scenarios.

Виявлення об’єктів в процесі розпізнавання зображень за допомогою трансформерів

Modern object detection methods in the image recognition process us-ing transformer technology are analyzed.The various methods advantages and disadvantages are identified. An own network was created based on the DETR transformer from the FAIR team, and its operation was analyzed. A comparison of the transformer networks perfor-mance with optimized architectures of convolutional neural networks is made.The cloud computing tools, graphics processors, Internet of Things clusters or embedded microprocessor systems were used in the research process.To ensure high object detector accuracy and real-time detection results on different types of devices, an efficient object detector and model scaling technique are required.The transformer model learning is illustrated step-by-step process.

NextDet: Efficient Sparse-to-Dense Object Detection with Attentive Feature Aggregation

Object detection is a computer vision task of detecting instances of objects of a certain class, identifying types of objects, determining its location, and accurately labelling them in an input image or a video. The scope of the work presented within this paper proposes a modern object detection network called NextDet to efficiently detect objects of multiple classes which utilizes CondenseNeXt, an award-winning lightweight image classification convolutional neural network algorithm with reduced number of FLOPs and parameters as the backbone, to efficiently extract and aggregate image features at different granularities in addition to other novel and modified strategies such as attentive feature aggregation in the head, to perform object detection and draw bounding boxes around the detected objects. Extensive experiments and ablation tests, as outlined in this paper, are performed on Argoverse-HD and COCO datasets, which provide numerous temporarily sparse to dense annotated images, demonstrate that the proposed object detection algorithm with CondenseNeXt as the backbone result in an increase in mean Average Precision (mAP) performance and interpretability on Argoverse-HD’s monocular ego-vehicle camera captured scenarios by up to 17.39% as well as COCO’s large set of images of everyday scenes of real-world common objects by up to 14.62%.

A Task Decomposing and Cell Comparing Method for Cervical Lesion Cell Detection.

Automatic detection of cervical lesion cells or cell clumps using cervical cytology images is critical to computer-aided diagnosis (CAD) for accurate, objective, and efficient cervical cancer screening. Recently, many methods based on modern object detectors were proposed and showed great potential for automatic cervical lesion detection. Although effective, several issues still hinder further performance improvement of such known methods, such as large appearance variances between single-cell and multi-cell lesion regions, neglecting normal cells, and visual similarity among abnormal cells. To tackle these issues, we propose a new task decomposing and cell comparing network, called TDCC-Net, for cervical lesion cell detection. Specifically, our task decomposing scheme decomposes the original detection task into two subtasks and models them separately, which aims to learn more efficient and useful feature representations for specific cell structures and then improve the detection performance of the original task. Our cell comparing scheme imitates clinical diagnosis of experts and performs cell comparison with a dynamic comparing module (normal-abnormal cells comparing) and an instance contrastive loss (abnormal-abnormal cells comparing). Comprehensive experiments on a large cervical cytology image dataset confirm the superiority of our method over state-of-the-art methods.

Automated Detection of Atypical Aviation Obstacles from UAV Images Using a YOLO Algorithm.

Unmanned Aerial Vehicles (UAVs) are able to guarantee very high spatial and temporal resolution and up-to-date information in order to ensure safety in the direct vicinity of the airport. The current dynamic growth of investment areas in large agglomerations, especially in the neighbourhood of airports, leads to the emergence of objects that may constitute a threat for air traffic. In order to ensure that the obtained spatial data are accurate, it is necessary to understand the detection of atypical aviation obstacles by means of their identification and classification. Quite often, a common feature of atypical aviation obstacles is their elongated shape and irregular cross-section. These factors pose a challenge for modern object detection techniques when the processes used to determine their height are automated. This paper analyses the possibilities for the automated detection of atypical aviation obstacles based on the YOLO algorithm and presents an analysis of the accuracy of the determination of their height based on data obtained from UAV.

Dynamic Label Assignment for Object Detection by Combining Predicted IoUs and Anchor IoUs.

Label assignment plays a significant role in modern object detection models. Detection models may yield totally different performances with different label assignment strategies. For anchor-based detection models, the IoU (Intersection over Union) threshold between the anchors and their corresponding ground truth bounding boxes is the key element since the positive samples and negative samples are divided by the IoU threshold. Early object detectors simply utilize the fixed threshold for all training samples, while recent detection algorithms focus on adaptive thresholds based on the distribution of the IoUs to the ground truth boxes. In this paper, we introduce a simple while effective approach to perform label assignment dynamically based on the training status with predictions. By introducing the predictions in label assignment, more high-quality samples with higher IoUs to the ground truth objects are selected as the positive samples, which could reduce the discrepancy between the classification scores and the IoU scores, and generate more high-quality boundary boxes. Our approach shows improvements in the performance of the detection models with the adaptive label assignment algorithm and lower bounding box losses for those positive samples, indicating more samples with higher-quality predicted boxes are selected as positives.

Why Accuracy is Not Enough: The Need for Consistency in Object Detection

Object detectors are vital to many modern computer vision applications. However, even state-of-the-art object detectors are not perfect. On two images that look similar to human eyes, the same detector can make different predictions because of small image distortions like camera sensor noise and lighting changes. This problem is called inconsistency. Existing accuracy metrics do not properly account for inconsistency, and similar work in this area only targets improvements on artificial image distortions. Therefore, we propose a method to use nonartificial video frames to measure object detection consistency over time, across frames. Using this method, we show that the consistency of modern object detectors ranges from 83.2% to 97.1% on different video datasets from the multiple object tracking challenge. We conclude by showing that applying image distortion corrections such as WEBP Image Compression and Unsharp Masking can improve consistency by as much as 5.1%, with no loss in accuracy.

Meta Faster R-CNN: Towards Accurate Few-Shot Object Detection with Attentive Feature Alignment

Few-shot object detection (FSOD) aims to detect objects using only a few examples. How to adapt state-of-the-art object detectors to the few-shot domain remains challenging. Object proposal is a key ingredient in modern object detectors. However, the quality of proposals generated for few-shot classes using existing methods is far worse than that of many-shot classes, e.g., missing boxes for few-shot classes due to misclassification or inaccurate spatial locations with respect to true objects. To address the noisy proposal problem, we propose a novel meta-learning based FSOD model by jointly optimizing the few-shot proposal generation and fine-grained few-shot proposal classification. To improve proposal generation for few-shot classes, we propose to learn a lightweight metric-learning based prototype matching network, instead of the conventional simple linear object/nonobject classifier, e.g., used in RPN. Our non-linear classifier with the feature fusion network could improve the discriminative prototype matching and the proposal recall for few-shot classes. To improve the fine-grained few-shot proposal classification, we propose a novel attentive feature alignment method to address the spatial misalignment between the noisy proposals and few-shot classes, thus improving the performance of few-shot object detection. Meanwhile we learn a separate Faster R-CNN detection head for many-shot base classes and show strong performance of maintaining base-classes knowledge. Our model achieves state-of-the-art performance on multiple FSOD benchmarks over most of the shots and metrics.

Orientation aware weapons detection in visual data: a benchmark dataset

Automatic detection of weapons is significant for improving security and well being of individuals, nonetheless, it is a difficult task due to large variety of size, shape and appearance of weapons. View point variations and occlusion also are reasons which makes this task more difficult. Further, the current object detection algorithms process rectangular areas, however a slender and long rifle may really cover just a little portion of area and the rest may contain unessential details. To overcome these problem, we propose a CNN architecture for Orientation Aware Weapons Detection, which provides oriented bounding box with improved weapons detection performance. The proposed model provides orientation not only using angle as classification problem by dividing angle into eight classes but also angle as regression problem. For training our model for weapon detection a new dataset comprising of total 7801 weapons images is gathered from the web and then manually annotated with position oriented bounding boxes. Our dataset provides not only oriented bounding box as ground truth but also horizontal bounding box. We also provide our dataset in multiple formats of modern object detectors for further research in this area. The proposed model is evaluated on this dataset, and the comparative analysis with off-the shelf object detectors yields superior performance of proposed model, measured with standard evaluation strategies. The dataset and the model implementation are made publicly available at this link: https://bit.ly/2TyZICF .

Multi-View correlation distillation for incremental object detection

In real applications, new object classes often emerge after the detection model has been trained on a prepared dataset with fixed classes. Fine-tuning the old model with only new data will lead to a well-known phenomenon of catastrophic forgetting, which severely degrades the performance of modern object detectors. Due to the storage burden, data privacy and time consumption, sometimes it is impractical to train the model from scratch with all data of both old and new classes. In this paper, we propose a novel Multi-View Correlation Distillation (MVCD) based incremental object detection method, which explores the intra-feature correlations in the feature space of the object detector. To better transfer the knowledge learned from the old classes and maintain the ability to learn new classes, we select the sample-specific discriminative features from channel-wise, point-wise and instance-wise views. Meanwhile, the correlation distillation losses on the selective features are designed to regularize the learning of the incremental object detector. A new metric named Stability-Plasticity-mAP (SPmAP) is proposed to evaluate the incremental learning performance as a complementary metric to mAP, which integrates the metrics for the stability on old classes and the plasticity on new classes in incremental object detection. The extensive experiments conducted on VOC2007 and COCO demonstrate that MVCD achieves a better trade-off between stability and plasticity than state-of-the-art first-order distillation-based incremental object detection methods.