Object Detection Method Research Articles

Accurate crowd counting for intelligent video surveillance systems

The paper presents a novel deep learning approach for crowd counting in intelligent video surveillance systems, addressing the growing need for accurate monitoring of public spaces in urban environments. The demand for precise crowd estimation arises from challenges related to security, public safety, and efficiency in urban areas, particularly during large public events. Existing crowd counting techniques, including feature-based object detection and regression-based methods, face limitations in high-density environments due to occlusions, lighting variations, and diverse human figures. To overcome these challenges, the authors propose a new deep encoder-decoder architecture based on VGG16, which incorporates hierarchical feature extraction with spatial and channel attention mechanisms. This architecture enhances the model’s ability to manage variations in crowd density, leveraging adaptive pooling and dilated convolutions to extract meaningful features from dense crowds. The model’s decoder is further refined to handle sparse and crowded scenes through separate density maps, improving its adaptability and accuracy. Evaluations of the proposed model on benchmark datasets, including Shanghai Tech and UCF CC 50, demonstrate superior performance over state-of-the-art methods, with significant improvements in mean absolute error and mean squared error metrics. The paper emphasizes the importance of addressing environmental variability and scale differences in crowded environments and shows that the proposed model is effective in both sparse and dense crowd conditions. This research contributes to the advancement of intelligent video surveillance systems by providing a more accurate and efficient method for crowd counting, with potential applications in public safety, transportation management, and urban planning.

Depression detection based on the temporal-spatial-frequency feature fusion of EEG

Depression is a prevalent affective psychiatric disorder projected to be the leading contributor to the world’s disease burden by 2030. Due to its high prevalence and low recognition rate, an objective and effective detection method is urgently needed. Deep learning methods based on electroencephalography (EEG) have shown significant potential in depression detection. However, excessive channels can increase redundancy and computational complexity in EEG, while irrelevant channels may reduce accuracy. Additionally, existing models often overlook the complementarity between the temporal-, spatial-, and frequency-domain features of EEG, limiting their detection capabilities. To address these issues, we propose a method that fuses the temporal, spatial, and frequency domain features of EEG to enhance the detection accuracy while eliminating redundant channels. We introduce an EEG channel selection method based on frequency domain weighting that automatically adjusts the channel weights to select the EEG channels that best capture spatial information across the delta, theta, alpha, beta, and gamma bands, thereby optimizing the extraction of spatial-frequency features. In addition, we designed a multiscale spatiotemporal convolutional attention network to extract the spatiotemporal features of EEG. In this network, the multiscale convolutional attention module enhanced the model’s ability to capture spatial features, whereas the temporal trend-aware self-attention module extracted long-term temporal features by analyzing global correlations across different time points. Experimental results on the MODMA dataset show that our method achieved a 97.24% detection accuracy, surpassing current state-of-the-art models. This study offers a novel approach for constructing depression detection models, providing a foundation for future research and application.

DEDBNet: DoG-enhanced dual-branch object detection network for remote sensing object detection

With the improvement of spatial resolution of remote sensing images, object detection of remote sensing images has gradually become a difficult task. Extracted object features are usually hidden in a large amount of interference information in the background due to the complexity and large area of backgrounds, as well as the multi-scale nature of objects in remote sensing images. Still, many existing background weakening methods face difficulties in practical applications and are prone to high rates of false positives and false negatives. Therefore, remote sensing object detection has become increasingly challenging. To address these challenges, a novel background weakening method called Difference of Gaussian (DoG) to weaken background (DWB) module is proposed. Then, we develop a dual-branch network, named DoG-Enhanced Dual-Branch Object Detection Network (DEDBNet) for Remote Sensing Object Detection. The base branch network is responsible for detecting objects, while the DWB's branch network corrects the detected objects using feature-level attention. To combine the features of these branches, we propose two new methods Self-Mutual-Correcter with Detect heads (SMCD) for corrective learning and Map Channel Attention (MCA) for channel attention. Self-Corrector (SC) enables modification and integration of features, while the Mutual-Corrector (MC) enhances the features and further fuses them. We evaluate our proposed network, DEDBNet, through extensive experiments on four public datasets (DOTA with an mAP of 0.836, DIOR with an mAP of 0.871, NWPU VHR-10 with an mAP of 0.973, and RSOD with an mAP of 0.975). The results demonstrate that our method outperforms other state-of-the-art object detection methods significantly for remote sensing images.

Night Lighting Fault Detection Based on Improved YOLOv5

Night lighting is essential for urban life, and the occurrence of faults can significantly affect the presentation of lighting effects. Many reasons can cause lighting faults, including the damage of lamps and circuits, and the typical manifestation of the faults is that the lights do not light up. The current troubleshooting mainly relies on artificial visual inspection, making detecting faults difficult and time-consuming. Therefore, it is necessary to introduce technical means to detect lighting faults. However, current research on lighting fault detection mainly focuses on using non-visual methods such as sensor data analysis, which has the disadvantages of having a high cost and difficulty adapting to large-scale fault detection. Therefore, this study mainly focuses on solving the problem of the automatic detection of night lighting faults using machine vision methods, especially object detection methods. Based on the YOLOv5 model, two data fusion models have been developed based on the characteristics of lighting fault detection inverse problems: YOLOv5 Channel Concatenation and YOLOv5 Image Fusion. Based on the dataset obtained from the developed automatic image collection and annotation system, the training and evaluation of these three models, including the original YOLOv5, YOLOv5 Channel Concatenation, and YOLOv5 Image Fusion, have been completed. Research has found that applying complete lighting images is essential for the problem of lighting fault detection. The developed Image Fusion model can effectively fuse information and accurately detect the occurrence and area of faults, with a mAP value of 0.984. This study is expected to play an essential role in the intelligent development of urban night lighting.

Intelligent Video Analytics for Abnormal Event Detection

Surveillance cameras have become a primary tool for monitoring human movement and preventing unwanted or unintended activities. In today's world, security management professionals increasingly rely on video surveillance to combat crime and mitigate incidents that could adversely affect society. The number of surveillance camera installations has dramatically increased in both the private and public sectors, serving to monitor public activities effectively. Video surveillance is considered one of the most efficient methods for ensuring security. Installing a surveillance camera allows the captured footage to be transmitted to security personnel; however, merely having video footage is insufficient for identifying abnormal activities. To effectively analyze this footage, it is essential to integrate an intelligent system.This paper aims to design and implement an Intelligent Video Analytics Model (IVAM), also known as the Human Object Detection (HOD) method, for analyzing and detecting video-based anomalies and abnormal human activities. IVAM can be deployed alongside surveillance cameras in various public locations such as Institutions, airports, hospitals, shopping malls, and railway stations, allowing for the automatic identification of unusual events. The IVAM was experimented with using MATLAB software, and the results were thoroughly verified. The performance of IVAM was assessed by comparing the obtained results with those from existing approaches, demonstrating that IVAM outperforms contemporary methods concerning accurate anomaly detection, lower error rates, and higher classification accuracy.

YOLO‐RSFM: An efficient road small object detection method

AbstractTo tackle challenges in road multi‐object detection, such as object occlusion, small object detection, and multi‐scale object detection difficulties, a new YOLOv8n‐RSFM structure is proposed. The key improvement of this structure lies in the introduction of the transformer decoder head, which optimizes the matching between the ground truth and predicted boxes, thereby effectively addressing issues of object overlap and multi‐scale detection. Additionally, a small object detection layer is incorporated to retain crucial information beneficial for detecting small objects, significantly improving the detection accuracy for small targets. To enhance learning capacity and reduce redundant computations, the FasterNet backbone is employed to replace CSPDarknet53, thus accelerating the training process. Finally, the INNER‐MPDIoU loss function is introduced to replace the original algorithm's complete IoU to accelerate the convergence and obtain more accurate regression results. A series of experiments were conducted on different datasets. The experimental results show that the proposed model YOLOv8N‐RSFM outperforms the original model YOLOv8n in small target detection. On the VisDrone, TinyPerson, and VSCrowd datasets, the mean accuracy percentage improved by 7.9%, 12.3%, and 4.5%, respectively.

Nighttime Pothole Detection: A Benchmark

In the field of computer vision, the detection of road potholes at night represents a critical challenge in enhancing the safety of intelligent transportation systems. Ensuring road safety is of paramount importance, particularly in promptly repairing pothole issues. These abrupt road depressions can easily lead to vehicle skidding, loss of control, and even traffic accidents, especially when water has pooled in or submerged the potholes. Therefore, the detection and recognition of road potholes can significantly reduce vehicle damage and the incidence of safety incidents. However, research on road pothole detection lacks high-quality annotated datasets, particularly under low-light conditions at night. To address this issue, this study introduces a novel Nighttime Pothole Dataset (NPD), independently collected and comprising 3831 images that capture diverse scene variations. The construction of this dataset aims to counteract the insufficiency of existing data resources and strives to provide a richer and more realistic benchmark. Additionally, we develop a baseline detector, termed WT-YOLOv8, for the proposed dataset, based on YOLOv8. We also evaluate the performance of the improved WT-YOLOv8 method and eight state-of-the-art object detection methods on the NPD and the COCO dataset. The experimental results on the NPD demonstrate that WT-YOLOv8 achieves a 2.3% improvement in mean Average Precision (mAP) over YOLOv8. In terms of the key metrics—AP@0.5 and AP@0.75—it shows enhancements of 1.5% and 2.8%, respectively, compared to YOLOv8. The experimental results provide valuable insights into each method’s strengths and weaknesses under low-light conditions. This analysis highlights the importance of a specialized dataset for nighttime pothole detection and shows variations in accuracy and robustness among methods, emphasizing the need for improved nighttime pothole detection techniques. The introduction of the NPD is expected to stimulate further research, encouraging the development of advanced algorithms for nighttime pothole detection, ultimately leading to more flexible and reliable road maintenance and road safety.

A lightweight real-time detection method of small objects for home service robots

A lightweight real-time detection method of small objects for home service robots

Relevance Pooling Guidance and Class-Balanced Feature Enhancement for Fine-Grained Oriented Object Detection in Remote Sensing Images

Fine-grained object detection in remote sensing images is highly challenging due to class imbalance and high inter-class indistinguishability. The strategies employed by most existing methods to resolve these two challenges are relatively rudimentary, resulting in suboptimal model performance. To address these issues, we propose a fine-grained object-oriented detection method based on relevance pooling guidance and class balance feature enhancement. Firstly, we propose a global attention mechanism that dynamically retains spatial features pertinent to objects through relevance pooling during down-sampling, thereby enabling the model to acquire more discriminative features. Next, a class balance correction module is proposed to alleviate the class imbalance problem. This module employs feature translation and a learnable reinforcement coefficient to highlight the boundaries of tail class features while maintaining their distinctiveness. Furthermore, we present an enhanced contrastive learning strategy. By dynamically adjusting the contribution of inter-class samples and intra-class similarity measures, this strategy not only constrains inter-class feature distances but also facilitates tighter intra-class clustering, making it more suitable for imbalanced datasets. Evaluation on the FAIR1M and MAR20 datasets demonstrates that our method is superior compared to other methods in object detection and achieves 46.44 and 85.05% mean average precision, respectively.

License Plate Recognition Using Three-Dimensional Rotated Character Recognition and Instance Segmentation by Deep Learning

License plate recognition is currently used in various situations, such as parking lot vehicle management and the tracking of wanted vehicles. These applications require recognition from all possible camera angles. Therefore, we propose a method for license plate recognition in images captured from various camera angles. First, the license plate area is detected from the input image using the YOLOv5 object detection method. The subsequent process can be performed in two manners: by rotating the license plate to the front using projective transformation and trapezoidal correction, or by using graph matching. The first method performs Hough transforms or linear approximations as preprocessing, and then calculates the rectangle surrounding the license plate from the YOLOv5 output. After performing trapezoidal correction, the maximally stable extremal regions (MSER) are used to detect character candidates, and characters are recognized using 3D rotated character recognition. The second method performs character candidate detection and recognition without trapezoidal correction, followed by graph matching. These methods are versatile because they do not depend on the layout of license plates, which varies among countries and regions. Two types of datasets were used: a set of images containing Japanese license plates collected by ourselves, and a set of publicly available images containing Taiwanese license plates. Two recognition rates were evaluated: the license plate character recognition rate and license plate recognition rate in which all characters in one plate are recognized. The license plate character recognition rates using graph matching were 91.9% for the Japanese license plates and 89.3% for the Taiwanese license plates. The license plate recognition rates were 84.0% and 84.5%, respectively.

Research on a small target object detection method for aerial photography based on improved YOLOv7

Research on a small target object detection method for aerial photography based on improved YOLOv7

Evolutionary computation-based self-supervised learning for image processing: a big data-driven approach to feature extraction and fusion for multispectral object detection

The image object recognition and detection technology are widely used in many scenarios. In recent years, big data has become increasingly abundant, and big data-driven artificial intelligence models have attracted more and more attention. Evolutionary computation has also provided a powerful driving force for the optimization and improvement of deep learning models. In this paper, we propose an image object detection method based on self-supervised and data-driven learning. Differ from other methods, our approach stands out due to its innovative use of multispectral data fusion and evolutionary computation for model optimization. Specifically, our method uniquely combines visible light images and infrared images to detect and identify image targets. Firstly, we utilize a self-supervised learning method and the AutoEncoder model to perform high-dimensional feature extraction on the two types of images. Secondly, we fuse the extracted features from the visible light and infrared images to detect and identify objects. Thirdly, we introduce a model parameter optimization method using evolutionary learning algorithms to enhance model performance. Validation on public datasets shows that our method achieves comparable or superior performance to existing methods.

MFLD: lightweight object detection with multi-receptive field and long-range dependency in remote sensing images

PurposeBased on the strong feature representation ability of the convolutional neural network (CNN), generous object detection methods in remote sensing (RS) have been proposed one after another. However, due to the large variation in scale and the omission of relevant relationships between objects, there are still great challenges for object detection in RS. Most object detection methods fail to take the difficulties of detecting small and medium-sized objects and global context into account. Moreover, inference time and lightness are also major pain points in the field of RS.Design/methodology/approachTo alleviate the aforementioned problems, this study proposes a novel method for object detection in RS, which is called lightweight object detection with a multi-receptive field and long-range dependency in RS images (MFLD). The multi-receptive field extraction (MRFE) and long-range dependency information extraction (LDIE) modules are put forward.FindingsTo concentrate on the variability of objects in RS, MRFE effectively expands the receptive field by a combination of atrous separable convolutions with different dilated rates. Considering the shortcomings of CNN in extracting global information, LDIE is designed to capture the relationships between objects. Extensive experiments over public datasets in RS images demonstrate that our MFLD method surpasses the state-of-the-art methods. Most of all, on the NWPU VHR-10 dataset, our MFLD method achieves 94.6% mean average precision with 4.08 M model volume.Originality/valueThis paper proposed a method called lightweight object detection with multi-receptive field and long-range dependency in RS images.

Rapid automatic multiple moving objects detection method based on feature extraction from images with non-sidereal tracking

ABSTRACT Optically observing and monitoring moving objects, both natural and artificial, is important to human space security. Non-sidereal tracking can improve the system’s limiting magnitude for moving objects, which benefits the surveillance. However, images with non-sidereal tracking include complex background, as well as objects with different brightness and moving mode, posing a significant challenge for accurate multi-object detection in such images, especially in wide field-of-view telescope images. To achieve a higher detection precision in a higher speed, we proposed a novel object detection method, which combines the source feature extraction and the neural network. First, our method extracts object features from optical images such as centroid, shape, and flux. Then, it conducts a naive labelling based on those features to distinguish moving objects from stars. After balancing the labelled data, we employ it to train a neural network aimed at creating a classification model for point-like and streak-like objects. Ultimately, based on the neural network model’s classification outcomes, moving objects whose motion modes consistent with the tracked objects are detected via track association, while objects with different motion modes are detected using morphological statistics. The validation, based on the space objects images captured in target tracking mode with the 1-m telescope at Nanshan, Xinjiang Astronomical Observatory, demonstrates that our method achieves 94.72 per cent detection accuracy with merely 5.02 per cent false alarm rate, and a processing time of 0.66 s per frame. Consequently, our method can rapidly and accurately detect objects with different motion modes from wide-field images with non-sidereal tracking.

AEPF: Attention-Enabled Point Fusion for 3D Object Detection.

Current state-of-the-art (SOTA) LiDAR-only detectors perform well for 3D object detection tasks, but point cloud data are typically sparse and lacks semantic information. Detailed semantic information obtained from camera images can be added with existing LiDAR-based detectors to create a robust 3D detection pipeline. With two different data types, a major challenge in developing multi-modal sensor fusion networks is to achieve effective data fusion while managing computational resources. With separate 2D and 3D feature extraction backbones, feature fusion can become more challenging as these modes generate different gradients, leading to gradient conflicts and suboptimal convergence during network optimization. To this end, we propose a 3D object detection method, Attention-Enabled Point Fusion (AEPF). AEPF uses images and voxelized point cloud data as inputs and estimates the 3D bounding boxes of object locations as outputs. An attention mechanism is introduced to an existing feature fusion strategy to improve 3D detection accuracy and two variants are proposed. These two variants, AEPF-Small and AEPF-Large, address different needs. AEPF-Small, with a lightweight attention module and fewer parameters, offers fast inference. AEPF-Large, with a more complex attention module and increased parameters, provides higher accuracy than baseline models. Experimental results on the KITTI validation set show that AEPF-Small maintains SOTA 3D detection accuracy while inferencing at higher speeds. AEPF-Large achieves mean average precision scores of 91.13, 79.06, and 76.15 for the car class's easy, medium, and hard targets, respectively, in the KITTI validation set. Results from ablation experiments are also presented to support the choice of model architecture.

Improved region proposal network for enhanced few-shot object detection

Despite significant success of deep learning in object detection tasks, the standard training of deep neural networks requires access to a substantial quantity of annotated images across all classes. Data annotation is an arduous and time-consuming endeavor, particularly when dealing with infrequent objects. Few-shot object detection (FSOD) methods have emerged as a solution to the limitations of classic object detection approaches based on deep learning. FSOD methods demonstrate remarkable performance by achieving robust object detection using a significantly smaller amount of training data. A challenge for FSOD is that instances from novel classes that do not belong to the fixed set of training classes appear in the background and the base model may pick them up as potential objects. These objects behave similarly to label noise because they are classified as one of the training dataset classes, leading to FSOD performance degradation. We develop a semi-supervised algorithm to detect and then utilize these unlabeled novel objects as positive samples during the FSOD training stage to improve FSOD performance. Specifically, we develop a hierarchical ternary classification region proposal network (HTRPN) to localize the potential unlabeled novel objects and assign them new objectness labels to distinguish these objects from the base training dataset classes. Our improved hierarchical sampling strategy for the region proposal network (RPN) also boosts the perception ability of the object detection model for large objects. We test our approach and COCO and PASCAL VOC baselines that are commonly used in FSOD literature. Our experimental results indicate that our method is effective and outperforms the existing state-of-the-art (SOTA) FSOD methods. Our implementation is provided as a supplement to support reproducibility of the results https://github.com/zshanggu/HTRPN.11Early partial results of this work is presented in the 2023 ICCV Workshop on Visual Continual Learning (Shangguan and Rostami, 2023).

Motion Prediction and Object Detection for Image-Based Visual Servoing Systems Using Deep Learning

This study primarily investigates advanced object detection and time series prediction methods in image-based visual servoing systems, aiming to capture targets better and predict the motion trajectory of robotic arms in advance, thereby enhancing the system’s performance and reliability. The research first implements object detection on the VOC2007 dataset using the Detection Transformer (DETR) and achieves ideal detection scores. The particle swarm optimization algorithm and 3-5-3 polynomial interpolation methods were utilized for trajectory planning, creating a unique dataset through simulation. This dataset contains randomly generated trajectories within the workspace, fully simulating actual working conditions. Significantly, the Bidirectional Long Short-Term Memory (BILSTM) model was improved by substituting its traditional Multilayer Perceptron (MLP) components with Kolmogorov–Arnold Networks (KANs). KANs, inspired by the K-A theorem, improve the network representation ability by placing learnable activation functions on fixed node activation functions. By implementing KANs, the model enhances parameter efficiency and interpretability, thus addressing the typical challenges of MLPs, such as the high parameter count and lack of transparency. The experiments achieved favorable predictive results, indicating that the KAN not only reduces the complexity of the model but also improves learning efficiency and prediction accuracy in dynamic visual servoing environments. Finally, Gazebo software was used in ROS to model and simulate the robotic arm, verify the effectiveness of the algorithm, and achieve visual servoing.

Spatial awareness enhancement based single-stage anchor-free 3D object detection for autonomous driving

The real-time and accurate detection of three-dimensional (3D) objects based on LiDAR is a focal problem in the field of autonomous driving environment perception. Compared to two-stage and anchor-based 3D object detection methods that suffer from inference latency challenges, single-stage anchor-free 3D object detection approaches are more suitable for deployment in autonomous driving vehicles with the strict real-time requirement. However, they face the issue of insufficient spatial awareness, which can result in detection errors such as false positives and false negatives, thereby increasing the potential risks of autonomous driving. In response to this, we focus on enhancing the spatial awareness of CenterPoint, a widely used single-stage anchor-free 3D object detector in the industry. Considering the limited allocation of computational resources and the performance bottleneck caused by pillar encoder, we propose an efficient SSDCM backbone to strengthen feature representation and extraction. Furthermore, a simple BGC neck is devised to weight and exchange contextual information in order to deeply fuse multi-scale features. Combining improved backbone and neck networks, we construct a single-stage anchor-free 3D object detection model with spatial awareness enhancement, named CenterPoint-Spatial Awareness Enhancement (CenterPoint-SAE). We evaluate CenterPoint-SAE on two large-scale and challenging autonomous driving datasets, nuScenes and Waymo. It achieves 53.3% mAP and 62.5% NDS on nuScenes detection benchmark, and runs inference at a speed of 11.1 FPS. Compared to the baseline, the upgraded networks deliver a performance improvement of 1.6% mAP and 1.2% NDS at minor cost. Notably, on Waymo dataset, our method achieves competitive detection performance compared to two-stage and point-based methods.

Performance evaluation and integration of distortion mitigation methods for fisheye video object detection

Performance evaluation and integration of distortion mitigation methods for fisheye video object detection

The Resident Space Object Detection Method Based on the Connection between the Fourier Domain Image of the Video Data Difference Frame and the Orbital Velocity Projection

The Resident Space Object Detection Method Based on the Connection between the Fourier Domain Image of the Video Data Difference Frame and the Orbital Velocity Projection