Object Detection Method Research Articles

A structural description of pedestrian movement behavior in multiple surveillance videos

This study purpose of recording pedestrians' activities in all surveillance videos and then convert these records into structured descriptions to analyze a large number of surveillance videos. This paper proposes a four-stage processing method. The first stage uses convolutional neural network (CNN) to detect objects in the surveillance video and classify them. The second stage finds the correlation of pedestrians in continuous images and describes multiple objects' trajectories in the surveillance video. The third stage calculates the similarity of the pedestrian object sets of the surveillance video at different times as the basis for pedestrian re-identification. In the fourth stage, extract the walking path of the pedestrian in each surveillance video. Each pedestrian's actions in each surveillance video, including whether to stand, walking path, direction, clothing color and walking time. To construct a correlation table of pedestrian motion in multiple surveillance videos.This experiment uses two image datasets for experimentation and analysis to evaluate this method's effectiveness for pedestrian movement in multiple surveillance videos. The image datasets are the Multi-Camera Object Tracking (MCT) Challenge and PETS 2009 Benchmark Data. This experiment includes pedestrian detection, pedestrian identification, and multi-pedestrian tracking of pedestrians in single and multiple surveillance cameras. The results show that in the pedestrian detection experiment, the convolutional neural network can effectively solve the problems caused by static objects, light changes, similar colors, crowded pedestrians, and incomplete objects. In the pedestrian recognition and re-recognition experiments, the accuracy is reduced due to the influence of low-resolution surveillance videos. This detection method of the pedestrian is more accurate than traditional dynamic object detection methods. After the pedestrian tracking and re-identification are completed, the pedestrian relationships in multiple surveillance cameras are integrated, and a pedestrian behavior association table is built for surveillance videos.

Object Detection Using CNN

Object detection system using Convolutional Neural Network(CNN) that can accurately identify and classify objects in videos. The purpose of object detection using CNN to enhance technology such as security cameras, smart devices by enabling them to identify and understand objects in videos. Object detection using CNN is a fascinating filed in computer vision. Detection can be difficult since there are all kinds of variations in orientation, lighting, background that can result in completely different videos of the very same object. Now with the advance of deep learning and neural network, we can finally tackle such problems without coming up with various heuristics real-time. The project “Object detection using CNN for video streaming” detects objects efficiently based on CNN algorithm and apply the algorithm on image or video data. In this project, we develop a technique to identify an object considering the deep learning pre-trained model MobileNet for Single Shot Multi-Box Detector (SSD). This algorithm is used for real-time detection and for webcam, which detects the objects in a video stream. Therefore, we use an object detection module that can detect what is in the video stream. In order to implement the module, we combine the MobileNet and the SSD framework for a fast and efficient deep learning-based method of object detection. The main purpose of our research is to elaborate the accuracy of an object detection method SSD and the importance of pre-trained deep learning model MobileNet. The experimental results show that the Average Precision (AP) of the algorithm to detect different classes as car, person and chair is 99.76%, 97.76% and 71.07%, respectively. The main objective of our project is to make clear the object detecting accuracy. The existing methods are Region Based Convolutional Neural Network(R-CNN) and You Only Look Once(YOLO).R-CNN could not pushed real time speed though its system is updated and new versions of it are deployed and YOLO network is popular but YOLO is to struggle to detect objects grouped close together, especially smaller ones. To avoid the drawbacks of these methods we proposed this model which included single shot multi-box detector (SSD), this algorithm is used for real time detection and Mobile-Net architecture.

G-YOLO: A YOLOv7-based target detection algorithm for lightweight hazardous chemical vehicles.

Hazardous chemical vehicles are specialized vehicles used for transporting flammable gases, medical waste, and liquid chemicals, among other dangerous chemical substances. During their transportation, there are risks of fire, explosion, and leakage of hazardous materials, posing serious threats to human safety and the environment. To mitigate these possible hazards and decrease their probability, this study proposes a lightweight object detection method for hazardous chemical vehicles based on the YOLOv7-tiny model.The method first introduces a lightweight feature extraction structure, E-GhostV2 network, into the trunk and neck of the model to achieve effective feature extraction while reducing the burden of the model. Additionally, the PConv is used in the model's backbone to effectively reduce redundant computations and memory access, thereby enhancing efficiency and feature extraction capabilities. Furthermore, to address the problem of performance degradation caused by overemphasizing high-quality samples, the model adopts the WIoU loss function, which balances the training effect of high-quality and low-quality samples, enhancing the model's robustness and generalization performance. Experimental results demonstrate that the improved model achieves satisfactory detection accuracy while reducing the number of model parameters, providing robust support for theoretical research and practical applications in the field of hazardous chemical vehicle object detection.

Floating object detection using double-labelled domain generalization

Detection of floating objects in complicated aquatic environments has a wide range of applications, but it confronts significant hurdles due to the imbalance in detection accuracy and efficiency, and low domain generalization performance. To address these issues, this study proposes a novel floating object detection method based on double-labelled domain generalization. First, the Single Shot Multibox Detector (SSD) is improved by replacing the backbone network with a lightweight feature extraction network, and dynamic feature pyramid network is introduced to balance accuracy and efficiency. Then, this study initializes the improved SSD network based on the double-labelled data of the source domain, and filters the feature extractor bias, classification bias and location bias using pseudo-labelling and feature projection based on the un-labelled source domain data to minimize the bias to improve the domain generalization performance. The proposed method is trained on a self-constructed floating object dataset and is compared with state-of-the-art methods based on multiple scenarios. The results show that the proposed method achieves better performance in double-labelled domain generalization and conventional domain generalization tasks compared to other methods, achieving 70.33%, 22.38 f/s and 85.29% and 17.81 f/s in accuracy and speed respectively, satisfying the need for multi-scale floating object detection in complex environments and also alleviating the data labelling problem. This work effectively solves the problem of slow detection of floating objects due to the complex model structure and low generalization ability, and provides support for the rapid detection of floating objects in complex scenarios and the promotion of technology applications.

The evolution of object detection methods

Object detection is one of the most important domains in computer vision tasks, which is an important branch of artificial intelligence. It aims at finding and locating the accurate position of objects in given pictures or videos. With the development of deep learning techniques, more powerful and robust algorithms have emerged to deal with multi-scale, high-level features to overcome the limitations of traditional pipeline of object detectors. The popularity of transformer framework enables larger capacity datasets by processing self-attention mechanism, and the object detection methods have evolved into a new era. This paper first reviews traditional object detection pipeline and brief history of deep learning, afterwards it focuses on the classification of deep learning-based object detection methods covering Convolution Neural Network based and transformer-based methods. Commonly used datasets and metrics are also covered in the next part. The Convolution Neural Network based methods mainly contain two-stage and one-stage detectors, Convolution Neural Network is the underlying structure of these methods convolutional stages are fundamental parts. Transformer-based models convert traditional object detection issues into end-to-end detection, which is widely used in dealing with images. Finally, the promising future of object detection areas are listed to show guidance on future work.

SKZC: self-distillation and k-nearest neighbor-based zero-shot classification

Zero-shot learning represents a formidable paradigm in machine learning, wherein the crux lies in distilling and generalizing knowledge from observed classes to novel ones. The objective is to identify unfamiliar objects that were not included in the model’s training, leveraging learned patterns and knowledge from previously encountered categories. As a crucial subtask of open-world object detection, zero-shot classification can also provide insights and solutions for this field. Despite its potential, current zero-shot classification models often suffer from a performance gap due to limited transfer ability and discriminative capability of learned representations. In pursuit of advancing the subpar state of zero-shot object classification, this paper introduces a novel model for image classification which can be applied to object detection, namely, self-distillation and k-nearest neighbor-based zero-shot classification method. First, we employ a diffusion detector to identify potential objects in images. Then, self-distillation and distance-based classifiers are used for distinguishing unseen objects from seen classes. The k-nearest neighbor-based cluster heads are designed to cluster the unseen objects. Extensive experiments and visualizations were conducted on publicly available datasets on the efficacy of the proposed approach. Precisely, our model demonstrates performance improvement of over 20% compared to contrastive clustering. Moreover, it achieves a precision of 0.910 and a recall of 0.842 on CIFAR-10 datasets, a precision of 0.737, and a recall of 0.688 on CIFAR-100 datasets for the macro average. Compared to a more recent model (SGFR), our model realized improvements of 10.9%, 13.3%, and 7.8% in Sacc, Uacc, and H metrics, respectively. This study aims to introduce fresh ideas into the domain of zero-shot image classification, and it can be applied to open-world object detection tasks. Our code is available at https://www.github.com/CmosWolf1/Code_implementation_for_paper_SKZC.

SIAM: A parameter-free, Spatial Intersection Attention Module

Attention mechanisms have been shown to play a crucial role in enhancing visual perception tasks. However, in most existing approaches, channel and spatial attention maps are estimated separately without considering the varying importance of each other. This results in a coarse attention weight for objects of interest from a holistic 3-D perspective. To address this issue, we propose a novel Parameter-free Spatial Intersection Attention Module (SIAM), which estimates 3D attention maps with spatial intersection using a parameter-free way. Specifically, SIAM first generates two independent mean queries from two spatial axes and views input as keys. Then, by computing a dot product between these mean queries and keys, SIAM generates two cross-dimension (channel and spatial) attention maps from two spatial directions and combines them into 3-D attention maps. By doing so, the produced attention maps reason important areas with spatial intersection, which can capture location-aware information to facilitate difficult objects’ location in the images. We evaluate our method in image classification, object detection, and object segmentation tasks. Extensive experimental results consistently demonstrate our approach is superior to its counterparts.

Lightweight UAV Object-Detection Method Based on Efficient Multidimensional Global Feature Adaptive Fusion and Knowledge Distillation

Lightweight UAV Object-Detection Method Based on Efficient Multidimensional Global Feature Adaptive Fusion and Knowledge Distillation

Aircraft Objection Detection Method of Airport Surface based on Improved YOLOv5

An aircraft object detection method on the basis of improved YOLOv5 was proposed to address the issues of large model size, high number of parameters, and inability to meet real-time monitoring requirements of aircrafts in traditional object detection. Firstly, the basic unit of ShuffleNetv2 network was optimized through replacing 3x3 convolution with 5x5 convolution and removing subsequent 1x1 convolution. Simultaneously, the original ReLU activation function was replaced with PReLU. Secondly, CBAM (Convolutional Block Attention Module) attention mechanism was developed to enhance the detection accuracy of the improved network. Finally, improved ShuffleNetv2 network was applied as the backbone structure of YOLOv5. Experimental results revealed that the parameter number of the improved YOLOv5 method introduced in this paper was decreased by 18 times, with a model size of 1.03M. Therefore, a 20.8% increase was achieved in frames per second (FPS) in GPU environments and a 234.6% increase was observed in FPS in CPU environments, while a mean average precision (mAP@0.5) of 0.99 was maintained compared with traditional YOLOv5 network. Because of the advantages of fewer parameters, faster recognition speed, higher localization accuracy, and smaller memory requirement, the developed method was found to be suitable for real-time monitoring of aircrafts in airport surface.

Large Span Sizes and Irregular Shapes Target Detection Methods Using Variable Convolution-Improved YOLOv8.

In this work, an object detection method using variable convolution-improved YOLOv8 is proposed to solve the problem of low accuracy and low efficiency in detecting spanning and irregularly shaped samples. Aiming at the problems of the irregular shape of a target, the low resolution of labeling frames, dense distribution, and the ease of overlap, a deformable convolution module is added to the original backbone network. This allows the model to deal flexibly with the problem of the insufficient perceptual field of the target corresponding to the detection point, and the situations of leakage and misdetection can be effectively improved. In order to solve the issue that small target detection is susceptible to image background and noise interference, the Sim-AM (simple parameter-free attention mechanism) module is added to the backbone network of YOLOv8, which enhances the attention to the underlying features and, thus, improves the detection accuracy of the model. More importantly, the Sim-AM module does not need to add parameters to the original network, which reduces the computation of the model. To address the problem of complex model structures that can lead to slower detection, the spatial pyramid pooling of the backbone network is replaced with focal modulation networks, which greatly simplifies the computation process. The experimental validation was carried out on the scrap steel dataset containing a large number of targets of multiple shapes and sizes. The results showed that the improved YOLOv8 network model improves the AP (average precision) by 2.1%, the mAP (mean average precision value) by 0.8%, and reduces the FPS (frames per second) by 5.4, which meets the performance requirements of real-time industrial inspection.

Detection of Serrated Adenoma in NBI Based on Multi-Scale Sub-Pixel Convolution

Colorectal cancer ranks third in global malignancy incidence, and serrated adenoma is a precursor to colon cancer. However, current studies primarily focus on polyp detection, neglecting the crucial discrimination of polyp nature, hindering effective cancer prevention. This study established a static image dataset for serrated adenoma (SA) and developed a deep learning SA detection model. The proposed MSSDet (Multi-Scale Sub-pixel Detection) innovatively modifies each layer of the original feature pyramid’s structure to retain high-resolution polyp features. Additionally, feature fusion and optimization modules were incorporated to enhance multi-scale information utilization, leveraging the narrow-band imaging endoscope’s ability to provide clearer colonoscopy capillary and texture images. This paper utilized 639 cases of colonic NBI endoscopic images to construct the model, achieving a mean average precision (mAP) of 86.3% for SA in the test set. The SA detection rate via this approach has significantly surpassed conventional object detection methods.

A novel jujube tree trunk and branch salient object detection method for catch-and-shake robotic visual perception

Visual perception has become a prerequisite for automated jujube harvesting robot operations under complex orchard conditions. Catch-and-Shake harvesting, as the most efficient and common harvesting method, has widely been applied on various manually operated harvesters to complete large-area jujube fruit harvesting. However, the main factors restricting the development of existing harvesters are labor shortage, high labor cost, and low operating efficiency. To address the issues, we designed a catch-and-shake harvesting robot for jujube tree trunks and branches visual perception that can provide a barrier-free catch-and-shake operation area and guide the manipulator to reach the area to complete the harvesting operation. Meanwhile, a visual perception system including tree trunks and branches detection, skeleton extraction, catch-and-shake area confirmation was presented to guide robot intelligent operations. In the visual perception system, a novel salientobjectdetectionmodel called feature intersection and fusion Transformer (FIT-Transformer) network was proposed to split branches and background to provide reference for determining safe catch-and-shake areas. Moreover, we designed a diverse feature aggregation (DFA) and an attention feature fusion module (AFFM) to strengthen feature learning capabilities and obtain robust perception models. Comparative experimental results showed that our proposed FIT-Transformer model outperformed 12 state-of-the-art (SOTA) algorithms including C2FNet, RAS, BASNet, U2Net, SCRNet, PiCANet, EDRNet, EGNet, ICONR, VST, TransSOD and ABiU_Net. Specifically, the segmentation accuracy of jujube tree trunks and branches using our method showed the satisfactory result on five evaluation indexes under natural environment (the EM, SM, WF, FM and MAE reached 0.9713, 0.8991, 0.8854, 0.8905, and 0.0302, respectively). Field experiments also proved that our method could meet the requirements of operational accuracy and real-time operations.

Intracranial aneurysm detection: an object detection perspective.

Intracranial aneurysm detection from 3D Time-Of-Flight Magnetic Resonance Angiography images is a problem of increasing clinical importance. Recently, a streak of methods have shown promising performance by using segmentation neural networks. However, these methods may be less relevant in a clinical settings where diagnostic decisions rely on detecting objects rather than their segmentation. We introduce a 3D single-stage object detection method tailored for small object detection such as aneurysms. Our anchor-free method incorporates fast data annotation, adapted data sampling and generation to address class imbalance problem, and spherical representations for improved detection. A comprehensive evaluation was conducted, comparing our method with the state-of-the-art SCPM-Net, nnDetection and nnUNet baselines, using two datasets comprising 402 subjects. The evaluation used adapted object detection metrics. Our method exhibited comparable or superior performance, with an average precision of 78.96%, sensitivity of 86.78%, and 0.53 false positives per case. Our method significantly reduces the detection complexity compared to existing methods and highlights the advantages of object detection over segmentation-based approaches for aneurysm detection. It also holds potential for application to other small object detection problems.

HDNet: Human-like discrimination with visual key for few-shot cross-domain object detection

Most existing few-shot object detection (FSD) methods implicitly assume that the target domain data with few samples conform to the same statistical distribution as the source domain. However, this assumption is impractical, especially when dealing with unconstrained scenarios. Also, the decline of fine-grained hidden samples caused by scene switching, significant morphological changes, etc. has brought great challenges to object detection. To solve the above few-shot cross-domain detection (FS-CDD) problems, in this work, we propose a novel and flexible Human-like Discrimination Network (HDNet), which is composed of four modules. Firstly, multi-level key generation (MKG) fully mining multi-level comprehensive abstract representation. The precious knowledge that contains diverse low and high levels is obtained by three parallel branches. After decoupling the hidden space, the rich patches from each pair of target and source domains are closely matched in the Embedded Space Implicit Association (ESIA) using degree description. In order to enhance the discriminative capability of the model for a few samples, the instances are additionally encoded and corrected at the classification end using the Prediction head with instance embedding (PHIE). Finally, the Adaptive Reweighted Module (ARM) is redesigned to determine coefficient of multiple loss functions, which avoids the improper operation of setting coefficients according to experience in the past. Extensive experiments demonstrate that despite the dual challenges of limited samples and cross-domain scenarios, the proposed HDNet exhibits remarkable performance.

Multiple-in-Single-Out Object Detector Leveraging Spiking Neural Membrane Systems and Multiple Transformers.

Most existing multi-scale object detectors depend on multi-level feature maps. The Feature Pyramid Networks (FPN) is a significant architecture for object detection that utilizes these multi-level feature maps. However, the use of FPN also increases the detector's complexity. For object detection methods that only use a single-level feature map, the detection performance is limited to some extent because the single-level feature map cannot balance deep semantic information and shallow detail information. We introduce a novel detector- the Spiking Neural P Multiple-in-Single-out (SNPMiSo) detector to address these challenges. The SNPMiSo detector is constructed based on SNP-like neurons. In SNPMiSo, we employ two kinds of Transformers to boost the important features across different-level feature maps separately. After enhancing the features, we use an incremental upsampling module to upsample and merge the two feature maps. This combined feature map is input into the NAF dilated residual module and the NAF dual-branch detection head. This process allows us to extract multi-scale features and carry out detection tasks. Our tests show promising results: On the COCO dataset, SNPMiSo attains an Average Precision (AP) of 38.7, an improvement of 1.0 AP over YOLOF.In addition, SNPMiSo demonstrates a quicker detection speed, outperforming some advanced multi-level and single-level object detectors.

Deep Learning for Table Detection and Structure Recognition: A Survey

Tables are everywhere, from scientific journals, papers, websites, and newspapers all the way to items we buy at the supermarket. Detecting them is thus of utmost importance to automatically understanding the content of a document. The performance of table detection has substantially increased thanks to the rapid development of deep learning networks. The goals of this survey are to provide a profound comprehension of the major developments in the field of Table Detection, offer insight into the different methodologies, and provide a systematic taxonomy of the different approaches. Furthermore, we provide an analysis of both classic and new applications in the field. Lastly, the datasets and source code of the existing models are organized to provide the reader with a compass on this vast literature. Finally, we go over the architecture of utilizing various object detection and table structure recognition methods to create an effective and efficient system, as well as a set of development trends to keep up with state-of-the-art algorithms and future research. We have also set up a public GitHub repository where we will be updating the most recent publications, open data, and source code. The GitHub repository is available at https://github.com/abdoelsayed2016/table-detection-structure-recognition.

Research on pedestrian object detection algorithm in urban road scenes based on improved YOLOv5

Pedestrians have random distribution and dynamic characteristics. Aiming to this problem, this paper proposes a pedestrian object detection method based on improved YOLOv5 in urban road scenes. Firstly, the last C3 module was replaced in the Backbone with the SE attention mechanism to enhance the network’s extraction of pedestrian object features and improve the detection accuracy of small-scale pedestrians. Secondly, the EIOU loss function was introduced to optimize the object detection performance of the detection network. To validate the effectiveness of the algorithm, experiments were conducted on a dataset composed of filtered Caltech pedestrian detection data and images taken by ourselves. The experiments showed that the improved algorithm has P-value, R-value, and mAP of 98.4%, 95.5%, and 98%, respectively. Compared to the YOLOv5 model, it has increased P-value by 1.4%, R-value by 2.7%, and mAP by 1.3%. The improved algorithm also boosts the detection speed. The detection speed is 0.8 ms faster than the YOLOv5 model. It is also faster than other mainstream algorithms including Faster R-CNN and SSD. The improved algorithm enhances the effectiveness of pedestrian detection significantly and has important application value.

Deep Learning-Based Fish Detection Using Above-Water Infrared Camera for Deep-Sea Aquaculture: A Comparison Study.

Long-term, automated fish detection provides invaluable data for deep-sea aquaculture, which is crucial for safe and efficient seawater aquafarming. In this paper, we used an infrared camera installed on a deep-sea truss-structure net cage to collect fish images, which were subsequently labeled to establish a fish dataset. Comparison experiments with our dataset based on Faster R-CNN as the basic objection detection framework were conducted to explore how different backbone networks and network improvement modules influenced fish detection performances. Furthermore, we also experimented with the effects of different learning rates, feature extraction layers, and data augmentation strategies. Our results showed that Faster R-CNN with the EfficientNetB0 backbone and FPN module was the most competitive fish detection network for our dataset, since it took a significantly shorter detection time while maintaining a high AP50 value of 0.85, compared to the best AP50 value of 0.86 being achieved by the combination of VGG16 with all improvement modules plus data augmentation. Overall, this work has verified the effectiveness of deep learning-based object detection methods and provided insights into subsequent network improvements.

TiTDet: A tiny text detector with scale-sensitive loss and effective fusion factor

The text in natural scenes is often smaller compared to artificially designed text. Due to the small proportion of pixels, low resolution, less semantic information, and susceptibility to complex scenes, tiny text detection often results in many missed detections. To address this issue, this paper draws inspiration from small object detection methods and proposes TiTDet, a detection algorithm more suitable for tiny text. Due to the small proportion of pixels, low resolution, less semantic information, and susceptibility to complex scenes, tiny text detection often results in many missed detections. To address this issue, this paper draws inspiration from small object detection methods and proposes TiTDet, a detection algorithm more suitable for tiny text. Firstly, this paper incorporates a context extraction module and an attention-guided module. These modules guide contextual information learning through a self attention mechanism, while eliminating the possible negative impact caused by redundant information. Regarding multi-scale feature fusion, this paper proposes a fine-grained effective fusion factor, making the fusion process emphasize small object learning more and highlight the feature expression of tiny texts. In terms of post-processing, this paper proposes a differentiable binarization module, incorporating the binarization process into model training. Leveraging the implicit information in the data to drive model improvement can enhance the post-processing effect. Lastly, this paper proposes a scale-sensitive loss, which can handle tiny texts more fairly, fully considering the positional relationship between the predicted and real regions, and better guiding the model training. This paper proves that TiTDet exhibits high sensitivity and accuracy in detecting tiny texts, achieving an 86.0% F1-score on ICDAR2015. The paper also compares the superiority of the method on CTW1500 and Total-Text.

LAGSwin: Local attention guided Swin-transformer for thermal infrared sports object detection.

Compared with visible light images, thermal infrared images have poor resolution, low contrast, signal-to-noise ratio, blurred visual effects, and less information. Thermal infrared sports target detection methods relying on traditional convolutional networks capture the rich semantics in high-level features but blur the spatial details. The differences in physical information content and spatial distribution of high and low features are ignored, resulting in a mismatch between the region of interest and the target. To address these issues, we propose a local attention-guided Swin-transformer thermal infrared sports object detection method (LAGSwin) to encode sports objects' spatial transformation and orientation information. On the one hand, Swin-transformer guided by local attention is adopted to enrich the semantic knowledge of low-level features by embedding local focus from high-level features and generating high-quality anchors while increasing the embedding of contextual information. On the other hand, an active rotation filter is employed to encode orientation information, resulting in orientation-sensitive and invariant features to reduce the inconsistency between classification and localization regression. A bidirectional criss-cross fusion strategy is adopted in the feature fusion stage to enable better interaction and embedding features of different resolutions. At last, the evaluation and verification of multiple open-source sports target datasets prove that the proposed LAGSwin detection framework has good robustness and generalization ability.