Vision-based Target Detection Research Articles

Automated construction safety reporting system integrating deep learning-based real-time advanced detection and visual question answering

The construction sector is globally acknowledged as one of the most hazardous industries, owing to the vulnerability of its workers to accidents, injuries, and even loss of life. Effective precautionary measures are necessary and ensuring the use of personal protective equipment (PPE) by workers is crucial for protecting them from accidents. Existing deep learning-based PPE detection systems mainly use simple vision-based target detection methods for tasks such as the identification of helmets or vests, and they tend to be task-specific. However, the identification of specific PPE based on respective job types and maintaining detailed safety records, requires further innovative approaches. In this paper, we propose an innovative intelligent system that not only accurately recognizes specific PPE according to the needs of different work types but also automatically generates safety inspection reports and establishes complete safety records, thus providing critical data to support accident investigations. The proposed system integrates a target detection model, visual question answering model, and text-based analysis of the relevant regulations to realize real-time detection of PPE and automatic generation of safety inspection reports. The experimental results show that the proposed YOLOv8n-DCA network strikes a good balance between performance and computational cost—, with a mAP value of 86%. Compared to the original YOLOv8n network, the mAP value is improved by 5.1%, while the model parameters and size are significantly reduced. Further, the visual question answering model exhibited a precision is 95.9. Finally, the automatic generation of safety inspection reports was successfully realized, verifying the feasibility of the developed system. This innovative system promises a comprehensive and efficient PPE management solution for the construction industry to ensure worker safety and provide strong data support.

CNN and Binocular Vision-Based Target Detection and Ranging Framework of Intelligent Railway System

CNN and Binocular Vision-Based Target Detection and Ranging Framework of Intelligent Railway System

Improved Mask R-CNN Multi-Target Detection and Segmentation for Autonomous Driving in Complex Scenes.

Vision-based target detection and segmentation has been an important research content for environment perception in autonomous driving, but the mainstream target detection and segmentation algorithms have the problems of low detection accuracy and poor mask segmentation quality for multi-target detection and segmentation in complex traffic scenes. To address this problem, this paper improved the Mask R-CNN by replacing the backbone network ResNet with the ResNeXt network with group convolution to further improve the feature extraction capability of the model. Furthermore, a bottom-up path enhancement strategy was added to the Feature Pyramid Network (FPN) to achieve feature fusion, while an efficient channel attention module (ECA) was added to the backbone feature extraction network to optimize the high-level low resolution semantic information graph. Finally, the bounding box regression loss function smooth L1 loss was replaced by CIoU loss to speed up the model convergence and minimize the error. The experimental results showed that the improved Mask R-CNN algorithm achieved 62.62% mAP for target detection and 57.58% mAP for segmentation accuracy on the publicly available CityScapes autonomous driving dataset, which were 4.73% and 3.96%% better than the original Mask R-CNN algorithm, respectively. The migration experiments showed that it has good detection and segmentation effects in each traffic scenario of the publicly available BDD autonomous driving dataset.

Development of a YOLO-KCF Coupling Algorithm for Miniature Fixed-Wing UAVs in Target Detection and Tracking

Target detection and tracking represent key challenges facing miniature fixed-wing unmanned aerial vehicles (UAVs), particularly at high cruising speeds. Therefore, this paper proposes a vision-based target detection and tracking algorithm that systematically couples two mainstream methods, namely, you only look once (YOLO) and kernel correlation filter (KCF) algorithms. This combination enables small fixed-wing UAVs to achieve reliable target detection and rapid target tracking. A customized vision-guidance module is constructed to implement this algorithm, and a dual-thread execution mechanism is developed to ensure that the computational resources are used effectively. A miniature fixed-wing UAV experimental platform is also constructed and evaluated. Flight experiments are performed, and the results demonstrate that the developed algorithm can achieve satisfactory detection and tracking accuracy for stationary and moving ground targets in complex environments.

Vision-Based Target Detection and Positioning Approach for Underwater Robots

The accurate target detection under different environmental conditions and the real-time target positioning are vital for the successful accomplishment of underwater missions of Remotely operated vehicles (ROVs). In this paper, we propose a vision-based underwater target detection and positioning approach to detect and estimate the position and attitude of artificial underwater targets. The proposed approach is composed of an underwater target detection algorithm YOLO-T and a target positioning algorithm. Firstly, we modify the structure of YOLOv5 algorithm using Ghost module and SE attention module to improve the calculation time of target detection. Secondly, a series of image processing operations are performed on the improved YOLOv5 detection results to increase the detection accuracy. Thirdly, a cooperative marker is designed as the artificial underwater target, and the corresponding positioning algorithm is presented to calculated the position and attitude of the target according to the geometric information of the designed marker. We validate our approach through experimental tests respectively in a water tank, an anechoic tank, and the sea trial in Huanghai Sea in China. The results demonstrate the accurate performance of the proposed detection and positioning method.

Vision-Based Target Detection and Tracking for a Miniature Pan-Tilt Inertially Stabilized Platform

This paper presents a novel visual servoing sheme for a miniature pan-tilt intertially stabilized platform (ISP). A fully customized ISP can be mounted on a miniature quadcopter to achieve stationary or moving target detection and tracking. The airborne pan-tilt ISP can effectively isolate a disturbing rotational motion of the carrier, ensuring the stabilization of the optical axis of the camera in order to obtain a clear video image. Meanwhile, the ISP guarantees that the target is always on the optical axis of the camera, so as to achieve the target detection and tracking. The vision-based tracking control design adopts a cascaded control structure based on the mathematical model, which can accurately reflect the dynamic characteristics of the ISP. The inner loop of the proposed controller employs a proportional lag compensator to improve the stability of the optical axis, and the outer loop adopts the feedback linearization-based sliding mode control method to achieve the target tracking. Numerical simulations and laboratory experiments demonstrate that the proposed controller can achieve satisfactory tracking performance.

A Machine Vision-Based Method Optimized for Restoring Broiler Chicken Images Occluded by Feeding and Drinking Equipment.

Simple SummaryThe equipment in the poultry house can occlude top view images of broiler chickens and limit the efficiency of vision-based target detection. In this study, we sought to improve the efficiency of a previously developed method to detect and restore broiler chicken areas blocked by feeders and drinkers. To do this, we developed and tested linear and elliptical fitting restoration methods under different occlusion scenarios to restore occluded broiler chicken areas. The restoration method correctly restored the occluded broiler chicken area >80% of the time. This study provides a practical approach to enhancing the image quality in applying a machine vision-based method for monitoring poultry health and welfare.The presence equipment (e.g., water pipes, feed buckets, and other presence equipment, etc.) in the poultry house can occlude the areas of broiler chickens taken via top view. This can affect the analysis of chicken behaviors through a vision-based machine learning imaging method. In our previous study, we developed a machine vision-based method for monitoring the broiler chicken floor distribution, and here we processed and restored the areas of broiler chickens which were occluded by presence equipment. To verify the performance of the developed restoration method, a top-view video of broiler chickens was recorded in two research broiler houses (240 birds equally raised in 12 pens per house). First, a target detection algorithm was used to initially detect the target areas in each image, and then Hough transform and color features were used to remove the occlusion equipment in the detection result further. In poultry images, the broiler chicken occluded by equipment has either two areas (TA) or one area (OA). To reconstruct the occluded area of broiler chickens, the linear restoration method and the elliptical fitting restoration method were developed and tested. Three evaluation indices of the overlap rate (OR), false-positive rate (FPR), and false-negative rate (FNR) were used to evaluate the restoration method. From images collected on d2, d9, d16, and d23, about 100-sample images were selected for testing the proposed method. And then, around 80 high-quality broiler areas detected were further evaluated for occlusion restoration. According to the results, the average value of OR, FPR, and FNR for TA was 0.8150, 0.0032, and 0.1850, respectively. For OA, the average values of OR, FPR, and FNR were 0.8788, 0.2227, and 0.1212, respectively. The study provides a new method for restoring occluded chicken areas that can hamper the success of vision-based machine predictions.

Vision-Based Target Detection and Tracking System for a Quadcopter

Tracking a maneuvering target with a quadcopter is a challenging problem, that involves a variety of fields such as visual tracking, state estimation, and control algorithms. Most existing unmanned aerial vehicle (UAV) systems fail to track targets accurately in the long term and cannot relocate targets after target loss. This paper aims to design and implement a vision-based target tracking system for quadcopters that can steadily and accurately track the ground target as well as the air target without any prior information. We employ a vision detection algorithm to select the target quickly and precisely. To fit complex practical conditions, a target tracking algorithm is developed based on correlation filters, which is capable of tracking targets with large-scale variation and fast motion. In addition, an efficient redetection algorithm based on the support vector machine (SVM) is designed to handle target occlusions and loss. The target states are estimated from the visual information by an improved Lucas-Kanade (LK) optical flow method and an extended Kalman filter (EKF). Moreover, a double closed-loop Proportion Integral Differential (PID) controller using the estimated states is designed to follow the target. By implementing the main algorithms on an onboard NUC computer, an extensive outdoor flight is evaluated for a quadcopter platform equipped with a stereo camera. The experimental results validate the feasibility and practicability of the developed system.

Autonomous Parking-lots Detection with Multi-sensor Data Fusion Using Machine Deep Learning Techniques

The rapid development and progress in deep machine-learning techniques have become a key factor in solving the future challenges of humanity. Vision-based target detection and object classification have been improved due to the development of deep learning algorithms. Data fusion in autonomous driving is a fact and a prerequisite task of data preprocessing from multi-sensors that provide a precise, well-engineered, and complete detection of objects, scene or events. The target of the current study is to develop an in-vehicle information system to prevent or at least mitigate traffic issues related to parking detection and traffic congestion detection. In this study we examined to solve these problems described by (1) extracting region-of-interest in the images (2) vehicle detection based on instance segmentation, and (3) building deep learning model based on the key features obtained from input parking images. We build a deep machine learning algorithm that enables collecting real video-camera feeds from vision sensors and predicting free parking spaces. Image augmentation techniques were performed using edge detection, cropping, refined by rotating, thresholding, resizing, or color augment to predict the region of bounding boxes. A deep convolutional neural network F-MTCNN model is proposed that simultaneously capable for compiling, training, validating and testing on parking video frames through video-camera. The results of proposed model employing on publicly available PK-Lot parking dataset and the optimized model achieved a relatively higher accuracy 97.6% than previous reported methodologies. Moreover, this article presents mathematical and simulation results using state-of-the-art deep learning technologies for smart parking space detection. The results are verified using Python, TensorFlow, OpenCV computer simulation frameworks.

Autonomous Vision-based Target Detection and Safe Landing for UAV

Autonomous Vision-based Target Detection and Safe Landing for UAV

Autonomous Vision-based Target Detection and Safe Landing for UAV

Target detection is crucial for many applications of Unmanned Aerial Vehicles (UAVs) such as search and rescue, object transportation, object detection, inspection, and mapping. One of the considerable applications of target detection is the safe landing of UAV to the drone station for battery charging and its maintenance. For this, vision-based target detection methods are utilized. Generally, high-cost cameras and expensive CPU’s were used for target detection. With the recent development of Raspberry Pi (RPi), it is possible to use the embedded system with cheap price for such applications. In the current research, RPi based drone target detection and safe landing system are proposed with the integration of PID controller for target detection, and Fuzzy Logic controller for safe landing. The proposed system is equipped with a USB camera which is connected to RPi for detecting the target and a laser rangefinder (LIDAR) for measuring the distance for safe landing. To verify the performance of the developed system, a practical test bench based on a quadcopter and a target drone station is developed. Several experiments were conducted under different scenarios. The result shows that the proposed system works well for the target finding and safe landing of the quadcopter.

Line-Based Simultaneous Detection and Tracking of Triangles

AbstractIn order to capture noncooperative targets at close distance, vision-based target detection and tracking have been used as key technologies with great challenges. This paper proposes a nove...

Vision-Based Target Detection and Localization via a Team of Cooperative UAV and UGVs

Unmanned vehicles (UVs) play a key role in autonomous surveillance scenarios. A major task needed by these UVs in undertaking autonomous patrol missions is to detect the targets and find their locations in real-time. In this paper, a new vision-based target detection and localization system is presented to make use of different capabilities of UVs as a cooperative team. The scenario considered in this paper is a team of an unmanned aerial vehicle (UAV) and multiple unmanned ground vehicles (UGVs) tracking and controlling crowds on a border area. A customized motion detection algorithm is applied to follow the crowd from the moving camera mounted on the UAV. Due to UAVs lower resolution and broader detection range, UGVs with higher resolution and fidelity are used as the individual human detectors, as well as moving landmarks to localize the detected crowds with unknown independently moving patterns at each time point. The UAVs localization algorithm, proposed in this paper, then converts the crowds’ image locations into their real-world positions, using perspective transformation. A rule-of-thumb localization method by a UGV is also presented, which estimates the geographic locations of the detected individuals. Moreover, an agent-based simulation model is developed for system verification, with different parameters, such as flight altitude, number of landmarks, and landmark assignment method. The performance measure considered in this paper is the average Euclidean distance between the estimated locations and simulated geographic waypoints of the crowd. Experimental results demonstrate the effectiveness of the proposed framework for autonomous surveillance by UVs.

Vision-based long-range target detection using coarse-to-fine particle filter

In this study, we develop a coarse-to-fine particle filter algorithm for track-before-detect in order to track a subpixel-sized, low signal-to-noise ratio target in sensor data. The proposed algorithm enhances tracking performance in the presence of target motion uncertainty and it also maintains the computational load without increasing the number of particles. This coarse-to-fine particle filter, which is newly applied to track-before-detect, has two recursive stages: a coarse stage for extensive searches of the target’s state space and a fine stage that narrows down the tracking results. During the coarse stage, particles are propagated with uniformly distributed noise to compensate for highly nonlinear target motion. The fine stage disturbs the particles filtered from the coarse stage using Gaussian distributed noise. Monte Carlo simulation results using artificial image sequences indicate improved performance with the proposed algorithm when uncertain large frame-to-frame pixel differences are caused by nonlinear target motions such as jittering effects. The algorithm is also applied to the real camera image frames to verify its detecting performance.

Tablet PC-based Visual Target-Following System for Quadrotors

This paper proposes a vision-based, closed-loop target-following system for quadrotors. The system consists of a vision-based target detection algorithm that uses the color and image moment of a given target. Flight control commands are directly generated based on the offset of the target from the image frame center. The image processing and control algorithms have been implemented on a latest tablet computer, which is capable of running those algorithms in real time. The proposed system was demonstrated using a commercially available quadrotor platform equipped with a forward-facing camera. Experiments and their analyses showed satisfactory target following performance.