Multi-object Detection Research Articles

Visual localization with a monocular camera for unmanned aerial vehicle based on landmark detection and tracking using YOLOv5 and DeepSORT

Absolute visual localization is of significant importance for unmanned aerial vehicles when the satellite-based localization system is not available. With the rapid evolution in the field of deep learning, the real-time visual detection and tracking of landmarks by an unmanned aerial vehicle could be implemented onboard. This study demonstrates a landmark-based visual localization framework for unmanned aerial vehicles flying at low altitudes. YOLOv5 and DeepSORT are used for multi-object detection and tracking, respectively. The unmanned aerial vehicle localization is achieved according to the geometric similarity between the geotagged transmission towers and the annotated images captured by a monocular camera. The validation is accomplished both in the Rflysim-based simulation and the quadrotor-based real flight. The localization precision is about 10 m, and the location update frequency reaches 5 Hz with a commercially available entry-level edge artificial intelligence platform. The proposed visual localization strategy needs no satellite image as a reference map, which saves a significant amount of the GPU memory and makes possible the end-to-end implementation on small unmanned aerial vehicles.

Benchmarking 2D Multi-Object Detection and Tracking Algorithms in Autonomous Vehicle Driving Scenarios.

Self-driving vehicles must be controlled by navigation algorithms that ensure safe driving for passengers, pedestrians and other vehicle drivers. One of the key factors to achieve this goal is the availability of effective multi-object detection and tracking algorithms, which allow to estimate position, orientation and speed of pedestrians and other vehicles on the road. The experimental analyses conducted so far have not thoroughly evaluated the effectiveness of these methods in road driving scenarios. To this aim, we propose in this paper a benchmark of modern multi-object detection and tracking methods applied to image sequences acquired by a camera installed on board the vehicle, namely, on the videos available in the BDD100K dataset. The proposed experimental framework allows to evaluate 22 different combinations of multi-object detection and tracking methods using metrics that highlight the positive contribution and limitations of each module of the considered algorithms. The analysis of the experimental results points out that the best method currently available is the combination of ConvNext and QDTrack, but also that the multi-object tracking methods applied on road images must be substantially improved. Thanks to our analysis, we conclude that the evaluation metrics should be extended by considering specific aspects of the autonomous driving scenarios, such as multi-class problem formulation and distance from the targets, and that the effectiveness of the methods must be evaluated by simulating the impact of the errors on driving safety.

A Fast and Low-Power Detection System for the Missing Pin Chip Based on YOLOv4-Tiny Algorithm.

In the current chip quality detection industry, detecting missing pins in chips is a critical task, but current methods often rely on inefficient manual screening or machine vision algorithms deployed in power-hungry computers that can only identify one chip at a time. To address this issue, we propose a fast and low-power multi-object detection system based on the YOLOv4-tiny algorithm and a small-size AXU2CGB platform that utilizes a low-power FPGA for hardware acceleration. By adopting loop tiling to cache feature map blocks, designing an FPGA accelerator structure with two-layer ping-pong optimization as well as multiplex parallel convolution kernels, enhancing the dataset, and optimizing network parameters, we achieve a 0.468 s per-image detection speed, 3.52 W power consumption, 89.33% mean average precision (mAP), and 100% missing pin recognition rate regardless of the number of missing pins. Our system reduces detection time by 73.27% and power consumption by 23.08% compared to a CPU, while delivering a more balanced boost in performance compared to other solutions.

Smart Waste Management: Waste Segregation using Machine Learning

In the digitized era, the role of smart mechanisms plays a vital role and one among them is the segregation of waste. To make use of proper disposal and waste management techniques, the segregation of wastes is essential. In the existing systems, drones are used for identifying waste using image processing, and deep learning and use GPS, and GSM methods to identify and send locations to the authorities. The enhancement achieved is to analyze and implement waste segregation with the help of image classification and multi-object detection. Waste management may therefore be done more efficiently with an accuracy of 95% with a mean average of 87.4% which in turn helps significantly to reduce labor costs.

Vision meets algae: A novel way for microalgae recognization and health monitor

Marine microalgae are widespread in the ocean and play a crucial role in the ecosystem. Automatic identification and location of marine microalgae in microscopy images would help establish marine ecological environment monitoring and water quality evaluation system. We proposed a new dataset for the detection of marine microalgae and a range of detection methods, the dataset including images of different genus of algae and the same genus in different states. We set the number of unbalanced classes in the data set and added images of mixed water samples in the test set to simulate the actual situation in the field. Then we trained, validated and tested the, TOOD, YOLOv5, YOLOv8 and variants of RCNN algorithms on this dataset. The results showed both one-stage and two-stage object detection models can achieve high mean average precision, which proves the ability of computer vision in multi-object detection of microalgae, and provides basic data and models for real-time detection of microalgal cells.

SF-YOLOv5: Improved YOLOv5 with swin transformer and fusion-concat method for multi-UAV detection

When dealing with complex trajectories, and the interference by the unmanned aerial vehicle (UAV) itself or other flying objects, the traditional detecting methods based on YOLOv5 network mainly focus on one UAV and difficult to detect the multi-UAV effectively. In order to improve the detection method, a novel algorithm combined with swin transformer blocks and a fusion-concat method based on YOLOv5 network, so called SF-YOLOv5, is proposed. Furthermore, by using the distance intersection over union and non-maximum suppression (DIoU-NMS) as post-processing method, the proposed network can remove redundant detection boxes and improve the efficiency of the multi-UAV detection. Experimental results verify the feasibility and effectiveness of the proposed network, and show that the mAP trained on the two datasets used in experiments has been improved by 2.5 and 4.11% respectively. The proposed network can detect multi-UAV while ensuring accuracy and speed, and can be effectively used in the field of UAV monitoring or other types of multi-object detection applications.

Autonomous sequential surgical skills assessment for the peg transfer task in a laparoscopic box-trainer system with three cameras

AbstractIn laparoscopic surgery, surgeons should develop several manual laparoscopic skills before carrying out real operative procedures using a low-cost box trainer. The Fundamentals of Laparoscopic Surgery (FLS) program was developed as a program to assess fundamental knowledge and surgical skills, required for basic laparoscopic surgery. The peg transfer task is a hands-on exam in the FLS program that assists a trainee to understand the relative minimum amount of grasping force necessary to move the pegs from one place to another place without dropping them. In this paper, an autonomous, sequential assessment algorithm based on deep learning, a multi-object detection method, and, several sequential If-Then conditional statements have been developed to monitor each step of a surgeon’s performance. Images from three different cameras are used to assess whether the surgeon executes the peg transfer task correctly and to display a notification on any errors on the monitor immediately. This algorithm improves the performance of a laparoscopic box-trainer system using top, side, and front cameras and removes the need for any human monitoring during a peg transfer task. The developed algorithm can detect each object and its status during a peg transfer task and notifies the resident about the correct or failed outcome. In addition, this system can correctly determine the peg transfer execution time, and the move, carry, and dropped states for each object by the top, side, and front-mounted cameras. Based on the experimental results, the proposed surgical skill assessment system can identify each object at a high score of fidelity, and the train-validation total loss for the single-shot detector (SSD) ResNet50 v1 was about 0.05. Also, the mean average precision (mAP) and Intersection over Union (IoU) of this detection system were 0.741, and 0.75, respectively. This project is a collaborative research effort between the Department of Electrical and Computer Engineering and the Department of Surgery, at Western Michigan University.

Hybrid Uncalibrated Visual Servoing Control of Harvesting Robots With RGB-D Cameras

Visual servoing (VS) control has seen wide adoption in harvesting robots. However, parameter calibration is cumbersome, which makes the use of VS robotic systems inconvenient. Besides, dynamic fruits usually lead to a degeneration of control while tracking. To overcome the drawbacks, we present a new image-based uncalibrated visual servoing (IBUVS) control approach, consisting of a hybrid visual configuration and an adaptive tracking controller, referred to as hybrid-IBUVS. Specifically, our hybrid-IBUVS employs an eye-in-hand camera and a fixed red–green–blue-depth camera to construct a hybrid VS system, basing on multiobject detection and edge-computing technologies. Meanwhile, we also propose adaptive laws to online estimate the uncalibrated parameters of the cameras and robot dynamics. Furthermore, our hybrid-IBUVS uses an adaptive tracking controller to guarantee the harvesting robot to track a predefined trajectory to approach a fruit target. By Lyapunov stability theory, asymptotic convergence of the proposed control scheme is rigorously proven. Experimental results demonstrate the effectiveness of the proposed scheme. All shown results supported the research claims.

EMDS-7: Environmental microorganism image dataset seventh version for multiple object detection evaluation.

Nowadays, the detection of environmental microorganism indicators is essential for us to assess the degree of pollution, but the traditional detection methods consume a lot of manpower and material resources. Therefore, it is necessary for us to make microbial data sets to be used in artificial intelligence. The Environmental Microorganism Image Dataset Seventh Version (EMDS-7) is a microscopic image data set that is applied in the field of multi-object detection of artificial intelligence. This method reduces the chemicals, manpower and equipment used in the process of detecting microorganisms. EMDS-7 including the original Environmental Microorganism (EM) images and the corresponding object labeling files in ".XML" format file. The EMDS-7 data set consists of 41 types of EMs, which has a total of 2,65 images and 13,216 labeled objects. The EMDS-7 database mainly focuses on the object detection. In order to prove the effectiveness of EMDS-7, we select the most commonly used deep learning methods (Faster-Region Convolutional Neural Network (Faster-RCNN), YOLOv3, YOLOv4, SSD, and RetinaNet) and evaluation indices for testing and evaluation. EMDS-7 is freely published for non-commercial purpose at: https://figshare.com/articles/dataset/EMDS-7_DataSet/16869571.

FPGA-Based Vehicle Detection and Tracking Accelerator.

A convolutional neural network-based multiobject detection and tracking algorithm can be applied to vehicle detection and traffic flow statistics, thus enabling smart transportation. Aiming at the problems of the high computational complexity of multiobject detection and tracking algorithms, a large number of model parameters, and difficulty in achieving high throughput with a low power consumption in edge devices, we design and implement a low-power, low-latency, high-precision, and configurable vehicle detector based on a field programmable gate array (FPGA) with YOLOv3 (You-Only-Look-Once-version3), YOLOv3-tiny CNNs (Convolutional Neural Networks), and the Deepsort algorithm. First, we use a dynamic threshold structured pruning method based on a scaling factor to significantly compress the detection model size on the premise that the accuracy does not decrease. Second, a dynamic 16-bit fixed-point quantization algorithm is used to quantify the network parameters to reduce the memory occupation of the network model. Furthermore, we generate a reidentification (RE-ID) dataset from the UA-DETRAC dataset and train the appearance feature extraction network on the Deepsort algorithm to improve the vehicles' tracking performance. Finally, we implement hardware optimization techniques such as memory interlayer multiplexing, parameter rearrangement, ping-pong buffering, multichannel transfer, pipelining, Im2col+GEMM, and Winograd algorithms to improve resource utilization and computational efficiency. The experimental results demonstrate that the compressed YOLOv3 and YOLOv3-tiny network models decrease in size by 85.7% and 98.2%, respectively. The dual-module parallel acceleration meets the demand of the 6-way parallel video stream vehicle detection with the peak throughput at 168.72 fps.

Boost Correlation Features with 3D-MiIoU-Based Camera-LiDAR Fusion for MODT in Autonomous Driving

Three-dimensional (3D) object tracking is critical in 3D computer vision. It has applications in autonomous driving, robotics, and human–computer interaction. However, methods for using multimodal information among objects to increase multi-object detection and tracking (MOT) accuracy remain a critical focus of research. Therefore, we present a multimodal MOT framework for autonomous driving boost correlation multi-object detection and tracking (BcMODT) in this research study to provide more trustworthy features and correlation scores for real-time detection tracking using both camera and LiDAR measurement data. Specifically, we propose an end-to-end deep neural network using 2D and 3D data for joint object detection and association. A new 3D mixed IoU (3D-MiIoU) computational module is also developed to acquire more precise geometric affinity by increasing the aspect ratio and length-to-height ratio between linked frames. Meanwhile, a boost correlation feature (BcF) module is proposed for the affinity calculation of the appearance of similar objects, which comprises an appearance affinity calculation module for similar objects in adjacent frames that are calculated directly using the feature distance and feature direction’s similarity. The KITTI tracking benchmark shows that our method outperforms other methods with respect to tracking accuracy.

Real-Time Multi-Object Detection Using Enhanced Yolov5-7S on Multi-GPU for High-Resolution Video

Multiple objects tracking in a video sequence can be performed by detecting and distinguishing the objects that appear in the sequence. In the context of computer vision, the robust multi-object tracking problem is a difficult problem to solve. Visual tracking of multiple objects is a vital part of an autonomous driving vehicle’s vision technology. Wide-area video surveillance is increasingly using advanced imaging devices with increased megapixel resolution and increased frame rates. As a result, there is a huge increase in demand for high-performance computation system of video surveillance systems for real-time processing of high-resolution videos. As a result, in this paper, we used a single stage framework to solve the MOT problem. We proposed a novel architecture in this paper that allows for the efficient use of one and multiple GPUs are used to process Full High Definition video in real time. For high-resolution video and images, the suggested approach is real-time multi-object detection based on Enhanced Yolov5-7S on Multi-GPU Vertex. We added one more layer at the top in backbone to increase the resolution of feature extracted image to detect small object and increase the accuracy of model. In terms of speed and accuracy, our proposed approach outperforms the state-of-the-art techniques.

Automated Assessment of Radiographic Bone Loss in the Posterior Maxilla Utilizing a Multi-Object Detection Artificial Intelligence Algorithm

Periodontitis is one of the most prevalent diseases worldwide. The degree of radiographic bone loss can be used to assess the course of therapy or the severity of the disease. Since automated bone loss detection has many benefits, our goal was to develop a multi-object detection algorithm based on artificial intelligence that would be able to detect and quantify radiographic bone loss using standard two-dimensional radiographic images in the maxillary posterior region. This study was conducted by combining three recent online databases and validating the results using an external validation dataset from our organization. There were 1414 images for training and testing and 341 for external validation in the final dataset. We applied a Keypoint RCNN with a ResNet-50-FPN backbone network for both boundary box and keypoint detection. The intersection over union (IoU) and the object keypoint similarity (OKS) were used for model evaluation. The evaluation of the boundary box metrics showed a moderate overlapping with the ground truth, revealing an average precision of up to 0.758. The average precision and recall over all five folds were 0.694 and 0.611, respectively. Mean average precision and recall for the keypoint detection were 0.632 and 0.579, respectively. Despite only using a small and heterogeneous set of images for training, our results indicate that the algorithm is able to learn the objects of interest, although without sufficient accuracy due to the limited number of images and a large amount of information available in panoramic radiographs. Considering the widespread availability of panoramic radiographs as well as the increasing use of online databases, the presented model can be further improved in the future to facilitate its implementation in clinics.

A Recognition Method of Ewe Estrus Crawling Behavior Based on Multi-Target Detection Layer Neural Network.

There are some problems with estrus detection in ewes in large-scale meat sheep farming: mainly, the manual detection method is labor-intensive and the contact sensor detection method causes stress reactions in ewes. To solve the abovementioned problems, we proposed a multi-objective detection layer neural network-based method for ewe estrus crawling behavior recognition. The approach we proposed has four main parts. Firstly, to address the problem of mismatch between our constructed ewe estrus dataset and the YOLO v3 anchor box size, we propose to obtain a new anchor box size by clustering the ewe estrus dataset using the K-means++ algorithm. Secondly, to address the problem of low model recognition precision caused by small imaging of distant ewes in the dataset, we added a 104 × 104 target detection layer, making the total target detection layer reach four layers, strengthening the model's ability to learn shallow information and improving the model's ability to detect small targets. Then, we added residual units to the residual structure of the model, so that the deep feature information of the model is not easily lost and further fused with the shallow feature information to speed up the training of the model. Finally, we maintain the aspect ratio of the images in the data-loading module of the model to reduce the distortion of the image information and increase the precision of the model. The experimental results show that our proposed model has 98.56% recognition precision, while recall was 98.04%, F1 value was 98%, mAP was 99.78%, FPS was 41 f/s, and model size was 276 M, which can meet the accurate and real-time recognition of ewe estrus behavior in large-scale meat sheep farming.

Multi-Object Detection Algorithm in Wind Turbine Nacelles Based on Improved YOLOX-Nano

With more and more wind turbines coming into operation, inspecting wind farms has become a challenging task. Currently, the inspection robot has been applied to inspect some essential parts of the wind turbine nacelle. The detection of multiple objects in the wind turbine nacelle is a prerequisite for the condition monitoring of some essential parts of the nacelle by the inspection robot. In this paper, we improve the original YOLOX-Nano model base on the short monitoring time of the inspected object by the inspection robot and the slow inference speed of the original YOLOX-Nano. The accuracy and inference speed of the improved YOLOX-Nano model are enhanced, and especially, the inference speed of the model is improved by 72.8%, and it performs better than other lightweight network models on embedded devices. The improved YOLOX-Nano greatly satisfies the need for a high-precision, low-latency algorithm for multi-object detection in wind turbine nacelle.

Verification of improved YOLOX model in detection of greenhouse crop organs: Considering tomato as example

Efficient and accurate monitoring crop growth is an important prerequisite for modern greenhouse crop growth control and management. However, it is still challenging to achieve efficient in coordinated detection of tomato organs. Therefore, in this study, a multiobjective detection model was established to detect the blossom, fruit, and fruit color of crop population under dense planting, leaf shading, fruit overlap, and interference from light change in greenhouses. Specifically, sample equalization, random transformation, and mosaic enhancement were applied to identify multiple targets and small-sample images. A new detection network, YOLOX, was used, which introduces a decoupled head structure and redesigns a detection head according to the anchor-free concept to effectively improve the recognition of small-target occlusion. An efficient tomato flower and fruit recognition model was constructed based on a two-step fine-tuned multi-round optimization strategy training. The results showed that the YOLOX target detection model improved the average recognition accuracy (mAP) by 52.49–10.75 %, was lighter, was only 14 % original YOLOv4, and improved the detection speed by 25.53 FPS compared to six main target detection algorithms. It can be used as a method for the collaborative recognition of tomato flowers and fruits. Accordingly, two sets of lightweight detection networks with different performances were designed according to different agricultural scenarios. YOLOXMOB modeled with a small size (only 34.2 MB), high mAP of 81.39 %, and detection speed of 61.85 FPS. It is recommended for use as a deployment model for mobile devices. YOLOXPC, with a deep network layer and strong performance, achieved 86.16 % mAP, and is recommended for monitoring and dispatching computers. This study puts forward a set of multiobjective detection model scheme adapted to the real environment of greenhouse, which provides ideas for the construction of a highly intelligent monitoring system platform for the agricultural Internet of Things.

A Multi-Object Detection Method Based on Adaptive Feature Adjustment of 3D Point Cloud in Indoor Scenes

Due to the complexity and diversity of indoor environment objects and interference occlusions, the accuracy of multi-object target detection based on 3D point cloud is limited. To address this issue, we present a multi-target detection method based on adaptive feature adjustment (AFA) of 3D point cloud. First, our method preprocesses the dataset and constructs a backbone module. Afterwards, our method uses an improved PointNet[Formula: see text] network for feature adaptive learning, where an AFA module is added to learn the influence relationship between point pairs. The proposed method then establishes the relationship between contexts in the local point set area and extracts the feature of point cloud. Using the idea of Hough voting, our method can generate some votes close to the particle. Using these votes to generate proposal, the proposed method adds CBAM attention mechanisms to both modules of voting and proposal, which can fuse the feature information of the channel and expand the receptive field in space. Our method can enhance the important features and weaken the unimportant features, making the extracted features more directional and enhancing the expressiveness of the network. Finally, the generated results are visualized to complete the multi-target detection of 3D point cloud. To verify the effectiveness of our proposed method, two large datasets with real 3D scanning, scanNet2 and SunRGB-D, are used for training the network. The experimental results show that the proposed method can improve the effectiveness of point cloud target detection in indoor scenes, getting a higher detection accuracy.

Retracted: Trajectory Tracking of Soccer Motion Based on Multiobject Detection Algorithm

Retracted: Trajectory Tracking of Soccer Motion Based on Multiobject Detection Algorithm

Computer succoured vaticination of multi-object detection and histogram enhancement in low vision

Computer succoured vaticination of multi-object detection and histogram enhancement in low vision

Computer Succored Vaticination of Multi-Object Detection and Histogram Enhancement in Low Vision

Computer Succored Vaticination of Multi-Object Detection and Histogram Enhancement in Low Vision