Deep SORT Research Articles

Navigation System for Personal Mobility Vehicles Following a Cluster of Pedestrians in a Corridor Using Median of Candidate Vectors Observer

In recent years, personal mobility vehicles have been required to operate autonomously in places with numerous pedestrians. A navigation method using a single human-following scheme is used to avoid collision with pedestrians. However, in many cases, a single human-following method cannot be successfully used for guidance. In crowded places, pedestrians do not always keep walking in the desired direction a user wants to go, and the vehicle must change the target pedestrian frequently. Instead of following a single pedestrian, we propose a method for the vehicle to follow a cluster of pedestrians for stable and robust following. First, the pedestrians around the vehicle are detected by multiple RGB-D cameras, and the pedestrians are tracked using YOLO and Deep Sort. Pedestrians are classified according to their walking direction, and the cluster of pedestrians walking toward the goal is selected and followed. However, the position of pedestrian is sometimes lost in occlusions and the accuracy of the walking direction depends on the distance and pose detected by the sensors. A notable problem is that the cluster of pedestrians is unstable in the cluster following; therefore, a median of candidate vectors (MCV) observer is used to remove outliers caused by observation errors. The proposed method is applied to a scenario involving pedestrians walking toward an elevator hall in a building, and its effectiveness is verified through experiments.

Fusion of Deep Sort and Yolov5 for Effective Vehicle Detection and Tracking Scheme in Real-Time Traffic Management Sustainable System

In recent years, advancements in sustainable intelligent transportation have emphasized the significance of vehicle detection and tracking for real-time traffic flow management on the highways. However, the performance of existing methods based on deep learning is still a big challenge due to the different sizes of vehicles, occlusions, and other real-time traffic scenarios. To address the vehicle detection and tracking issues, an intelligent and effective scheme is proposed which detects vehicles by You Only Look Once (YOLOv5) with a speed of 140 FPS, and then, the Deep Simple Online and Real-time Tracking (Deep SORT) is integrated into the detection result to track and predict the position of the vehicles. In the first phase, YOLOv5 extracts the bounding box of the target vehicles, and in second phase, it is fed with the output of YOLOv5 to perform the tracking. Additionally, the Kalman filter and the Hungarian algorithm are employed to anticipate and track the final trajectory of the vehicles. To evaluate the effectiveness and performance of the proposed algorithm, simulations were carried out on the BDD100K and PASCAL datasets. The proposed algorithm surpasses the performance of existing deep learning-based methods, yielding superior results. Finally, the multi-vehicle detection and tracking process illustrated that the precision, recall, and mAP are 91.25%, 93.52%, and 92.18% in videos, respectively.

Improved YOLOv7 Intelligent Environment Perception Algorithm Based on Loss Function

To address the difficulty of environment perception in unmanned driving technology, an improved intelligent environment perception model based on YOLOv7 loss function is proposed. Simultaneously, the algorithm of deep sort target tracking is added for the sake of perceiving the surrounding environment through the camera to ensure the safety of the driver. And by modifying the loss term of the aspect ratio to the distinction between the predicted width and the target width, the fitting speed of the model can be effectively improved. Focal Loss is introduced to optimize the problem with the sample imbalance inside the target bounding box regression and reduce the weight proportion of numerous anchor boxes with less overlap between the predicted target bounding box and the true target bounding one to the overall model, such that the model centers more upon overlapping targets. The data applied to this article is BDD100K. The map of this algorithm reaches 60.3 %, which reaches 2.2 percentage points exceeding the baseline. Tests indicate that the raised algorithm attains certain advancements in the intelligent perception model of unmanned driving.

Improve multi-object tracking using yolov5 and deep-sort for Chinese yam counting model

<p>Accurate counting information is important for crop yield and quality. Obtaining yield estimation of Chinese Yam is critical to improving productivity. Referring to the method of counting pedestrian flow in surveillance video and its accuracy, a method for counting the number of Chinese Yams harvested in the field is proposed by improving YOLOv5s detection combined with Deep-SORT tracking. In order to improve the recognition effect of the detector, the attention module CBAM is fused with the Neck part of the YOLOv5s network to improve the feature extraction ability of the network; CIoU Loss is used instead of GIoU Loss as the target bounding box regression loss function to speed up the bounding box regression rate while improving positioning Accuracy; use DIoU-NMS to replace NMS to improve the missed detection problem when the target is crowded. Adjust the structure of the Deep SORT appearance feature extraction network and retrain on the yam re-identification dataset to reduce the identity switching caused by target occlusion. Connect the improved YOLOv5s detector and Deep SORT, and set a virtual detection line in the video to count the number of yams. The experimental results show that the number of yams can be counted more accurately. Compared with the original algorithm, the improved YOLOv5s has an average accuracy rate of 2.1 percentage points. Combined with Deep-SORT tracking, the counting statistics accuracy rate reaches 92.7%.<strong></strong></p>

임베디드 환경에서 SSD MobileNet V2 및 Deep SORT 기반 객체 추적을 이용한 피플 카운터

임베디드 환경에서 SSD MobileNet V2 및 Deep SORT 기반 객체 추적을 이용한 피플 카운터

Broiler Mobility Assessment via a Semi-Supervised Deep Learning Model and Neo-Deep Sort Algorithm.

Mobility is a vital welfare indicator that may influence broilers' daily activities. Classical broiler mobility assessment methods are laborious and cannot provide timely insights into their conditions. Here, we proposed a semi-supervised Deep Learning (DL) model, YOLOv5 (You Only Look Once version 5), combined with a deep sort algorithm conjoined with our newly proposed algorithm, neo-deep sort, for individual broiler mobility tracking. Initially, 1650 labeled images from five days were employed to train the YOLOv5 model. Through semi-supervised learning (SSL), this narrowly trained model was then used for pseudo-labeling 2160 images, of which 2153 were successfully labeled. Thereafter, the YOLOv5 model was fine-tuned on the newly labeled images. Lastly, the trained YOLOv5 and the neo-deep sort algorithm were applied to detect and track 28 broilers in two pens and categorize them in terms of hourly and daily travel distances and speeds. SSL helped in increasing the YOLOv5 model's mean average precision (mAP) in detecting birds from 81% to 98%. Compared with the manually measured covered distances of broilers, the combined model provided individual broilers' hourly moved distances with a validation accuracy of about 80%. Eventually, individual and flock-level mobilities were quantified while overcoming the occlusion, false, and miss-detection issues.

Abnormal Behavior Fish and Population Detection Method based on Deep Learning

This paper presents a detection model of fish with abnormal behavior and their number based on YOLO v8 and Deep Sort algorithm. The method firstly uses computer and acquisition system to monitor and analyze the fish behavior in real time, and can effectively detect the abnormal behavior of fish, such as abnormal swimming trajectory and abnormal residence time. The main work of this paper is to preprocess fish behavior videos, including video segmentation, data enhancement and other operations, and use data enhancement technology to improve the problem of fish occlusion in data set, which is easy to cause model false detection. Then, YOLO v8 and Deep Sort algorithm were used for multi-target tracking and target detection to extract the key information of fish behavior. Finally, through the analysis and comparison of the extracted information, the detection of fish with abnormal behavior and its quantity are realized. The experimental results show that the method proposed in this paper can effectively detect the abnormal behavior of fish, has high accuracy and real-time, and has certain application and popularization value.

A Ship Tracking and Speed Extraction Framework in Hazy Weather Based on Deep Learning

Obtaining ship navigation information from maritime videos can significantly improve maritime supervision efficiency and enable timely safety warnings. Ship detection and tracking are essential technologies for mining video information. However, current research focused on these advanced vision tasks in maritime supervision is not sufficiently comprehensive. Taking into account the application of ship detection and tracking technology, this study proposes a deep learning-based ship speed extraction framework under the haze environment. First, a lightweight convolutional neural network (CNN) is used to remove haze from images. Second, the YOLOv5 algorithm is used to detect ships in dehazed marine images, and a simple online and real-time tracking method with a Deep association metric (Deep SORT) is used to track ships. Then, the ship’s displacement in the images is calculated based on the ship’s trajectory. Finally, the speed of the ships is estimated by calculating the mapping relationship between the image space and real space. Experiments demonstrate that the method proposed in this study effectively reduces haze interference in maritime videos, thereby enhancing the image quality while extracting the ship’s speed. The mean squared error (MSE) for multiple scenes is 0.3 Kn on average. The stable extraction of ship speed from the video achieved in this study holds significant value in further ensuring the safety of ship navigation.

Identification and Tracking of Groups of People using Object Detection and Object Tracking

Object detection is one of the most important areas in the fields of Data Science and Computer Vision. In this paper, we present a novel approach to identifying and tracking groups of people, couples, and individuals in videos by using deep learning-based object detection and object tracking techniques along with a proposed grouping algorithm. In this approach, transfer learning is applied on YOLO v3 model for the detection of people in video frames, and Deep SORT is applied for tracking each detected person throughout the video. Results obtained from person detection and person tracking were used by the proposed grouping algorithm to identify and track groups, couples, and individuals who are appearing in input videos. Our proposed grouping algorithm is based on the proximity between each individual and the time duration that proximity is maintained for. It also considers how to identify and track groups, when people are moving within the groups. This approach was evaluated using CCTV videos captured from the restaurant domain and it was able to perform the group detection and tracking tasks successfully with a precision of 0.7083, recall of 0.7906 and F1 score of 0.7471.

Suspicious Behavior Detection with Temporal Feature Extraction and Time-Series Classification for Shoplifting Crime Prevention.

The rise in crime rates in many parts of the world, coupled with advancements in computer vision, has increased the need for automated crime detection services. To address this issue, we propose a new approach for detecting suspicious behavior as a means of preventing shoplifting. Existing methods are based on the use of convolutional neural networks that rely on extracting spatial features from pixel values. In contrast, our proposed method employs object detection based on YOLOv5 with Deep Sort to track people through a video, using the resulting bounding box coordinates as temporal features. The extracted temporal features are then modeled as a time-series classification problem. The proposed method was tested on the popular UCF Crime dataset, and benchmarked against the current state-of-the-art robust temporal feature magnitude (RTFM) method, which relies on the Inflated 3D ConvNet (I3D) preprocessing method. Our results demonstrate an impressive 8.45-fold increase in detection inference speed compared to the state-of-the-art RTFM, along with an F1 score of 92%,outperforming RTFM by 3%. Furthermore, our method achieved these results without requiring expensive data augmentation or image feature extraction.

Efficient Roundabout Supervision: Real-Time Vehicle Detection and Tracking on Nvidia Jetson Nano

In recent years, a significant number of people in Morocco have been commuting daily to Casablanca, the country’s economic capital. This heavy traffic flow has led to congestion and accidents during certain times of the day as the city’s roads cannot handle the high volume of vehicles passing through. To address this issue, it is essential to expand the infrastructure based on accurate traffic-flow data. In collaboration with the municipality of Bouskoura, a neighboring city of Casablanca, we proposed installing a smart camera on the primary route connecting the two cities. This camera would enable us to gather accurate statistics on the number and types of vehicles crossing the road, which can be used to adapt and redesign the existing infrastructure. We implemented our system using the YOLOv7-tiny object detection model to detect and classify the various types of vehicles (such as trucks, cars, motorcycles, and buses) crossing the main road. Additionally, we used the Deep SORT tracking method to track each vehicle appearing on the camera and to provide the total number of each class for each lane, as well as the number of vehicles passing from one lane to another. Furthermore, we deployed our solution on an embedded system, specifically the Nvidia Jetson Nano. This allowed us to create a compact and efficient system that is capable of a real-time processing of camera images, making it suitable for deployment in various scenarios where limited resources are required. Deploying our solution on the Nvidia Jetson Nano showed promising results, and we believe that this approach could be applied in similar traffic-surveillance projects to provide accurate and reliable data for better decision-making.

Artificial Intelligence Aided Crowd Analytics in Rail Transit Station

Crowd analysis and management is a key area of study for transit agencies seeking to optimize their operations and to facilitate safety risk management activities. Key features of crowd analytics include passenger flow volume, crowd density, and walking speed. This study proposes a generalized artificial intelligence (AI)-based crowd analytics model framework for rail transit stations, by analyzing and visualizing crowd analysis data from video records of high-density crowds. Specifically, we propose a generalized AI-aided methodological framework (AI-Crowd) for calculating flow volume, crowd density, and walking speed. You Only Look Once (YOLO) and Deep SORT are integrated into the model framework to detect and track each individual’s dynamic position. Camera calibration is utilized to transform detected trajectories into a real-world coordinate system. Methods for calculating crowd dynamic metrics are formulated based on the data. To validate the model framework, several video records from a platform scenario at a major rail transit station are used. The model’s pedestrian counting accuracy can reach 95% and the fundamental diagrams of density–speed are shown to be consistent with empirical studies. Further crowd analysis of a stair scenario and a transferring passage scenario using the proposed model framework shows some differentiations in walking behavior. The methodology has further practical applications, such as monitoring social distancing.

Wrong Way Vehicle Detection in Single and Double Lane

Wrong-way driving is one of the primary causes of traffic jams and accidents globally. It is possible to identify vehicles going the wrong direction, which lessens accidents and traffic congestion. Surveillance footage has become an important source of data due to the accessibility of less priced cameras and the expanding use of real-time traffic management systems. In this paper, we propose a technique for automatically identifying automobiles moving against traffic. Our system uses the You Only Look Once (CNN) algorithm to recognize and track vehicles from video inputs and the centroid tracking method to determine each vehicle's orientation inside a given region of interest (ROI) in order to identify vehicles traveling in the wrong direction. It functions in three steps. The Deep sort tracking method is particularly good in detecting and tracking objects, and the centroid tracking technique can effectively monitor the direction of travel. Experiments with a variety of traffic films show that the suggested method can detect and identify wrong-way moving vehicles in a variety of lighting and weather scenarios. The interface of the system is quite simple and easy to use.

Drill-Rep: Repetition counting for automatic shot hole depth recognition based on combined deep learning-based model

In order to solve the problems of low efficiency and dependability of human experience in shot hole depth recognition by manual interpretation of the massive drilling videos, this paper proposed a repetition counting framework from the perspective of representation object motion, and firstly studied and realized the automatic shot hole depth recognition by adopting the proposed framework by combining multiple deep learning-based models. Firstly, the periodic motion characteristics of the objects in the drilling video were analyzed, and the motion extraction model of the representative object was established based on YOLOv5 and Deep SORT algorithm to obtain the spatiotemporal sequence. Then, the sequence noise and redundant information were filtered according to the spatiotemporal relationship between the drill pipes and workers, to obtain a coherent, smooth, and clean motion sequence. On this basis, a drill pipe motion counting model was established by using the YOLOv5 network to detect the temporal boundary and repetition number of the drill pipe motion. Finally, combined with the pipe motion extraction model and counting model, the comprehensive reliable recognition method of shot hole depth was presented. The shot hole depth recognition trial was carried out on 1029 drilling videos from the Bureau of Geophysical Prospecting INC. The proposed method achieved 90.5% recognition accuracy (Acc) and 0.027 mean absolute error (MAE) with 64 frames per second (FPS) and 0.39 average recognition time ratio (Rt). Therefore, the proposed repetition counting framework solved the problem of efficient and intelligent drilling depth recognition of massive drilling videos and thus provided a new idea for repetition counting.

Recognition and Counting of Apples in a Dynamic State Using a 3D Camera and Deep Learning Algorithms for Robotic Harvesting Systems.

Recognition and 3D positional estimation of apples during harvesting from a robotic platform in a moving vehicle are still challenging. Fruit clusters, branches, foliage, low resolution, and different illuminations are unavoidable and cause errors in different environmental conditions. Therefore, this research aimed to develop a recognition system based on training datasets from an augmented, complex apple orchard. The recognition system was evaluated using deep learning algorithms established from a convolutional neural network (CNN). The dynamic accuracy of the modern artificial neural networks involving 3D coordinates for deploying robotic arms at different forward-moving speeds from an experimental vehicle was investigated to compare the recognition and tracking localization accuracy. In this study, a Realsense D455 RGB-D camera was selected to acquire 3D coordinates of each detected and counted apple attached to artificial trees placed in the field to propose a specially designed structure for ease of robotic harvesting. A 3D camera, YOLO (You Only Look Once), YOLOv4, YOLOv5, YOLOv7, and EfficienDet state-of-the-art models were utilized for object detection. The Deep SORT algorithm was employed for tracking and counting detected apples using perpendicular, 15°, and 30° orientations. The 3D coordinates were obtained for each tracked apple when the on-board camera in the vehicle passed the reference line and was set in the middle of the image frame. To optimize harvesting at three different speeds (0.052 ms-1, 0.069 ms-1, and 0.098 ms-1), the accuracy of 3D coordinates was compared for three forward-moving speeds and three camera angles (15°, 30°, and 90°). The mean average precision (mAP@0.5) values of YOLOv4, YOLOv5, YOLOv7, and EfficientDet were 0.84, 0.86, 0.905, and 0.775, respectively. The lowest root mean square error (RMSE) was 1.54 cm for the apples detected by EfficientDet at a 15° orientation and a speed of 0.098 ms-1. In terms of counting apples, YOLOv5 and YOLOv7 showed a higher number of detections in outdoor dynamic conditions, achieving a counting accuracy of 86.6%. We concluded that the EfficientDet deep learning algorithm at a 15° orientation in 3D coordinates can be employed for further robotic arm development while harvesting apples in a specially designed orchard.

Detecting Human Falls in Poor Lighting: Object Detection and Tracking Approach for Indoor Safety

Falls are one the leading causes of accidental death for all people, but the elderly are at particularly high risk. Falls are severe issue in the care of those elderly people who live alone and have limited access to health aides and skilled nursing care. Conventional vision-based systems for fall detection are prone to failure in conditions with low illumination. Therefore, an automated system that detects falls in low-light conditions has become an urgent need for protecting vulnerable people. This paper proposes a novel vision-based fall detection system that uses object tracking and image enhancement techniques. The proposed approach is divided into two parts. First, the captured frames are optimized using a dual illumination estimation algorithm. Next, a deep-learning-based tracking framework that includes detection by YOLOv7 and tracking by the Deep SORT algorithm is proposed to perform fall detection. On the Le2i fall and UR fall detection (URFD) datasets, we evaluate the proposed method and demonstrate the effectiveness of fall detection in dark night environments with obstacles.

Camera-Based Local and Global Target Detection, Tracking, and Localization Techniques for UAVs

Multiple-object detection, localization, and tracking are desirable in many areas and applications, as the field of deep learning has developed and has drawn the attention of academics in computer vision, having a plethora of networks now achieving excellent accuracy in detecting multiple objects in an image. Tracking and localizing objects still remain difficult processes which require significant effort. This work describes an optical camera-based target detection, tracking, and localization solution for Unmanned Aerial Vehicles (UAVs). Based on the well-known network YOLOv4, a custom object detection model was developed and its performance was compared to YOLOv4-Tiny, YOLOv4-608, and YOLOv7-Tiny. The target tracking algorithm we use is based on Deep SORT, providing cutting-edge tracking. The proposed localization approach can accurately determine the position of ground targets identified by the custom object detection model. Moreover, an implementation of a global tracker using localization information from up to four UAV cameras at a time. Finally, a guiding approach is described, which is responsible for providing real-time movement commands for the UAV to follow and cover a designated target. The complete system was evaluated in Gazebo with up to four UAVs utilizing Software-In-The-Loop (SITL) simulation.

Violation Behavior Detection for Non-motor Vehicles

<p>Non-motor vehicles are widely used in the urban and rural transportation system for their portability, but the related violations also occur frequently and are difficult to be supervised intelligently, considering their colossal quantity, various styles, and small volumes. To solve this problem, this paper presents a non-motor vehicle violation detection algorithm with efficient target detection and deliberate logical calculation. A target detection network with high speed and accuracy is constructed firstly by fusing two different types of attention mechanism. Specifically, the Squeeze-and-Excitation Network is employed to optimize the extraction of local features, which can effectively reduce the error rate of target detection. Meanwhile, the Transformer Network is adopted to strengthen the extraction of global features, which can improve the target location performance for target tracking in high-density scenarios. As the global and local features are integrated with attention mechanism, the proposed network can accurately identify and locate the numerous small targets in real-time, and avoid identity switching caused by target occlusion. Finally, the violation of non-motor vehicles is recognized in real-time by constructing the logical calculation between the target features and their motion trajectories. Experiments on datasets show that the detection accuracy of the proposed algorithm is better than the current mainstream algorithms, especially the accuracy of small targets such as Head and Helmet is higher than 92.2%. And our ID Switch is dropped by more than 60% compared with the classical Deep SORT algorithm. In real-life scenarios, the proposed algorithm also shows excellent accuracy and real-time performance for non-motor vehicle violations.</p> <p> </p>

Detection and Tracking of Broiler Flock Movements in The Chicken Coop using YOLO

Observation of the movement of broilers in the chicken coop is done to monitor the welfare and health condition of broilers. Currently, observing broiler flock activity in the chicken coop is generally still done conventionally, with manual observations made by farmers. But on a large scale, this observation method takes a lot of time and manpower, and is subjective. Therefore, an automatic observation system is needed that continuously monitors broiler activity, so as to increase the efficiency of farmer resources and reduce operational costs for observing broiler activity. This study developed an automatic detection and tracking system for broiler chicken movements using You Only Look Once, Version 4 (YOLOv4) as the base model, Yolo Weights as the Transfer Learning Pretrained Model, and Deep Sort as the Tracker Model. For comparison of base models, use Single Shot Multibox Detector (SSD), You Only Look Once, Version 3 (YOLOv3), You Only Look Once, Version 4 - tiny (YOLOv4-tiny). For comparison, the Network Model uses MobileNet and MobileNet v2. For comparison of Transfer Learning Pretrained Model using Caffe Model Weights and Tensorflow Weights. For comparison of Tracker Models using Centroid Tracker, Centroid Kalman Filter Tracker, Simple Online and Realtime Tracking (SORT) and Intersection over Union (IOU). The results showed that Model You Only Look Once, Version 4 (YOLOv4) with Transfer Learning Pretrained Model = Yolo Weights, and Tracker Model = Deep SORT was able to detect and track the most chicken herds in cages compared to others, with the number of broilers detected as many as 17.

Novel multiple object tracking method for yellow feather broilers in a flat breeding chamber based on improved YOLOv3 and deep SORT

Novel multiple object tracking method for yellow feather broilers in a flat breeding chamber based on improved YOLOv3 and deep SORT