Deep SORT Research Articles

스마트 축사내 상황인지 자율이동형 환경센서 개발 및 가축행동 분석에 관한 연구

This study aims to develop a system that predicts the health status of cattle based on behavior patterns and environmental data within a smart barn using an autonomous driving system. Maintaining a unique ID for each cow using only a camera, without external devices (such as RFID tags), is essential. This enables the tracking of behavior patterns such as standing, sitting, and lying for each cow over time. Additionally, environmental data such as temperature and humidity are integrated to comprehensively assess the cows' health conditions. To achieve this, we propose a unique ID retention algorithm that combines object detection using YOLO, tracking with Deep SORT, and re-identification (ReID). Experimental results show that the YOLO + Deep SORT + ReID algorithm delivers the best performance in maintaining unique IDs, and the LSTM-based behavior analysis model demonstrates high accuracy in predicting behavior patterns. This system can serve as an effective tool for real-time prediction of livestock health conditions, such as disease or stress, through comprehensive analysis of environmental data and behavior patterns inside the barn.

Log Volume Measurement and Counting Based on Improved Cascade Mask R-CNN and Deep SORT

Logs require multiple verifications to ensure accurate volume and quantity measurements. Log end detection is a crucial step in measuring log volume and counting logs. Currently, this task primarily relies on the Mask R-CNN instance segmentation model. However, the Feature Pyramid Network (FPN) in Mask R-CNN may compromise accuracy due to feature redundancy during multi-scale fusion, particularly with small objects. Moreover, counting logs in a single image is challenging due to their large size and stacking. To address the above issues, we propose an improved log segmentation model based on Cascade Mask R-CNN. This method uses ResNet for multi-scale feature extraction and integrates a hierarchical Convolutional Block Attention Module (CBAM) to refine feature weights and enhance object emphasis. Then, a Region Proposal Network (RPN) is employed to generate log segmentation proposals. Finally, combined with Deep SORT, the model tracks log ends in video streams and counts the number of logs in the stack. Experiments demonstrate the effectiveness of our method, achieving an average precision (AP) of 82.3, APs of 75.3 for small, APm of 70.9 for medium, and APl of 86.2 for large objects. These results represent improvements of 1.8%, 3.7%, 2.6%, and 1.4% over Mask R-CNN, respectively. The detection rate reached 98.6%, with a counting accuracy of 95%. Compared to manually measured volumes, our method shows a low error rate of 4.07%.

Occluded vehicle tracking by integrating full-scale features and trajectory correction

In order to improve the tracking drift and identity switching (IDs) problems caused by occlusion in vehicle tracking, a method for tracking occluded vehicles that integrates full-scale features and trajectory correction is proposed based on the Deep SORT algorithm. Firstly, a full-scale feature extraction network is introduced to extract the features of different scales of the target and realize adaptive dynamic fusion to enhance the target appearance characteristics. Then, a trajectory correction algorithm is proposed to repair the tracking trajectory during occlusion, and the Kalman filter parameters are updated to reduce the accumulated linear error during occlusion and optimize the target motion characteristics. Finally, the occluded vehicle is tracked by combining appearance features and motion features. The feasibility of the proposed method is verified by ablation experiments and visualization analysis. The experimental results on the KITTI dataset show that compared with the existing typical methods, the proposed method achieves the highest comprehensive score of 78.13 and the lowest number of IDs of 192, which effectively improves the IDs problem in occluded vehicle tracking and improves the robustness of vehicle tracking.

A New Method for Non-Destructive Identification and Tracking of Multi-Object Behaviors in Beef Cattle Based on Deep Learning.

The method proposed in this paper provides theoretical and practical support for the intelligent recognition and management of beef cattle. Accurate identification and tracking of beef cattle behaviors are essential components of beef cattle production management. Traditional beef cattle identification and tracking methods are time-consuming and labor-intensive, which hinders precise cattle farming. This paper utilizes deep learning algorithms to achieve the identification and tracking of multi-object behaviors in beef cattle, as follows: (1) The beef cattle behavior detection module is based on the YOLOv8n algorithm. Initially, a dynamic snake convolution module is introduced to enhance the ability to extract key features of beef cattle behaviors and expand the model's receptive field. Subsequently, the BiFormer attention mechanism is incorporated to integrate high-level and low-level feature information, dynamically and sparsely learning the behavioral features of beef cattle. The improved YOLOv8n_BiF_DSC algorithm achieves an identification accuracy of 93.6% for nine behaviors, including standing, lying, mounting, fighting, licking, eating, drinking, working, and searching, with average 50 and 50:95 precisions of 96.5% and 71.5%, showing an improvement of 5.3%, 5.2%, and 7.1% over the original YOLOv8n. (2) The beef cattle multi-object tracking module is based on the Deep SORT algorithm. Initially, the detector is replaced with YOLOv8n_BiF_DSC to enhance detection accuracy. Subsequently, the re-identification network model is switched to ResNet18 to enhance the tracking algorithm's capability to gather appearance information. Finally, the trajectory generation and matching process of the Deep SORT algorithm is optimized with secondary IOU matching to reduce ID mismatching errors during tracking. Experimentation with five different complexity levels of test video sequences shows improvements in IDF1, IDS, MOTA, and MOTP, among other metrics, with IDS reduced by 65.8% and MOTA increased by 2%. These enhancements address issues of tracking omission and misidentification in sparse and long-range dense environments, thereby facilitating better tracking of group-raised beef cattle and laying a foundation for intelligent detection and tracking in beef cattle farming.

Visual Detection of Traffic Incident through Automatic Monitoring of Vehicle Activities

Intelligent transportation systems (ITSs) derive significant advantages from advanced models like YOLOv8, which excel in predicting traffic incidents in dynamic urban environments. Roboflow plays a crucial role in organizing and preparing image data essential for computer vision models. Initially, a dataset of 1000 images is utilized for training, with an additional 500 images reserved for validation purposes. Subsequently, the Deep Simple Online and Real-time Tracking (Deep-SORT) algorithm enhances scene analyses over time, offering continuous monitoring of vehicle behavior. Following this, the YOLOv8 model is deployed to detect specific traffic incidents effectively. By combining YOLOv8 with Deep SORT, urban traffic patterns are accurately detected and analyzed with high precision. The findings demonstrate that YOLOv8 achieves an accuracy of 98.4%, significantly surpassing alternative methodologies. Moreover, the proposed approach exhibits outstanding performance in the recall (97.2%), precision (98.5%), and F1 score (95.7%), underscoring its superior capability in accurate prediction and analyses of traffic incidents with high precision and efficiency.

Multi-stage deep sorting strategy for retired batteries based on the clustering of static and dynamic features

Echelon utilization is the optimal solution for handling retired lithium-ion batteries. How to ensure the economy and safety of retired batteries for echelon utilization, consistency sorting technology is crucial and valuable. However, batteries are influenced by multiple factors and the mapping of internal and external parameters is complex. Finding external features with discrimination and interpretability, and using effective sorting methods for consistency sorting is still challenged. To address the above problem, a multistage deep sorting strategy for retired batteries based on the clustering of static and dynamic features is proposed in this paper. Firstly, the density-based spatial clustering of applications with noise is used to remove abnormal batteries in the initial sorting stage, which ensures the safety of battery echelon utilization and the efficiency of energy utilization. Secondly, low-correlation static features are selected by Pearson correlation coefficient analysis, and the WK-means clustering algorithm was employed for the secondary sorting of retired batteries. Finally, the sliding window method is used to extract the time-square dynamic features from the dynamic stress test data, and the clustering algorithm is employed to subdivide the retired batteries again with precision. The experimental results indicate that the proposed method in this paper can effectively achieve consistency assessment and sorting of retired batteries. After sorting, the capacity, resistance, and open-circuit voltage of the battery are compactly distributed, resulting in a reduction of their corresponding standard deviations by 36.76 %, 62.97 %, and 71.62 %, respectively. Furthermore, the cut-off time and voltage drop of dynamic profile are reduced by 8.87 % and 65.36 %, respectively.

Behavioral profiling for adaptive video summarization: From generalization to personalization

In today's world of managing multimedia content, dealing with the amount of CCTV footage poses challenges related to storage, accessibility and efficient navigation. To tackle these issues, we suggest an encompassing technique, for summarizing videos that merges machine-learning techniques with user engagement. Our methodology consists of two phases, each bringing improvements to video summarization. In Phase I we introduce a method for summarizing videos based on keyframe detection and behavioral analysis. By utilizing technologies like YOLOv5 for object recognition, Deep SORT for object tracking, and Single Shot Detector (SSD) for creating video summaries. In Phase II we present a User Interest Based Video summarization system driven by machine learning. By incorporating user preferences into the summarization process we enhance techniques with personalized content curation. Leveraging tools such as NLTK, OpenCV, TensorFlow, and the EfficientDET model enables our system to generate customized video summaries tailored to preferences. This innovative approach not only enhances user interactions but also efficiently handles the overwhelming amount of video data on digital platforms. By combining these two methodologies we make progress in applying machine learning techniques while offering a solution to the complex challenges presented by managing multimedia data.

A novel approach for reliable pedestrian trajectory collection with behavior-based trajectory reconstruction for urban surveillance systems

Collecting reliable pedestrian trajectories in pedestrian behavior analysis, trajectories broken by frame sampling and trajectories crossing in multi-object conditions often hinder their performance of existing pedestrian tracking models. Despite attempts to address these issues by performing detection and tracking simultaneously using deep learning algorithms, previous methods still struggle with errors such as mistaking a single pedestrian for multiple pedestrians. We propose a novel approach to efficiently collect and correct pedestrian trajectories with minimized practical errors in multi-object conditions for urban surveillance systems. Our system utilizes a single vision sensor to automatically collects trajectories of multiple pedestrians and employ simple, low-computational algorithms, particularly the Deep simple online real-time tracking (Deep SORT) method, to calibrate the trajectories from tracking-by-detection models. Additionally, our system identifies and merges broken pedestrian trajectories, treating them as potential single trajectories, while considering their spatiotemporal ranges. We evaluate the proposed system by implementing it on real testbed video footage. Our method significantly improves practical errors and achieves more accurate pedestrian trajectories compared to existing models, and exhibits robust characteristics, effectively handling complex situations such as occlusions and crowds.

Traffic Flow Optimization at Toll Plaza Using Proactive Deep Learning Strategies

Global urbanization and increasing traffic volume have intensified traffic congestion throughout transportation infrastructure, particularly at toll plazas, highlighting the critical need to implement proactive transportation infrastructure solutions. Traditional toll plaza management approaches, often relying on manual interventions, suffer from inefficiencies that fail to adapt to dynamic traffic flow and are unable to produce preemptive control strategies, resulting in prolonged queues, extended travel times, and adverse environmental effects. This study proposes a proactive traffic control strategy using advanced technologies to combat toll plaza congestion and optimize traffic management. The approach involves deep learning convolutional neural network models (YOLOv7–Deep SORT) for vehicle counting and an extended short-term memory model for short-term arrival rate prediction. When projected arrival rates exceed a threshold, the strategy proactively activates variable speed limits (VSLs) and ramp metering (RM) strategies during peak hours. The novelty of this study lies in its predictive and adaptive capabilities, ensuring efficient traffic flow management. Validated through a case study at Ravi Toll Plaza Lahore using PTV VISSIMv7, the proposed method reduces queue length by 57% and vehicle delays by 47% while cutting fuel consumption and pollutant emissions by 28.4% and 34%, respectively. Additionally, by identifying the limitations of conventional approaches, this study presents a novel framework alongside the proposed strategy to bridge the gap between theory and practice, making it easier for toll plaza operators and transportation authorities to adopt and benefit from advanced traffic management techniques. Ultimately, this study underscores the importance of integrated and proactive traffic control strategies in enhancing traffic management, minimizing congestion, and fostering a more sustainable transportation system.

An efficient visual servo tracker for herd monitoring by UAV

It is a challenging and meaningful task to carry out UAV-based livestock monitoring in high-altitude (more than 4500 m on average) and cold regions (annual average – 4 °C) on the Qinghai Tibet Plateau. The purpose of artificial intelligence (AI) is to execute automated tasks and to solve practical problems in actual applications by combining the software technology with the hardware carrier to create integrated advanced devices. Only in this way, the maximum value of AI could be realized. In this paper, a real-time tracking system with dynamic target tracking ability is proposed. It is developed based on the tracking-by-detection architecture using YOLOv7 and Deep SORT algorithms for target detection and tracking, respectively. In response to the problems encountered in the tracking process of complex and dense scenes, our work (1) Uses optical flow to compensate the Kalman filter, to solve the problem of mismatch between the target bounding box predicted by the Kalman filter (KF) and the input when the target detection in the current frame is complex, thereby improving the prediction accuracy; (2) Using a low confidence trajectory filtering method to reduce false positive trajectories generated by Deep SORT, thereby mitigating the impact of unreliable detection on target tracking. (3) A visual servo controller has been designed for the Unmanned Aerial Vehicle (UAV) to reduce the impact of rapid movement on tracking and ensure that the target is always within the field of view of the UAV camera, thereby achieving automatic tracking tasks. Finally, the system was tested using Tibetan yaks on the Qinghai Tibet Plateau as tracking targets, and the results showed that the system has real-time multi tracking ability and ideal visual servo effect in complex and dense scenes.

Estimating pedestrian walking speed at street crossings using the YOLOv4 and deep SORT algorithms: Proof of principle

There is evidence that existing standards for signal timing do not provide enough time for many pedestrians to safely cross intersections. Yet, current methods for studying this problem rely on inefficient manual observations. The objective of this work was to determine if the YOLOv4 and Deep SORT computer vision algorithms have the potential to be incorporated into automated measurement systems to measure and compare pedestrian walking speeds at one-stage and two-stage street crossings captured in birds-eye-view video. Walking speed was estimated for 1018 pedestrians at single-stage (591 pedestrians) and two-stage (427 pedestrians) street crossings. Pedestrians in the one-stage crossing were found to be significantly slower than pedestrians who crossed the two-stage crossing in one signal (1.19 ± 0.50 vs. 1.31 ± 0.49 m/s, p < 0.001). This proof of principle study demonstrated that the YOLOv4 and Deep SORT approaches are promising for estimating pedestrian walking speed.

Deep SORT Related Studies

Computer vision is the field of computer science in which computers are made capable to see and recognize like human being. Deep learning is using multiple layers for the purpose of understanding and recognizing various objects. Deep Simple Real Time Tracker is the area in which the objects are tracked in real time from multiple images and videos. Many researchers have contributed to the field and various algorithms have been proposed. The current study presents the deep SORT related studies in which the various algorithms have been presented for the sake of understanding and starting point for the researchers interested in computer vision and deep sorting. The single shot detection, feature extraction, have been explained along with the research conducted. Feature selection and extraction, matching recognition, object tracking through frames have been appended to the current study.

Study on Driver Behavior Pattern in Merging Area under Naturalistic Driving Conditions

To reduce the risk of traffic conflicts in merging area, driver’s behavior pattern was analyzed to provide a theoretical basis for traffic control and conflict risk warning. The unmanned aerial vehicle (UAV) was used to collect the videos in two different types of merging zones: freeway interchange and service area. A vehicle tracking detection model based on YOLOv5 (the fifth version of You Only Look Once) and Deep SORT was constructed to extract traffic flow, speed, vehicle type, and driving trajectory. Acceleration/deceleration distribution and vehicle lane-changing behavior were analyzed. The influence of different vehicle models on vehicle speed and lane-changing behavior was summarized. Based on this data, the mean and standard deviation of velocity, acceleration, and variable acceleration were selected as the characteristic variables for driving style clustering. To avoid redundant information between features, principal component dimensionality reduction was performed, and the dimensionality reduction data was used for K-means and K-means++ clustering to obtain three driving styles. The results show that there are obvious differences in the driving behaviors of vehicles in different types of merging areas, and the characteristics of different areas should be fully considered when conducting traffic conflict warnings.

A reliable unmanned aerial vehicle multi-target tracking system with global motion compensation for monitoring Procapra przewalskii

Procapra przewalskii, which inhabits plateau areas, faces the constant threat of poaching and unpredictable risks that impede its survival. The implementation of a comprehensive, real-time monitoring and tracking system for Procapra przewalskii using artificial intelligence and unmanned aerial vehicle (UAV) technology is crucial to safeguard its existence. Therefore, a UAV multi-object-tracking (MOT) system with global motion compensation (GMC) was proposed in this study. YOLOv7 and Deep SORT were employed for object detection and tracking, respectively. Furthermore, the Kalman filter (KF) in Deep SORT is optimized to enhance the accuracy of object-tracking. Moreover, a novel appearance feature-extraction network (FEN) is introduced to enable more effective multi-scale feature (MSF) extraction. In addition, a GMC module was proposed to align neighboring frames through feature matching. This facilitates the correction of the position of the target in the subsequent frame, mitigating the impact of UAV camera motion on tracking. The results demonstrated the remarkable tracking accuracy of the system. Compared with the Deep SORT model, the proposed system exhibited an increase of 6.4% in MOTA, 2.7% in MOTP, and 7.9% in IDF1. Through a comprehensive evaluation and analysis of real-world tracking scenarios, the system proposed in this study exhibits reliability in complex scenes and holds the potential to significantly enhance the protection of Procapra przewalskii from threats.

Deep Learning Based Multi-Camera Person Tracking and Re-identification System

This research paper presents a novel Deep Learning Based Multi-Camera Person Tracking and Re-identification System designed to enhance security and surveillance in public spaces. Leveraging advanced technologies including Object Detection through YOLO and Deep SORT, the system offers an efficient solution for monitoring and tracing individuals across multiple camera views. Through object detection, individuals are identified within each camera frame, while Deep SORT ensures seamless tracking as they move across different camera perspectives. Additionally, a person re-identification module enhances tracking accuracy by extracting distinctive features and linking individuals across various camera views using unique identifiers. This system represents a significant advancement in surveillance capabilities, contributing to the development of effective and user-friendly surveillance solutions.

Deep Learning-Based Traffic Accident Prediction: An Investigative Study for Enhanced Road Safety

INTRODUCTION: Traffic accidents cause enormous loss of life as well as property, which is a global concern. Effective accident prediction is essential for raising road safety and reducing the effects of accidents. To increase traffic safety, a deep learning-based technique for predicting accidents was developed in this research study.&#x0D; OBJECTIVES: It gathers a large amount of data on elements including weather, road features, volume of traffic, and past accident reports. The dataset goes through pre-processing, such as normalization, to ensure that the scales of the input characteristics are uniform. Normalizing the gathered dataset ensures consistent scaling for the input features during the data processing step. This process enables efficient model training and precise forecasting. In order to track and examine the movement patterns of automobiles, people, and other relevant entities, real-time tracking and monitoring technologies, such as the deep sort algorithm, are also employed.&#x0D; METHODS: The model develops a thorough grasp of the traffic situation by incorporating this tracking data with the dataset. Convolutional Neural Networks (CNN), in particular, are utilized in this research for feature extraction and prediction. CNNs capture crucial road characteristics by extracting spatial features from images or spatial data. With its insights into improved road safety, this study advances the prediction of traffic accidents.&#x0D; RESULTS: A safer transport infrastructure could result from the developed deep learning-based strategy, which has the potential to enable pre-emptive interventions, enhance traffic management, and eventually reduce the frequency and severity of traffic accidents.&#x0D; CONCLUSION: The proposed CNN demonstrates superior accuracy when compared to the existing method.

Enhanced YOLOv5s + DeepSORT method for highway vehicle speed detection and multi-sensor verification

Addressing the need for vehicle speed measurement in traffic surveillance, this study introduces an enhanced scheme combining YOLOv5s detection with Deep SORT tracking. Tailored to the characteristics of highway traffic and vehicle features, the dataset data augmentation process was initially optimized. To improve the detector’s recognition capabilities, the Swin Transformer Block module was incorporated, enhancing the model’s ability to capture local regions of interest. CIoU loss was employed as the loss function for the vehicle detection network, accelerating model convergence and achieving higher regression accuracy. The Mish activation function was utilized to reduce computational overhead and enhance convergence speed. The structure of the Deep SORT appearance feature extraction network was modified, and it was retrained on a vehicle re-identification dataset to mitigate identity switches due to obstructions. Subsequently, using known references in the image such as lane markers and contour labels, the transformation from image pixel coordinates to actual coordinates was accomplished. Finally, vehicle speed was measured by computing the average of instantaneous speeds across multiple frames. Through radar and video Multi-Sensor Verification, the experimental results show that the mean Average Precision (mAP) for target detection consistently exceeds 90%. The effective measurement distance for speed measurement is around 140 m, with the absolute speed error generally within 1–8 km/h, meeting the accuracy requirements for speed measurement. The proposed model is reliable and fully applicable to highway scenarios.

Mining the Micro-Trajectory of Two-Wheeled Non-Motorized Vehicles Based on the Improved YOLOx

Two-wheeled non-motorized vehicles (TNVs) have become the primary mode of transportation for short-distance travel among residents in many underdeveloped cities in China due to their convenience and low cost. However, this trend also brings corresponding risks of traffic accidents. Therefore, it is necessary to analyze the driving behavior characteristics of TNVs through their trajectory data in order to provide guidance for traffic safety. Nevertheless, the compact size, agile steering, and high maneuverability of these TNVs pose substantial challenges in acquiring high-precision trajectories. These characteristics complicate the tracking and analysis processes essential for understanding their movement patterns. To tackle this challenge, we propose an enhanced You Only Look Once Version X (YOLOx) model, which incorporates a median pooling-Convolutional Block Attention Mechanism (M-CBAM). This model is specifically designed for the detection of TNVs, and aims to improve accuracy and efficiency in trajectory tracking. Furthermore, based on this enhanced YOLOx model, we have developed a micro-trajectory data mining framework specifically for TNVs. Initially, the paper establishes an aerial dataset dedicated to the detection of TNVs, which then serves as a foundational resource for training the detection model. Subsequently, an augmentation of the Convolutional Block Attention Mechanism (CBAM) is introduced, integrating median pooling to amplify the model’s feature extraction capabilities. Subsequently, additional detection heads are integrated into the YOLOx model to elevate the detection rate of small-scale targets, particularly focusing on TNVs. Concurrently, the Deep Sort algorithm is utilized for the precise tracking of vehicle targets. The process culminates with the reconstruction of trajectories, which is achieved through a combination of video stabilization, coordinate mapping, and filtering denoising techniques. The experimental results derived from our self-constructed dataset reveal that the enhanced YOLOx model demonstrates superior detection performance in comparison to other analogous methods. The comprehensive framework accomplishes an average trajectory recall rate of 85% across three test videos. This significant achievement provides a reliable method for data acquisition, which is essential for investigating the micro-level operational mechanisms of TNVs. The results of this study can further contribute to the understanding and improvement of traffic safety on mixed-use roads.

System for Tracking Aircraft Ground Movement Utilizing Artificial Intelligence

Using a virtual ATC tower is one aspect of the airport digitization initiative, which aims to cut costs. Technology for tracking and detecting aircraft is required to guarantee the security of the deployment of virtual ATC towers. The invention of visual artificial intelligence that can recognize and follow aircraft ground movement in an airport is presented in this study. One feature that should be developed in this study is the capacity to issue a warning if two aircraft are in close proximity to one another. The techniques consist of a coordinate system conversion from pixels to meters for aircraft separation computation, the Deep SORT object tracking algorithm, and the YOLOv4 object recognition algorithm that has been trained using Image Dehazing Filter. Next, a fair-condition recorded airport video is used to validate the model. With a mean average precision score of 95.92%, the trained YOLOv4 model was able to track every aircraft in the video, and with an error of 5.09%, the aircraft separation warning system functioned as expected. An airport’s possible use of an aircraft ground movement tracker was demonstrated by the constructed model.

Ship imaging trajectory extraction via an aggregated you only look once (YOLO) model

Maritime traffic community has paid a huge amount of focuses to establish maritime intelligent transportation infrastructure for the purpose of enhancing maritime traffic safety and efficiency. Maritime surveillance video is considered as a type of fundamental data sources for establishing intelligent maritime transportation infrastructure towards smart ship era. To that end, the study proposes an aggregated deep learning model-supported ship imaging trajectory extraction framework. The proposed framework starts by detecting ships from maritime images via a novel You Only Look Once (YOLO) model. More specifically, the proposed ship trajectory extraction framework obtains ship positions in a frame-by-frame manner via the proposed poly-YOLO module. Then, the proposed model maps ship positions in neighboring consecutive maritime images via an Enhanced Deep Sort (EDS) module. Experimental results suggest that the proposed ship trajectory extraction model achieves satisfactory performance due to that the average values of index multiple-object tracking accuracy (MOTa), recall rate (Rid) and index aggregated detection accuracy (Aggid) are larger than 89% (which outperform the comparison algorithms). The study can help varied maritime traffic participants obtain accurate on-site traffic situations in the smart ship era.