Multi-Object Tracking Accuracy Research Articles

Improved DeepSORT-Based Object Tracking in Foggy Weather for AVs Using Sematic Labels and Fused Appearance Feature Network.

The presence of fog in the background can prevent small and distant objects from being detected, let alone tracked. Under safety-critical conditions, multi-object tracking models require faster tracking speed while maintaining high object-tracking accuracy. The original DeepSORT algorithm used YOLOv4 for the detection phase and a simple neural network for the deep appearance descriptor. Consequently, the feature map generated loses relevant details about the track being matched with a given detection in fog. Targets with a high degree of appearance similarity on the detection frame are more likely to be mismatched, resulting in identity switches or track failures in heavy fog. We propose an improved multi-object tracking model based on the DeepSORT algorithm to improve tracking accuracy and speed under foggy weather conditions. First, we employed our camera-radar fusion network (CR-YOLOnet) in the detection phase for faster and more accurate object detection. We proposed an appearance feature network to replace the basic convolutional neural network. We incorporated GhostNet to take the place of the traditional convolutional layers to generate more features and reduce computational complexities and costs. We adopted a segmentation module and fed the semantic labels of the corresponding input frame to add rich semantic information to the low-level appearance feature maps. Our proposed method outperformed YOLOv5 + DeepSORT with a 35.15% increase in multi-object tracking accuracy, a 32.65% increase in multi-object tracking precision, a speed increase by 37.56%, and identity switches decreased by 46.81%.

Vehicle Behavior Discovery and Three-Dimensional Object Detection and Tracking Based on Spatio-Temporal Dependency Knowledge and Artificial Fish Swarm Algorithm.

In complex traffic environments, 3D target tracking and detection are often occluded by various stationary and moving objects. When the target is occluded, its apparent characteristics change, resulting in a decrease in the accuracy of tracking and detection. In order to solve this problem, we propose to learn the vehicle behavior from the driving data, predict and calibrate the vehicle trajectory, and finally use the artificial fish swarm algorithm to optimize the tracking results. The experiments show that compared with the CenterTrack method, the proposed method improves the key indicators of MOTA (Multi-Object Tracking Accuracy) in 3D object detection and tracking on the nuScenes dataset, and the frame rate is 26 fps.

A Long-Term Video Tracking Method for Group-Housed Pigs

Pig tracking provides strong support for refined management in pig farms. However, long and continuous multi-pig tracking is still extremely challenging due to occlusion, distortion, and motion blurring in real farming scenarios. This study proposes a long-term video tracking method for group-housed pigs based on improved StrongSORT, which can significantly improve the performance of pig tracking in production scenarios. In addition, this research constructs a 24 h pig tracking video dataset, providing a basis for exploring the effectiveness of long-term tracking algorithms. For object detection, a lightweight pig detection network, YOLO v7-tiny_Pig, improved based on YOLO v7-tiny, is proposed to reduce model parameters and improve detection speed. To address the target association problem, the trajectory management method of StrongSORT is optimized according to the characteristics of the pig tracking task to reduce the tracking identity (ID) switching and improve the stability of the algorithm. The experimental results show that YOLO v7-tiny_Pig ensures detection applicability while reducing parameters by 36.7% compared to YOLO v7-tiny and achieving an average video detection speed of 435 frames per second. In terms of pig tracking, Higher-Order Tracking Accuracy (HOTA), Multi-Object Tracking Accuracy (MOTP), and Identification F1 (IDF1) scores reach 83.16%, 97.6%, and 91.42%, respectively. Compared with the original StrongSORT algorithm, HOTA and IDF1 are improved by 6.19% and 10.89%, respectively, and Identity Switch (IDSW) is reduced by 69%. Our algorithm can achieve the continuous tracking of pigs in real scenarios for up to 24 h. This method provides technical support for non-contact pig automatic monitoring.

Pattern Mining-Based Pig Behavior Analysis for Health and Welfare Monitoring.

The increasing popularity of pigs has prompted farmers to increase pig production to meet the growing demand. However, while the number of pigs is increasing, that of farm workers has been declining, making it challenging to perform various farm tasks, the most important among them being managing the pigs' health and welfare. This study proposes a pattern mining-based pig behavior analysis system to provide visualized information and behavioral patterns, assisting farmers in effectively monitoring and assessing pigs' health and welfare. The system consists of four modules: (1) data acquisition module for collecting pigs video; (2) detection and tracking module for localizing and uniquely identifying pigs, using tracking information to crop pig images; (3) pig behavior recognition module for recognizing pig behaviors from sequences of cropped images; and (4) pig behavior analysis module for providing visualized information and behavioral patterns to effectively help farmers understand and manage pigs. In the second module, we utilize ByteTrack, which comprises YOLOx as the detector and the BYTE algorithm as the tracker, while MnasNet and LSTM serve as appearance features and temporal information extractors in the third module. The experimental results show that the system achieved a multi-object tracking accuracy of 0.971 for tracking and an F1 score of 0.931 for behavior recognition, while also highlighting the effectiveness of visualization and pattern mining in helping farmers comprehend and manage pigs' health and welfare.

Vehicle Multi-Object Detection and Tracking Algorithm Based on Improved You Only Look Once 5s Version and DeepSORT

The increasing popularity of vehicles has led to traffic congestion and frequent traffic accidents. Intelligent transportation technology is an effective solution to this problem. In order to improve the accuracy and effectiveness of vehicle detection and tracking, this paper combined the improved YOLOv5s model with the optimized DeepSORT tracking algorithm to detect and track vehicles on traffic roads. Firstly, in the detection model of YOLOv5s, the Attention-based Intra-scale Feature Interaction (AIFI) module is introduced to detect vehicles more quickly and accurately. Secondly, the Kalman filtering (KF) algorithm of DeepSORT is optimized to improve the accuracy of predictions of the vehicle state by using the width to replace the length-to-width ratio of the vehicle prediction box in the original KF algorithm. Finally, in the re-recognition network of DeepSORT, the original Convolutional Neural Network (CNN) model is replaced by an improved ResNet36 as the backbone network for feature extraction. The experimental results show that, compared with the original algorithm, when examining the performance of the improved algorithm in terms of target detection, the recall rate, average accuracy (mAP), and detection speed, are increased by 7.7%, 15.5%, and 14.2%, respectively; in terms of multi-object tracking performance, such as multi-object tracking precision (MOTP) and multi-object tracking accuracy (MOTA), improvements of 14.84% and 9.62%, respectively, are obtained and the total number of times a trajectory is fragmented (Frag) is reduced by 32.52%.These results indicate that the proposed algorithm can meet the requirements of accuracy, real-time detection, and stable vehicle detection and tracking on traffic roads.

Efficient Sampling of Two-Stage Multi-Person Pose Estimation and Tracking from Spatiotemporal

Tracking the articulated poses of multiple individuals in complex videos is a highly challenging task due to a variety of factors that compromise the accuracy of estimation and tracking. Existing frameworks often rely on intricate propagation strategies and extensive exchange of flow data between video frames. In this context, we propose a spatiotemporal sampling framework that addresses the degradation of frames at the feature level, offering a simple yet effective network block. Our spatiotemporal sampling mechanism empowers the framework to extract meaningful features from neighboring video frames, thereby optimizing the accuracy of pose detection in the current frame. This approach results in significant improvements in running latency. When evaluated on the COCO dataset and the mixed dataset, our approach outperforms other methods in terms of average precision (AP), recall rate (AR), and acceleration ratio. Specifically, we achieve a 3.7% increase in AP, a 1.77% increase in AR, and a speedup of 1.51 times compared to mainstream state-of-the-art (SOTA) methods. Furthermore, when evaluated on the PoseTrack2018 dataset, our approach demonstrates superior accuracy in multi-object tracking, as measured by the multi-object tracking accuracy (MOTA) metric. Our method achieves an impressive 11.7% increase in MOTA compared to the prevailing SOTA methods.

Robust individual pig tracking

The locations of pigs in the group housing enable activity monitoring and improve animal welfare. Vision-based methods for tracking individual pigs are noninvasive but have low tracking accuracy owing to long-term pig occlusion. In this study, we developed a vision-based method that accurately tracked individual pigs in group housing. We prepared and labeled datasets taken from an actual pig farm, trained a faster region-based convolutional neural network to recognize pigs’ bodies and heads, and tracked individual pigs across video frames. To quantify the tracking performance, we compared the proposed method with the global optimization (GO) method with the cost function and the simple online and real-time tracking (SORT) method on four additional test datasets that we prepared, labeled, and made publicly available. The predictive model detects pigs’ bodies accurately, with F1-scores of 0.75 to 1.00, on the four test datasets. The proposed method achieves the largest multi-object tracking accuracy (MOTA) values at 0.75, 0.98, and 1.00 for three test datasets. In the remaining dataset, the proposed method has the second-highest MOTA of 0.73. The proposed tracking method is robust to long-term occlusion, outperforms the competitive baselines in most datasets, and has practical utility in helping to track individual pigs accurately.

Bidirectional Temporal Pose Matching for Tracking

Multi-person pose tracking is a challenging task. It requires identifying the human poses in each frame and matching them across time. This task still faces two main challenges. Firstly, sudden camera zooming and drastic pose changes between adjacent frames may result in mismatched poses between them. Secondly, the time relationships modeled by most existing methods provide insufficient information in scenarios with long-term occlusion. In this paper, to address the first challenge, we propagate the bounding boxes of the current frame to the previous frame for pose estimation, and match the estimated results with the previous ones, which we call the Backward Temporal Pose-Matching (BTPM) module. To solve the second challenge, we design an Association Across Multiple Frames (AAMF) module that utilizes long-term temporal relationships to supplement tracking information lost in the previous frames as a Re-identification (Re-id) technique. Specifically, we select keyframes with a fixed step size in the videos and label other frames as general frames. In the keyframes, we use the BTPM module and the AAMF module to perform tracking. In the general frames, we propagate poses in the previous frame to the current frame for pose estimation and association, which we call the Forward Temporal Pose-Matching (FTPM) module. If the pose association fails, the current frame will be set as a keyframe, and tracking will be re-performed. In the PoseTrack 2018 benchmark tests, our method shows significant improvements over the baseline methods, with improvements of 2.1 and 1.1 in mean Average Precision (mAP) and Multi-Object Tracking Accuracy (MOTA), respectively.

A computer vision approach to monitor activity in commercial broiler chickens using trajectory-based clustering analysis

To monitor changes in broiler behaviour related to animal health and welfare, farmers typically observe their flocks using manual observation. However, due to the labour intensive and continuous aspect of this task, the analysis of broiler behaviour could be automated using camera technology. This paper proposes a proof-of-concept camera surveillance system based on the automated detection and tracking of broilers to monitor activity bouts using unsupervised 2D trajectory clustering. Firstly, a convolutional neural network-based detector was trained and tested on our labelled dataset which resulted in a precision, recall and f score of 0.98, 0.90 and 0.94, respectively. Using a tracking-by-detection approach, the proposed system was able to track chickens across video frames with a multi-object tracking accuracy of 74.7%. A component-based feature saliency Gaussian mixture model (CFSGMM) was subsequently generated and applied to objectively cluster the trajectories based on their spatiotemporal information. Nineteen features were extracted from the trajectories, representing both static and dynamic characteristics of broiler movement, and three activity classes were identified: ‘least active/resting,’ ‘active’ and ‘highly-active.’ The proposed method was validated on one-minute monocular video sequences. CFSGMM was applied to cluster 2D trajectories relating to broiler activity bouts within the commercial rearing environment with an agreement ranging from 6.0 to 99.7% when compared to human observation. We demonstrate the potential of the computer vision system to monitor overt, short-term changes in broiler activity associated with on-farm events and discuss the opportunities of leveraging the technology to monitor longer term changes in welfare state. It is anticipated that further development of the detection and tracking systems will improve the performance of the trajectory clustering method.

AttMOT: Improving Multiple-Object Tracking by Introducing Auxiliary Pedestrian Attributes.

Multiobject tracking (MOT) is a fundamental problem in computer vision with numerous applications, such as intelligent surveillance and automated driving. Despite the significant progress made in MOT, pedestrian attributes, such as gender, hairstyle, body shape, and clothing features, which contain rich and high-level information, have been less explored. To address this gap, we propose a simple, effective, and generic method to predict pedestrian attributes to support general reidentification (Re-ID) embedding. We first introduce attribute multi-object tracking (AttMOT), a large, highly enriched synthetic dataset for pedestrian tracking, containing over 80k frames and six million pedestrian identity switches (IDs) with different times, weather conditions, and scenarios. To the best of authors' knowledge, AttMOT is the first MOT dataset with semantic attributes. Subsequently, we explore different approaches to fuse Re-ID embedding and pedestrian attributes, including attention mechanisms, which we hope will stimulate the development of attribute-assisted MOT. The proposed method attribute-assisted method (AAM) demonstrates its effectiveness and generality on several representative pedestrian MOT benchmarks, including MOT17 and MOT20, through experiments on the AttMOT dataset. When applied to the state-of-the-art trackers, AAM achieves consistent improvements in multi-object tracking accuracy (MOTA), higher order tracking accuracy (HOTA), association accuracy (AssA), IDs, and IDF1 scores. For instance, on MOT17, the proposed method yields a + 1.1 MOTA, + 1.7 HOTA, and + 1.8 IDF1 improvement when used with FairMOT. To further encourage related research, we release the data and code at https://github.com/HengLan/AttMOT.

3D Vehicle Object Tracking Algorithm Based on Bounding Box Similarity Measurement

Effectively extracting features from a discrete point cloud is necessary for three-dimensional (3D) vehicle tracking. However, a point cloud data set is large and sparsely distributed, hindering the success of vehicle-tracking algorithms. To solve this problem, this paper proposes a 3D vehicle object tracking algorithm based on bounding box similarity measurement. The algorithm includes state prediction, temporal association, trajectory management, state update, and other processes. Also incorporated is a vehicle object temporal association method based on a siamese encoder. The bounding box is encoded into a high-dimensional space, the feature distance is calculated as the time series association cost, and triplet loss was introduced to urge the encoder to learn the geometric similarity of the truth matching box. A 3D Kalman filter and greedy matching are used to effectuate 3D vehicle object tracking algorithm. The experimental results using a KITTI multi-object tracking dataset show that the proposed algorithm can achieve good vehicle tracking performance. In the case of frame loss of point cloud in the frequency reduction simulation, compared with the benchmark method AB3DMOT, the average multi-object tracking accuracy ( AMOTA) and the average multi-object tracking accuracy ( AMOTP) of the improved method are increased by 2.62% and 1.41% respectively, and the performance is better in the frame loss scene.

Regulation of Meat Duck Activeness through Photoperiod Based on Deep Learning.

The regulation of duck physiology and behavior through the photoperiod holds significant importance for enhancing poultry farming efficiency. To clarify the impact of the photoperiod on group-raised duck activeness and quantify duck activeness, this study proposes a method that employs a multi-object tracking model to calculate group-raised duck activeness. Then, duck farming experiments were designed with varying photoperiods as gradients to assess this impact. The constructed multi-object tracking model for group-raised ducks was based on YOLOv8. The C2f-Faster-EMA module, which combines C2f-Faster with the EMA attention mechanism, was used to improve the object recognition performance of YOLOv8. Furthermore, an analysis of the tracking performance of Bot-SORT, ByteTrack, and DeepSORT algorithms on small-sized duck targets was conducted. Building upon this foundation, the duck instances in the images were segmented to calculate the distance traveled by individual ducks, while the centroid of the duck mask was used in place of the mask regression box's center point. The single-frame average displacement of group-raised ducks was utilized as an intuitive indicator of their activeness. Farming experiments were conducted with varying photoperiods (24L:0D, 16L:8D, and 12L:12D), and the constructed model was used to calculate the activeness of group-raised ducks. The results demonstrated that the YOLOv8x-C2f-Faster-EMA model achieved an object recognition accuracy (mAP@50-95) of 97.9%. The improved YOLOv8 + Bot-SORT model achieved a multi-object tracking accuracy of 85.1%. When the photoperiod was set to 12L:12D, duck activeness was slightly lower than that of the commercial farming's 24L:0D lighting scheme, but duck performance was better. The methods and conclusions presented in this study can provide theoretical support for the welfare assessment of meat duck farming and photoperiod regulation strategies in farming.

A Multiple Object Tracker with Spatiotemporal Memory Network

Target tracking is an important application of unmanned aerial vehicles (UAVs). The template is the identity of the target and has a great impact on the performance of target tracking. Most methods only keep the latest template of the target, which is intuitive and convenient but has poor ability to resist the change of target appearance, especially to reidentify a target that has disappeared for a long time. In this paper, we propose a practical multiobject tracking (MOT) method, which uses historical information of targets for better adapting to appearance variations during tracking. To preserve the spatial-temporal information of the target, we introduce a memory pool to store masked feature maps at different moments, and precise masks are generated by a segmentation network. Meanwhile, we fuse the feature maps at different moments by calculating the pixel-level similarity between the current feature map and the masked historical feature maps. Benefiting from the powerful segmentation features and the utilization of historical information, our method can generate more accurate bounding boxes of the targets. Extensive experiments and comparisons with many trackers on MOTS, MOT17, and MOT20 demonstrate that our method is competitive. The ablation study showed that the introduction of memory improves the multiobject tracking accuracy (MOTA) by 2.1.

FishTrack: Multi-object tracking method for fish using spatiotemporal information fusion

Tracking the fish is a key step in analyzing fish behavior, evaluating their health levels, and warning of abnormal water quality, so it is of significant importance for intelligent monitoring in fish farming. However, multi-object tracking for fish is a very challenging task due to foam occlusion in the factory tank, high individual similarity, and fast-paced motion of fish. To address the above issues, an online multi-fish tracking model called FishTrack is proposed with 3 branches of target detection, track prediction, and re-identification, which establishes the motion model and appearance model for fish simultaneously, thus achieving the online multi-fish tracking. It is a complete Encoder-Decoder structure and the Pyramid Vision Transformer (PVT) is adopted as the backbone network to extract multi-level features. Then an Encoder is specially designed to encode the historical information of the positions of fish, and automatically update its spatiotemporal information in an autoregressive fashion, to perform the fusion of the spatiotemporal information of fish targets and avoid manual selection of spatiotemporal features. Finally, a parallel dual-Decoder is used to decode the motion and appearance features of fish to reduce the interference of the two optimization directions during the joint model training. The motion cues are first used in linear assignment to reduce the interference of occlusion and deformation issues, and when the motion model fails to track, the appearance cues can recover the identity to deal with long occlusion. The experiments on the established multiple fish tracking dataset showed that the higher order tracking accuracy (HOTA) reached 71.4%, the multi-object tracking accuracy (MOTA) reached 94.8%, the most tracked ratio (MT) reached 93.3%, and the identification F1 score (IDF1) reached 82.5%. The results show that FishTrack can solve the problem of tracking accuracy decrease caused by foam occlusion and fish deformation, and save the inference time by sharing features, increasing the robustness of online multi-fish tracking in factory farming.

SKT-MOT and DyTracker: A Multiobject Tracking Dataset and a Dynamic Tracker for Speed Skating Video

Speed skating serves as a significant application domain for multiobject tracking (MOT), presenting unique challenges such as frequent occlusion, highly similar appearances, and motion blur. To address these challenges, this paper constructs an MOT dataset called SKT-MOT for speed skating and analyzes the shortcomings of existing datasets and methods. Accordingly, we propose a dynamic MOT method called DyTracker. The method builds upon the DeepSORT baseline and enhances three key modules. At the global level, we design the track dynamic management (TDM) algorithm. In the motion branch, a novel metric is proposed to evaluate occlusion and Kalman filter dynamic update (KFDU) is implemented. In the appearance branch, we account for the difference in human posture and propose the feature dynamic selection and updating (FDSU) strategy. This makes our DyTracker flexible and efficient to achieve a multiobject tracking accuracy (MOTA) of 93.70% and identification F1 (IDF1) score of 92.39% on SKT-MOT, which is a significant advantage over existing SOTA methods. To validate the generalization of our proposed module, two dynamic update modules are inserted into other methods and validated on the public dataset MOT17, and the accuracy is generally improved by 0.2%–0.6%.

Improving the network architecture of YOLOv7 to achieve real-time grading of canola based on kernel health

The occurrence of heated and immature canola kernels caused by excessive drying and frost damage is undesired by grain buyers due to lower oil yield and diminished market value. The current grading process is visually examining each kernel's endosperm color and counting the damaged seeds. As this process is time-consuming, laborious, and prone to errors, this study proposes an automated grading technique based on object detection, multi-object tracking, and counting. The detection task was achieved via an improved YOLOv7 network (YOLOv7_ours) that was modified to increase its performance in accurately identifying small objects by adding two convolutional block attention modules in the neck region and decrease its computational complexity (cost) and size by substituting convolutional layers with ghost layers in all the Efficient Layer Aggregation Networks modules, and in the Spatial Pyramid Pooling Cross Stage Partial module present in YOLOv7. The weights of the trained network were fed to the ByteTrack multiple object tracker to track the detections frame-by-frame in a video feed. The unique identities generated by the tracker for each detected object of interest were then used to count the number of defects using a line cross algorithm. The mean average precision (mAP@0.5) obtained after training the YOLOv7_ours model was 1.02% better and its cost and size were 32.1% and 37.1% lower than the baseline YOLOv7 model. In a test video, the overall model achieved a multi-object tracking accuracy and counting accuracy of 84.8% and 93.9%, respectively. This three-stage model can be readily deployed in an edge device for accurate and real-time grading of canola kernels by grain buyers.

Tracking Passengers and Baggage Items Using Multiple Overhead Cameras at Security Checkpoints

We introduce a novel framework to track multiple objects in overhead camera videos for airport checkpoint security scenarios where targets correspond to passengers and their baggage items. We propose a self-supervised learning (SSL) technique to provide the model information about instance segmentation uncertainty from overhead images. Our SSL approach improves object detection by employing a test-time data augmentation and a regression-based, rotation-invariant pseudo-label refinement technique. Our pseudo-label generation method provides multiple geometrically transformed images as inputs to a convolutional neural network (CNN), regresses the augmented detections generated by the network to reduce localization errors, and then clusters them using the mean-shift algorithm. The self-supervised detector model is used in a single-camera tracking algorithm to generate temporal identifiers for the targets. Our method also incorporates a multiview trajectory association mechanism to maintain consistent temporal identifiers as passengers travel across camera views. An evaluation of detection, tracking, and association performances on videos obtained from multiple overhead cameras in a realistic airport checkpoint environment demonstrates the effectiveness of the proposed approach. Our results show that self-supervision improves object detection accuracy by up to <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$42\%$</tex-math> </inline-formula> without increasing the inference time of the model. Our multicamera association method achieves up to <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$89\%$</tex-math> </inline-formula> multiobject tracking accuracy with an average computation time of less than <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$15$</tex-math> </inline-formula> ms.

Dynamic fry counting based on multi-object tracking and one-stage detection

Fry counting is an essential task in aquaculture that provides valuable information for feedstuff feeding, culture density adjustment, and fish economic benefit estimation. However, most current computer vision-based fry counting methods are designed for statically counting, with a fixed number of fry in a container. As a result, these methods are not suitable for dynamic scenarios, such as flowing water, where the effects of factors like water velocity and uneven lighting can cause errors in counting. To address this issue and improve accuracy, we propose a dynamic fry counting method capable of dealing with large-number fry counting. Specifically, we regard fry counting as a Multiple Object Tracking (MOT) problem based on Tracking-By-Detection (TBD) framework. Firstly, a dataset of streaming fry is constructed for training the detection model, and videos with varying numbers of fry are manually counted for counting tests. Secondly, due to the highly similar appearance of fry, Simple Online and Realtime Tracking (SORT) is combined with You Only Look Once version 5 Nano (YOLOv5-Nano) to perform fry tracking. To achieve stable tracking of fry movement, the SORT algorithm is improved based on multi-matching and trajectory recovery, reaching 82.6 % multi-object tracking accuracy. Finally, fry counting is realized based on the intersection of a counting line and moving trajectories. The results indicate that the proposed counting method achieves 96.4 % accuracy and can be implemented on CPU at 5–10 frames per second (fps) and GPU at 15–35 fps. In conclusion, the proposed method automatically counts the number of fry in videos with higher speed and accuracy, facilitating automated fishery farming.

YOLO-BYTE: An efficient multi-object tracking algorithm for automatic monitoring of dairy cows

Dairy cows tracking is an essential means to obtain their behavioral information, real-time position, activity data, and health status. A multi-object tracking method (YOLO-BYTE) is proposed to address the problem of missed detection and false detection caused by complex environments in cow individual detection and tracking. The method improves upon the YOLO v7 Backbone network feature extraction module by adding a Self-Attention and Convolution mixed module (ACmix) to account for the uneven spatial distribution and target scale variation of the cows. Additionally, in order to reduce the number of model parameters, an improved lightweight Spatial Pyramid Pooling Cross Stage Partial Connections (SPPCSPC-L) module is adopted to reduce model complexity. At the same time, the state parameters in the Kalman filter are improved by directly predicting the width and height information of the tracking boxes, so as to improve the ByteTrack algorithm to make tracking boxes matching the cows more precisely and accurately. Experimental conducted on the dairy cow object detection and multi-object tracking dataset show that the proposed YOLO-BYTE model achieves a Precision (P) of 97.3% in the dairy cow target detection dataset, with an improved Recall (R) and Average Precision (AP) by 1.1% compared to the original algorithm, and an 18% reduction in model parameters. Moreover, the proposed method demonstrated significant improvements in High Order Tracking Accuracy (HOTA), Multi-Object Tracking Accuracy (MOTA), and Identification F1 (IDF1) by 4.4%, 6.1%, and 3.8%, respectively, compared to the original model, with a decrease of 37.5% in Identity Switch (IDS). The tracker runs in a real-time manner with an average analysis speed of 47 fps. Hence, it is demonstrated that the proposed approach is capable of effective multi-object tracking of dairy cows in natural scenes and provides technical support for non-contact dairy cow automatic monitoring.

Comparing State-of-the-Art Deep Learning Algorithms for the Automated Detection and Tracking of Black Cattle.

Effective livestock management is critical for cattle farms in today's competitive era of smart modern farming. To ensure farm management solutions are efficient, affordable, and scalable, the manual identification and detection of cattle are not feasible in today's farming systems. Fortunately, automatic tracking and identification systems have greatly improved in recent years. Moreover, correctly identifying individual cows is an integral part of predicting behavior during estrus. By doing so, we can monitor a cow's behavior, and pinpoint the right time for artificial insemination. However, most previous techniques have relied on direct observation, increasing the human workload. To overcome this problem, this paper proposes the use of state-of-the-art deep learning-based Multi-Object Tracking (MOT) algorithms for a complete system that can automatically and continuously detect and track cattle using an RGB camera. This study compares state-of-the-art MOTs, such as Deep-SORT, Strong-SORT, and customized light-weight tracking algorithms. To improve the tracking accuracy of these deep learning methods, this paper presents an enhanced re-identification approach for a black cattle dataset in Strong-SORT. For evaluating MOT by detection, the system used the YOLO v5 and v7, as a comparison with the instance segmentation model Detectron-2, to detect and classify the cattle. The high cattle-tracking accuracy with a Multi-Object Tracking Accuracy (MOTA) was 96.88%. Using these methods, the findings demonstrate a highly accurate and robust cattle tracking system, which can be applied to innovative monitoring systems for agricultural applications. The effectiveness and efficiency of the proposed system were demonstrated by analyzing a sample of video footage. The proposed method was developed to balance the trade-off between costs and management, thereby improving the productivity and profitability of dairy farms; however, this method can be adapted to other domestic species.