Object Tracking Method Research Articles

Motion-Aware Self-Supervised RGBT Tracking with Multi-Modality Hierarchical Transformers

Supervised RGBT (SRGBT) tracking tasks need both expensive and time-consuming annotations. Therefore, the implementation of Self-Supervised RGBT (SSRGBT) tracking methods has become increasingly important. Straightforward SSRGBT tracking methods use pseudo-labels for tracking, but inaccurate pseudo-labels can lead to object drift, which severely affects tracking performance. This article proposes a self-supervised RGBT object tracking method (S2OTFormer) to bridge the gap between tracking methods supervised under pseudo-labels and ground truth labels. Firstly, to provide more robust appearance features for motion cues, we introduce a multi-modality hierarchical transformer (MHT) module for feature fusion. This module allocates weights to both modalities and strengthens the expressive capability of the MHT module through multiple nonlinear layers to fully utilize the complementary information of the two modalities. Secondly, in order to solve the problems of motion blur caused by camera motion and inaccurate appearance information caused by pseudo-labels, we introduce a motion-aware mechanism (MAM). The MAM extracts the average motion vectors from the previous multi-frame search frame features and constructs the consistency loss with the motion vectors of the current search frame features. The motion vectors of inter-frame objects are obtained by reusing the inter-frame attention map to predict coordinate positions. Finally, to further reduce the effect of inaccurate pseudo-labels, we propose an Attention-Based Multi-Scale Enhancement Module. By introducing cross-attention to achieve more precise and accurate object tracking, this module overcomes the receptive field limitations of traditional CNN tracking heads. We demonstrate the effectiveness of S2OTFormer on four large-scale public datasets through extensive comparisons as well as numerous ablation experiments. The source code is available at https://github.com/LiShenglana/S2OTFormer .

Real-Time Detection and Localization of Weeds in Dictamnus dasycarpus Fields for Laser-Based Weeding Control

Traditional Chinese medicinal herbs have strict environmental requirements and are highly susceptible to weed damage, while conventional herbicides can adversely affect their quality. Laser weeding has emerged as an effective method for managing weeds in precious medicinal herbs. This technique allows for precise weed removal without chemical residue and protects the surrounding ecosystem. To maximize the effectiveness of this technology, accurate detection and localization of weeds in the medicinal herb fields are crucial. This paper studied seven species of weeds in the field of Dictamnus dasycarpus, a traditional Chinese medicinal herb. We propose a lightweight YOLO-Riny weed-detection algorithm and develop a YOLO-Riny-ByteTrack Multiple Object Tracking method by combining it with the ByteTrack algorithm. This approach enables accurate detection and localization of weeds in medicinal fields. The YOLO-Riny weed-detection algorithm is based on the YOLOv7-tiny network, which utilizes the FasterNet lightweight structure as the backbone, incorporates a lightweight upsampling operator, and adds structure reparameterization to the detection network for precise and rapid weed detection. The YOLO-Riny-ByteTrack Multiple Object Tracking method provides quick and accurate feedback on weed identification and location, reducing redundant weeding and saving on laser weeding costs. The experimental results indicate that (1) YOLO-Riny improves detection accuracy for Digitaria sanguinalis and Acalypha australis, ultimately amounting to 5.4% and 10%, respectively, compared to the original network. It also diminishes the model size by 2 MB and inference time by 10 ms, making it more suitable for resource-constrained edge devices. (2) YOLO-Riny-ByteTrack enhances Multiple Object Tracking accuracy by 3%, reduces ID switching by 14 times, and improves overall tracking accuracy by 3.4%. The proposed weed-detection and localization method for Dictamnus dasycarpus offers fast detection speed, high localization accuracy, and stable tracking, supporting the implementation of laser weeding during the seedling stage of Dictamnus dasycarpus.

Cobots at the Museum: The Role of Robotics in Digitisation

In the last decade, the Natural History Museum, UK (NHM), has been at the forefront of the digitisation of natural history collections, with almost six million of its 80 million specimens digitised. This momentous undertaking has led to numerous innovations on how to optimise digitisation workflows. One avenue that is currently being explored is the use of collaborative robots—cobots. Since acquiring a Techman TM5 900 robotic arm in 2023 (Scott et al. 2023), we have been experimenting with its capabilities. Experiments began with simple pick-and-place tasks, using artificial specimens. Next, we focused on two use-cases, based on the digitisation of shark teeth and pinned-insects. Both shark teeth and pinned insects are in abundance at the NHM, making their manual digitisation a tedious task. Currently, we have trained the cobot to pick up a specimen, move it elsewhere to photograph and scan the specimen, then move it back to its original position or to a new place. Thus far, this has all been coordinate-based. Focusing on pinned insect specimens, we have now begun training deep learning models to perform segmentation, classification, and tracking tasks on images and manually taken videos. Segmentation and classification tasks range from distinguishing specimens from one another within drawers, to classifying different pins, labels, and insects. Meanwhile, object tracking methods are utilised to track labels from videos taken around the specimen. By tracking different labels simultaneously from multiple frames, we can combine the views of the labels in order to obtain a full picture for each label (for example, using tools described in Salili-James et al. 2022). Thus far, our machine learning pipelines have proved successful, for example, with F1 scores of 96–98% to classify and segment insects and to locate pin heads from dorsal views. Soon, we will be establishing workflows that integrate computer vision (CV) and machine learning (ML) techniques directly with the robotic arm, with pipelines that could be applied to different datasets, and that can significantly enhance efficiency. Broadly, these pipelines can be split into four sections: Specimen Identification: CV/ML to locate individual specimens or certain parts of specimens e.g., pinheads within pinned insects. Handling: With custom grippers, the cobot can delicately pick up, move, and place specimens, to and from photography stations for high quality scanning. Imaging & Scanning: The cobot will scan and photograph specimens. As the camera moves around the specimen, ML is used to segment and track the specimen labels to image them from the optimal views. A built-in Optical Character Recognition process can also be integrated to perform automatic transcription from here. Identifiers: This step requires the cobot to attach identifier labels to specimens or drawers, after locating the optimal position to do this, using CV/ML. Specimen Identification: CV/ML to locate individual specimens or certain parts of specimens e.g., pinheads within pinned insects. Handling: With custom grippers, the cobot can delicately pick up, move, and place specimens, to and from photography stations for high quality scanning. Imaging & Scanning: The cobot will scan and photograph specimens. As the camera moves around the specimen, ML is used to segment and track the specimen labels to image them from the optimal views. A built-in Optical Character Recognition process can also be integrated to perform automatic transcription from here. Identifiers: This step requires the cobot to attach identifier labels to specimens or drawers, after locating the optimal position to do this, using CV/ML. In this talk, we will discuss the progress of the NHM’s cobot research and explore the future of robotics for the digitisation of natural history collections.

Transformer object tracking algorithm based on spatiotemporal template focal attention

Existing visual object tracking methods only use the target area of the first frame as a template, which makes it easy to fail in fast-changing and complex backgrounds. To solve this problem, this paper proposes a Transformer-based object tracking algorithm that focuses on the target focus information in the template and dynamically updates the template features. In order to reduce the interference of background information on attention, this algorithm uses a sparse Transformer module to achieve the interaction of feature information; a template focus attention module is also proposed to focus on the dynamic template features and the initial template features to retain the highly reliable feature information in the initial template. Experimental results show that the success rate and accuracy of this algorithm in the 0TB100 benchmark test are and respectively 70.9%, 91.6%and the success rate and accuracy are improved by 4.11%and respectively compared with the similar template update algorithm STARK 3.27%. This algorithm effectively addresses the limitations of existing visual object tracking methods and improves the accuracy and robustness of tracking.

Fusion 3D object tracking method based on region and point cloud registration

Fusion 3D object tracking method based on region and point cloud registration

Vpit: real-time embedded single object 3D tracking using voxel pseudo images

In this paper, we propose a novel voxel-based 3D single object tracking (3D SOT) method called Voxel Pseudo Image Tracking (VPIT). VPIT is the first method that uses voxel pseudo images for 3D SOT. The input point cloud is structured by pillar-based voxelization, and the resulting pseudo image is used as an input to a 2D-like Siamese SOT method. The pseudo image is created in the Bird’s-eye View (BEV) coordinates; and therefore, the objects in it have constant size. Thus, only the object rotation can change in the new coordinate system and not the object scale. For this reason, we replace multi-scale search with a multi-rotation search, where differently rotated search regions are compared against a single target representation to predict both position and rotation of the object. Experiments on KITTI [1] Tracking dataset show that VPIT is the fastest 3D SOT method and maintains competitive Success and Precision values. Application of a SOT method in a real-world scenario meets with limitations such as lower computational capabilities of embedded devices and a latency-unforgiving environment, where the method is forced to skip certain data frames if the inference speed is not high enough. We implement a real-time evaluation protocol and show that other methods lose most of their performance on embedded devices; while, VPIT maintains its ability to track the object.

Detecting and tracking moving objects in defocus blur scenes

Object tracking stands as a cornerstone challenge within computer vision, with blurriness analysis representing a burgeoning field of interest. Among the various forms of blur encountered in natural scenes, defocus blur remains significantly underexplored. To bridge this gap, this article introduces the Defocus Blur Video Object Tracking (DBVOT) dataset, specifically crafted to facilitate research in visual object tracking under defocus blur conditions. We conduct a comprehensive performance analysis of 18 state-of-the-art object tracking methods on this unique dataset. Additionally, we propose a selective deblurring framework based on Deblurring Auxiliary Learning Net (DID-Anet), innovatively designed to tackle the complexities of defocus blur. This framework integrates a novel defocus blurriness metric for the smart deblurring of video frames, thereby enhancing the efficacy of tracking methods in defocus blur scenarios. Our extensive experimental evaluations underscore the significant advancements in tracking accuracy achieved by incorporating our proposed framework with leading tracking technologies.

A Hybrid Neuromorphic Object Tracking and Classification Framework for Real-Time Systems.

Deep learning inference that needs to largely take place on the "edge" is a highly computational and memory intensive workload, making it intractable for low-power, embedded platforms such as mobile nodes and remote security applications. To address this challenge, this article proposes a real-time, hybrid neuromorphic framework for object tracking and classification using event-based cameras that possess desirable properties such as low-power consumption (5-14 mW) and high dynamic range (120 dB). Nonetheless, unlike traditional approaches of using event-by-event processing, this work uses a mixed frame and event approach to get energy savings with high performance. Using a frame-based region proposal method based on the density of foreground events, a hardware-friendly object tracking scheme is implemented using the apparent object velocity while tackling occlusion scenarios. The frame-based object track input is converted back to spikes for TrueNorth (TN) classification via the energy-efficient deep network (EEDN) pipeline. Using originally collected datasets, we train the TN model on the hardware track outputs, instead of using ground truth object locations as commonly done, and demonstrate the ability of our system to handle practical surveillance scenarios. As an alternative tracker paradigm, we also propose a continuous-time tracker with C++ implementation where each event is processed individually, which better exploits the low latency and asynchronous nature of neuromorphic vision sensors. Subsequently, we extensively compare the proposed methodologies to state-of-the-art event-based and frame-based methods for object tracking and classification, and demonstrate the use case of our neuromorphic approach for real-time and embedded applications without sacrificing performance. Finally, we also showcase the efficacy of the proposed neuromorphic system to a standard RGB camera setup when simultaneously evaluated over several hours of traffic recordings.

A robust method for multi object tracking in autonomous ship navigation systems

A novel Maritime Multi-Object Tracking method is proposed, combining a deep learning-based object detector with target association algorithms to achieve robust sea-surface multi-object tracking. Specifically, the Multi-Object Tracking employs You Only Look Once version 7 detector for object detection. In the data association part, a module for onboard camera motion compensation is developed, a maritime dynamic spatial information-based intersection-over-union is presented as a similarity metric, and a progressive refinement cascade matching strategy is designed to enhance the tracker's sea-surface multi-object tracking capabilities. The Jari Maritime Tracking Dataset is utilised to validate the effectiveness performance of the proposed method. Experimental results demonstrate that compared to the earlier process, the proposed method exhibits a significant enhancement in multiple object tracking accuracy, with an increase of 27.8% and an achieved score of 81.3. In particular, it reduces the number of identifications switching and missed targets, achieving holistically preferable performance. Meanwhile, the speed of the proposed Multi-Object Tracking fulfils the engineering application requirements for an autonomous ship navigation system.

A Diffusion Neural Network-Enhanced Object Tracking Approach Under Sports Scenarios

Object tracking for motion scenes is a common research concern in field of computer vision. Its goal is to accurately track targets in different time periods and predict their future states by utilizing the motion information in video sequences. However, traditional target-tracking methods in motion scenes often face challenges such as target blur, occlusion, and changes in lighting. To deal with this issue, this paper proposes a diffusion neural network-enhanced object-tracking approach under sports scenarios. In order to further improve tracking performance, the diffusion convolution operation is introduced, which propagates features at different time steps to enhance the modeling ability of target motion. Then, suitable influencing factors are selected based on motion scene object feature parameters. Finally, a target tracking method is established by integrating these two methods. In the experiment, we used a large number of real motion scene datasets to evaluate the proposed method. The experimental results show that by comparing with traditional moving object tracking methods, the proposal achieves significant improvement in tracking accuracy and methodology robustness. In addition, we also conducted stability experiments, proving that this method has good stability for models with varying kernel numbers.

Review and Analysis of RGBT Single Object Tracking Methods: A Fusion Perspective

Visual tracking is a fundamental task in computer vision with significant practical applications in various domains, including surveillance, security, robotics, and human-computer interaction. However, it may face limitations in visible light data, such as low-light environments, occlusion, and camouflage, which can significantly reduce its accuracy. To cope with these challenges, researchers have explored the potential of combining the visible and infrared modalities to improve tracking performance. By leveraging the complementary strengths of visible and infrared data, RGB-infrared fusion tracking has emerged as a promising approach to address these limitations and improve tracking accuracy in challenging scenarios. In this article, we present a review on RGB-infrared fusion tracking. Specifically, we categorize existing RGBT tracking methods into four categories based on their underlying architectures, feature representations, and fusion strategies, namely feature decoupling based method, feature selecting based method, collaborative graph tracking method, and traditional fusion method. Furthermore, we provide a critical analysis of their strengths, limitations, representative methods, and future research directions. To further demonstrate the advantages and disadvantages of these methods, we present a review of publicly available RGBT tracking datasets and analyze the main results on public datasets. Moreover, we discuss some limitations in RGBT tracking at present and provide some opportunities and future directions for RGBT visual tracking, such as dataset diversity, unsupervised and weakly supervised applications. In conclusion, our survey aims to serve as a useful resource for researchers and practitioners interested in the emerging field of RGBT tracking, and to promote further progress and innovation in this area.

Bokeh-based target tracking for structural dynamic monitoring: A novel approach in variable lighting conditions

The application of Computer Vision (CV) techniques in Structural Dynamic Monitoring (SDM) eliminates the need for sensor installation and calibration, providing reliable monitoring results. However, conventional CV methods typically require time-intensive supervised pre-processing steps such as pattern extraction and machine learning. This paper introduces a novel Bokeh-effect-based target object tracking method for SDM that forgoes the need for such pre-processing, allowing for unsupervised, real-time, non-contact monitoring. This method, adaptable for diverse lighting conditions and employing consumer-grade cameras and computers, creates circles around target objects positioned beyond the depth of field of the camera lens, then detects and tracks the movement of these circles to facilitate structural dynamic monitoring. The proposed method is applied to a wind tunnel experiment on bridge girder sections to verify its accuracy and reliability, and the result is validated with the measurement from LASER transducers in the same wind tunnel experiment. This innovative approach advances efficient SDM by offering rapid response and real-time non-contact tracking capability with a high tolerance for target objects under regular and low-light conditions while avoiding the complexities linked with the pre-processing inherent in other supervised CV detection methods.

Image hashing-based shallow object tracker for construction worker monitoring in scaffolding scenes

Multiple Object Tracking (MOT) has potential applications in construction site safety man- agement, particularly for the individualized assessment of scaffold workers' safety status. However, its accuracy can be degraded due to inconsistent detection results and insuffi- cient object association capabilities. This paper addresses these challenges by proposing an object tracking method, named as the Shallow Cascaded Buffered Intersection over Union (Shallow C-BIoU). This non-machine learning-based tracker employs a color hash- ing technique to enhance tracking performance. Employing rigorous evaluation metrics, the experimental results underscore the efficacy of the proposed method. Compared to state-of- the-art algorithms, the Shallow C-BIoU method improved association performance by 7.56%, reducing 52.44% of falsely assigned tracking IDs. Consequently, this paper contributes to the development of reliable object trackers, thereby advancing monitoring technologies for construction management purposes.

Enhanced Object Detection in Autonomous Vehicles through LiDAR—Camera Sensor Fusion

To realize accurate environment perception, which is the technological key to enabling autonomous vehicles to interact with their external environments, it is primarily necessary to solve the issues of object detection and tracking in the vehicle-movement process. Multi-sensor fusion has become an essential process in efforts to overcome the shortcomings of individual sensor types and improve the efficiency and reliability of autonomous vehicles. This paper puts forward moving object detection and tracking methods based on LiDAR—camera fusion. Operating based on the calibration of the camera and LiDAR technology, this paper uses YOLO and PointPillars network models to perform object detection based on image and point cloud data. Then, a target box intersection-over-union (IoU) matching strategy, based on center-point distance probability and the improved Dempster–Shafer (D–S) theory, is used to perform class confidence fusion to obtain the final fusion detection result. In the process of moving object tracking, the DeepSORT algorithm is improved to address the issue of identity switching resulting from dynamic objects re-emerging after occlusion. An unscented Kalman filter is utilized to accurately predict the motion state of nonlinear objects, and object motion information is added to the IoU matching module to improve the matching accuracy in the data association process. Through self-collected data verification, the performances of fusion detection and tracking are judged to be significantly better than those of a single sensor. The evaluation indexes of the improved DeepSORT algorithm are 66% for MOTA and 79% for MOTP, which are, respectively, 10% and 5% higher than those of the original DeepSORT algorithm. The improved DeepSORT algorithm effectively solves the problem of tracking instability caused by the occlusion of moving objects.

LSOTB-TIR: A Large-Scale High-Diversity Thermal Infrared Single Object Tracking Benchmark.

Unlike visual object tracking, thermal infrared (TIR) object tracking methods can track the target of interest in poor visibility such as rain, snow, and fog, or even in total darkness. This feature brings a wide range of application prospects for TIR object-tracking methods. However, this field lacks a unified and large-scale training and evaluation benchmark, which has severely hindered its development. To this end, we present a large-scale and high-diversity unified TIR single object tracking benchmark, called LSOTB-TIR, which consists of a tracking evaluation dataset and a general training dataset with a total of 1416 TIR sequences and more than 643 K frames. We annotate the bounding box of objects in every frame of all sequences and generate over 770 K bounding boxes in total. To the best of our knowledge, LSOTB-TIR is the largest and most diverse TIR object tracking benchmark to date. We spilt the evaluation dataset into a short-term tracking subset and a long-term tracking subset to evaluate trackers using different paradigms. What's more, to evaluate a tracker on different attributes, we also define four scenario attributes and 12 challenge attributes in the short-term tracking evaluation subset. By releasing LSOTB-TIR, we encourage the community to develop deep learning-based TIR trackers and evaluate them fairly and comprehensively. We evaluate and analyze 40 trackers on LSOTB-TIR to provide a series of baselines and give some insights and future research directions in TIR object tracking. Furthermore, we retrain several representative deep trackers on LSOTB-TIR, and their results demonstrate that the proposed training dataset significantly improves the performance of deep TIR trackers. Codes and dataset are available at https://github.com/QiaoLiuHit/LSOTB-TIR.

Dynamic center point learning for multiple object tracking under Severe occlusions

Multiple Object Tracking (MOT) methods based on per-pixel prediction and association have achieved remarkable progress recently. These approaches prefer to select points in central region of the bounding box as positive samples and other points as negative samples. Under severe occlusions, these sample allocation methods might lead to the central region being contaminated by occluding samples, resulting in significant degradation of both detection and association performance. To address this issue, we propose a novel Dynamic-Center-Point-based Multiple Object Tracking method (DCP-MOT), which aims to self-identify the visible center region for occluded objects. Compared with the previous bounding box center point, our Dynamic-Center-Point (DCP) can better represent the visible region of the occluded object. Specifically, we design an Iterative Refinement Branch to generate dynamic center points for each occluded object. It includes two parts: Center Probability Predictor and Center Generator. Initially, we utilize the Center Probability Predictor to derive an accurate probability map for the occluded object. Subsequently, we use Center Generator to quantify the probability map by introducing a Mutual Exclusive Potential Function, yielding a dynamic center point for the occluded object that is distinct from its occluding counterpart. Finally, by joining the bounding box center points for unoccluded objects and our dynamic center points for occluded objects, our JDE branch can achieve better tracking performance. Extensive experiments demonstrate our DCP-based MOT method surpasses the bounding box center based SOTA in metrics MOTA (+0.7,+2.1,+1.3) and IDF1 (+0.7,+0.9,+1.5) on the challenging MOT16, MOT17, MOT20 datasets.

A review on deep learning-based object tracking methods

A deep learning algorithm tracks an object’s movement during object tracking and the main challenge in the tracking of objects is to estimate or forecast the locations and other pertinent details of moving objects in a video. Typically, object tracking entails the process of object detection. In computer vision applications the detection, classification, and tracking of objects play a vital role, and gaining information about the various techniques available also provides significance. In this research, a systematic literature review of the object detection techniques is performed by analyzing, summarizing, and examining the existing works available. Various state of art works are collected from standard journals and the methods available, cons, and pros along with challenges are determined based on this the research questions are also formulated. Overall, around 50 research articles are collected, and the evaluation based on various metrics shows that most of the literary works used Deep convolutional neural networks (Deep CNN), and while tracking the objects object detection helps in enhancing the performance of these networks. The important issues that need to be resolved are also discussed in this research, which helps in leveling up the object-tracking techniques.

Object Tracking Using Computer Vision: A Review

Object tracking is one of the most important problems in computer vision applications such as robotics, autonomous driving, and pedestrian movement. There has been a significant development in camera hardware where researchers are experimenting with the fusion of different sensors and developing image processing algorithms to track objects. Image processing and deep learning methods have significantly progressed in the last few decades. Different data association methods accompanied by image processing and deep learning are becoming crucial in object tracking tasks. The data requirement for deep learning methods has led to different public datasets that allow researchers to benchmark their methods. While there has been an improvement in object tracking methods, technology, and the availability of annotated object tracking datasets, there is still scope for improvement. This review contributes by systemically identifying different sensor equipment, datasets, methods, and applications, providing a taxonomy about the literature and the strengths and limitations of different approaches, thereby providing guidelines for selecting equipment, methods, and applications. Research questions and future scope to address the unresolved issues in the object tracking field are also presented with research direction guidelines.

ROTracker: a novel MMW radar-based object tracking method for unmanned surface vehicle in offshore environments

Unmanned surface vehicles (USVs) offer significant value through their capability to undertake hazardous and time-consuming missions across water surfaces. Recently, as the application of USVs has been extended to nearshore waterways, object tracking is vital to the safe navigation of USVs in offshore scenes. However, existing tracking systems for USVs are mainly based on cameras or LiDAR sensors, which suffer from drawbacks such as lack of depth perception or high deployment costs. In contrast, millimeter-wave (MMW) radar offers advantages in terms of low cost and robustness in all weather and lighting conditions. In this work, to construct a robust and low-cost tracking system for USVs in complex offshore scenes, we propose a novel MMW radar-based object tracking method (ROTracker). The proposed ROTracker combines the physical properties of MMW radar with traditional tracking systems. Specifically, we introduce the radar Doppler velocity and a designed motion discriminator to improve the robustness of the tracking system toward low-speed targets. Moreover, we conducted real-world experiments to validate the efficacy of the proposed ROTracker. Compared to other baseline methods, ROTracker achieves excellent multiple object tracking accuracy in terms of 91.9% in our collected dataset. The experimental results demonstrated that the proposed ROTracker has significant application potential in both accuracy and efficiency for USVs, addressing the challenges posed by complex nearshore environments.

Versatile multiple object tracking in sparse 2D/3D videos via deformable image registration.

Tracking body parts in behaving animals, extracting fluorescence signals from cells embedded in deforming tissue, and analyzing cell migration patterns during development all require tracking objects with partially correlated motion. As dataset sizes increase, manual tracking of objects becomes prohibitively inefficient and slow, necessitating automated and semi-automated computational tools. Unfortunately, existing methods for multiple object tracking (MOT) are either developed for specific datasets and hence do not generalize well to other datasets, or require large amounts of training data that are not readily available. This is further exacerbated when tracking fluorescent sources in moving and deforming tissues, where the lack of unique features and sparsely populated images create a challenging environment, especially for modern deep learning techniques. By leveraging technology recently developed for spatial transformer networks, we propose ZephIR, an image registration framework for semi-supervised MOT in 2D and 3D videos. ZephIR can generalize to a wide range of biological systems by incorporating adjustable parameters that encode spatial (sparsity, texture, rigidity) and temporal priors of a given data class. We demonstrate the accuracy and versatility of our approach in a variety of applications, including tracking the body parts of a behaving mouse and neurons in the brain of a freely moving C. elegans. We provide an open-source package along with a web-based graphical user interface that allows users to provide small numbers of annotations to interactively improve tracking results.