Tracking Evaluation Research Articles

Food induced distractibility in restrictive anorexia nervosa: Different motor patterns for different foods as revealed by a mouse tracker evaluation.

An altered automatic processing of food stimuli may contribute to the maintenance of calorie restriction in patients with restrictive Anorexia Nervosa (AN-R). The present study aimed to assess whether task-irrelevant food distractors elicited a different interference effect in the motor actions of patients with AN-R compared to healthy controls (HC). 40 patients with acute AN-R and 40 HC performed an irrelevant distractor task in which they were required to perform a reaching movement from a starting point to a green dot, while an irrelevant distractor (a high-calorie food, low-calorie food, or neutral object) was presented in the middle of the screen. Mouse trajectories and response times (RT) were recorded. The analyses conducted on the kinematic variables revealed that while the trajectories of HC veered similarly toward the three categories of stimuli, AN-R patients showed an increased deviation toward low-calorie foods and a reduced deviation toward high-calorie foods compared to neutral objects. No significant results emerged as regards RT. The pattern of responses observed in patients with AN-R (deviation increased toward low-calorie and reduced toward high-calorie) is consistent with their eating habits and may thus represent an implicit mechanism sustaining calorie restriction in patients with AN-R.

Robot-Person Tracking in Uniform Appearance Scenarios: A New Dataset and Challenges

Person-tracking robots have many applications including security, surveillance, and autonomous driving. Despite the abundance of uniform appearance in many contexts and the challenges they exhibit, there is a lack of video datasets dedicated to benchmarking tracking algorithms in such contexts. In this article, we propose a new high-quality RGB-D benchmark called PTUA for robot–person tracking in uniform appearance scenarios. PTUA is recorded using an RGB-D sensor on top of a moving robot and consists of 45 sequences containing more than 85 K frames. Each frame is manually annotated with a bounding box and attributes, making PTUA the largest and the most challenging person tracking RGB-D dataset. To the best of our knowledge, such a densely annotated and properly synchronized RGB-D tracking benchmark does not exist in the literature. Each sequence comprises various challenges deriving from real-life scenarios where the target person appears highly similar to the background or distractors. By releasing PTUA, we expect to provide the community with a large-scale challenging RGB-D benchmark with high quality for the robust evaluation of trackers on uniform appearance scenarios for autonomous robots. We also present a rigorous experimental evaluation of the state-of-the-art trackers on the PTUA dataset with a comprehensive analysis. The findings evidence the challenges of person tracking in a uniform appearance scenario for both target tracking and robot–person tracking, and the need to bridge the performance gap. In addition, we propose a new RGB-D tracker that extracts features from RGB-D frames and it achieves the best performance on each challenging scenario of PTUA.

Design, Construction and Calibration of a Step Gauge of Nests for Performance Evaluation of Laser Trackers

According to documentary standards for the performance evaluation of laser trackers (LT), long length reference artifacts are required. In this paper, we discuss the design, construction, and calibration of a long length artifact called a step gauge of nests (SGN). The SGN has several nests in line in which to place the LT probe; the two extreme nests of the SGN are at a distance of 3 m, approximately. The documentary standards establishes that the gauge’s length shall be known no matter the orientations it takes. However, for long gauges, factors like gravitational force, fixturing forces, change in the environmental conditions, among others, deform the gauge, and its length changes when its orientation changes. To evaluate these factors, we use a finite element simulation of the SGN in the design stage to predict such deformations (mainly length variations between the two extreme nests). The simulation takes into account the used material, its stiffness, straightness, distribution of the nest’s weight, and geometry’s change of the SGN to reduce the variations in its length. For the construction stage, we describe how the SGN was manufactured and how using high module carbon fiber reduces the influence of the temperature factor. The results of the finite element simulation show a length variation of around 20 µm between the horizontal and vertical SGN positions. That variation was validated with the calibration results of two different methods. The first uses the line of sight (LOS) meth-od, which involves the same LT under evaluation. The second uses an accurate CMM, using the overlap method for calibration. The traceability of the LOS method is accomplished with the wavelength calibration of the LT interferometer while the overlap method uses a CMM evaluated with a laser interferometer with calibrated wavelength.

The Development of a Web-Based Tobacco Tracker Tool to Crowdsource Campus Environmental Reports for Smoke and Tobacco-Free College Policies: Mixed Methods Study.

BackgroundCollege campuses in the United States have begun implementing smoke and tobacco–free policies to discourage the use of tobacco. Smoke and tobacco–free policies, however, are contingent upon effective policy enforcement.ObjectiveThis study aimed to develop an empirically derived web-based tracking tool (Tracker) for crowdsourcing campus environmental reports of tobacco use and waste to support smoke and tobacco–free college policies.MethodsAn exploratory sequential mixed methods approach was utilized to inform the development and evaluation of Tracker. In October 2018, three focus groups across 2 California universities were conducted and themes were analyzed, guiding Tracker development. After 1 year of implementation, users were asked in April 2020 to complete a survey about their experience.ResultsIn the focus groups, two major themes emerged: barriers and facilitators to tool utilization. Further Tracker development was guided by focus group input to address these barriers (eg, information, policing, and logistical concerns) and facilitators (eg, environmental motivators and positive reinforcement). Amongst 1163 Tracker reports, those who completed the user survey (n=316) reported that the top motivations for using the tool had been having a cleaner environment (212/316, 79%) and health concerns (185/316, 69%).ConclusionsEnvironmental concerns, a motivator that emerged in focus groups, shaped Tracker’s development and was cited by the majority of users surveyed as a top motivator for utilization.

An Evaluation of Motion Trackers with Virtual Reality Sensor Technology in Comparison to a Marker-Based Motion Capture System Based on Joint Angles for Ergonomic Risk Assessment.

The reproduction and simulation of workplaces, and the analysis of body postures during work processes, are parts of ergonomic risk assessments. A commercial virtual reality (VR) system offers the possibility to model complex work scenarios as virtual mock-ups and to evaluate their ergonomic designs by analyzing motion behavior while performing work processes. In this study a VR tracking sensor system (HTC Vive tracker) combined with an inverse kinematic model (Final IK) was compared with a marker-based optical motion capture system (Qualisys). Marker-based optical motion capture systems are considered the gold standard for motion analysis. Therefore, Qualisys was used as the ground truth in this study. The research question to be answered was how accurately the HTC Vive System combined with Final IK can measure joint angles used for ergonomic evaluation. Twenty-six subjects were observed simultaneously with both tracking systems while performing 20 defined movements. Sixteen joint angles were analyzed. Joint angle deviations between and were identified. These high deviations must be considered in ergonomic risk assessments when using a VR system. The results show that commercial low-budget tracking systems have the potential to map joint angles. Nevertheless, substantial weaknesses and inaccuracies in some body regions must be taken into account. Recommendations are provided to improve tracking accuracy and avoid systematic errors.

GOT-10k: A Large High-Diversity Benchmark for Generic Object Tracking in the Wild.

We introduce here a large tracking database that offers an unprecedentedly wide coverage of common moving objects in the wild, called GOT-10k. Specifically, GOT-10k is built upon the backbone of WordNet structure [1] and it populates the majority of over 560 classes of moving objects and 87 motion patterns, magnitudes wider than the most recent similar-scale counterparts [19], [20], [23], [26]. By releasing the large high-diversity database, we aim to provide a unified training and evaluation platform for the development of class-agnostic, generic purposed short-term trackers. The features of GOT-10k and the contributions of this article are summarized in the following. (1) GOT-10k offers over 10,000 video segments with more than 1.5 million manually labeled bounding boxes, enabling unified training and stable evaluation of deep trackers. (2) GOT-10k is by far the first video trajectory dataset that uses the semantic hierarchy of WordNet to guide class population, which ensures a comprehensive and relatively unbiased coverage of diverse moving objects. (3) For the first time, GOT-10k introduces the one-shot protocol for tracker evaluation, where the training and test classes are zero-overlapped. The protocol avoids biased evaluation results towards familiar objects and it promotes generalization in tracker development. (4) GOT-10k offers additional labels such as motion classes and object visible ratios, facilitating the development of motion-aware and occlusion-aware trackers. (5) We conduct extensive tracking experiments with 39 typical tracking algorithms and their variants on GOT-10k and analyze their results in this paper. (6) Finally, we develop a comprehensive platform for the tracking community that offers full-featured evaluation toolkits, an online evaluation server, and a responsive leaderboard. The annotations of GOT-10k's test data are kept private to avoid tuning parameters on it.

Preliminary Evaluation of High Speed and Low Latency Eye Tracker using Custom CMOS Image Sensor

Preliminary Evaluation of High Speed and Low Latency Eye Tracker using Custom CMOS Image Sensor

LaSOT: A High-quality Large-scale Single Object Tracking Benchmark

Despite great recent advances in visual tracking, its further development, including both algorithm design and evaluation, is limited due to lack of dedicated large-scale benchmarks. To address this problem, we present LaSOT, a high-quality Large-scale Single Object Tracking benchmark. LaSOT contains a diverse selection of 85 object classes, and offers 1550 totaling more than 3.87 million frames. Each video frame is carefully and manually annotated with a bounding box. This makes LaSOT, to our knowledge, the largest densely annotated tracking benchmark. Our goal in releasing LaSOT is to provide a dedicated high quality platform for both training and evaluation of trackers. The average video length of LaSOT is around 2500 frames, where each video contains various challenge factors that exist in real world video footage,such as the targets disappearing and re-appearing. These longer video lengths allow for the assessment of long-term trackers. To take advantage of the close connection between visual appearance and natural language, we provide language specification for each video in LaSOT. We believe such additions will allow for future research to use linguistic features to improve tracking. Two protocols, full-overlap and one-shot, are designated for flexible assessment of trackers. We extensively evaluate 48 baseline trackers on LaSOT with in-depth analysis, and results reveal that there still exists significant room for improvement. The complete benchmark, tracking results as well as analysis are available at http://vision.cs.stonybrook.edu/~lasot/ .

Performance evaluation of laser trackers using the network method

Abstract Laser trackers (LTs) are widely used for large-scale dimensional metrology. Periodic LT performance evaluation is a key aspect in ensuring measurement reliability. The existing documentary standards for LT performance evaluation require scale bars and special artifacts that are often unavailable for many LT users. A network-based method is considered in this paper as a simple and effective approach to evaluate LT performance without the need for expensive artifacts or reference instruments. In this method, a set of fixed targets are measured from different locations of the LT. The reference values for the lengths between pairs of targets are determined by incorporating the geometrical error model of the LT (i.e., the LT data from each location are corrected for systematic geometric misalignments, quantities that are also obtained from the same data), thus the LT under test is itself used to calibrate the lengths in the network. Subsequently, these length values are used as references to evaluate the LT performance. We validate this method of establishing reference lengths by comparing the values for select lengths in the network against more traditional line-of-sight interferometry. We show that the uncertainties in the reference lengths obtained in this manner are sufficiently small in comparison to the maximum permissible error (MPE) specifications so that they can be used for performance evaluation. A case study using three LTs is presented in this paper. The results obtained from that case study is also verified by a simulation based on the ASME B89.4.19 standard, showing the feasibility of the proposed network-based LT performance evaluation method.

PTB-TIR: A Thermal Infrared Pedestrian Tracking Benchmark

Thermal infrared (TIR) pedestrian tracking is one of the important components among numerous applications of computer vision, which has a major advantage: it can track pedestrians in total darkness. The ability to evaluate the TIR pedestrian tracker fairly, on a benchmark dataset, is significant for the development of this field. However, there is not a benchmark dataset. In this paper, we develop a TIR pedestrian tracking dataset for the TIR pedestrian tracker evaluation. The dataset includes 60 thermal sequences with manual annotations. Each sequence has nine attribute labels for the attribute based evaluation. In addition to the dataset, we carry out the large-scale evaluation experiments on our benchmark dataset using nine publicly available trackers. The experimental results help us understand the strengths and weaknesses of these trackers.In addition, in order to gain more insight into the TIR pedestrian tracker, we divide its functions into three components: feature extractor, motion model, and observation model. Then, we conduct three comparison experiments on our benchmark dataset to validate how each component affects the tracker's performance. The findings of these experiments provide some guidelines for future research. The dataset and evaluation toolkit can be downloaded at {https://github.com/QiaoLiuHit/PTB-TIR_Evaluation_toolkit}.

Evaluation of Wheelchair In-Seat Activity Tracker

Evaluation of Wheelchair In-Seat Activity Tracker

Evaluation of detection and tracking performance in imaging systems

Imaging systems can be used to obtain situational awareness in maritime situations. Important tools for these systems are automatic detection and tracking of objects in the acquired imagery, in which numerous methods are being developed. When designing a detection or tracking algorithm, its quality should be ensured by a comparison with existing algorithms and/or with a ground truth. Detection and tracking methods are often designed for a specific task, so evaluation with respect to this task is crucial, which demands for different evaluation measures for different tasks. We, therefore, propose a variety of quantitative measures for the performance evaluation of detectors and trackers for a variety of tasks. The proposed measures are a rich set from which an algorithm designer can choose in order to optimally design and assess a detection or tracking algorithm for a specific task. We compare these different evaluation measures by using them to assess detection and tracking quality in different maritime detection and tracking situations, obtained from three real-life infrared video data sets. With the proposed set of evaluation measures, a user is able to quantitatively assess the performance of a detector or tracker, which enables an optimal design for his approach.

Impact of work hours and sleep on well-being and burnout for physicians-in-training: the Resident Activity Tracker Evaluation Study.

The Resident Activity Tracker Evaluation (RATE) is a prospective observational study evaluating the impact of work hours, sleep and physical activity on resident well-being, burnout and job satisfaction. Physician burnout is common and its incidence is increasing. The impact of work hours and sleep on resident well-being and burnout remains elusive. Activity trackers are an innovative tool for measuring sleep and physical activity. Residents were recruited from (i) general surgery and orthopaedics (SURG), (ii) internal medicine and neurology (MED) and (iii) anaesthesia and radiology (RCD). Groups 1 and 2 do not enforce restrictions on the duration of being on-call, and group 3 had restricted the duration of being on-call to 12 hours. Participants wore FitBit trackers for 14 days. Total hours worked, daily sleep, sleep on-call and daily steps were recorded. Participants completed validated surveys assessing self-reported well-being (Short-Form Health Survey), burnout (Maslach Burnout Inventory), and satisfaction with medicine. Surgical residents worked the most hours per week, followed by medical and RCD residents (SURG, 84.3 hours, 95% CI, 80.2-88.5; MED, 69.2 hours, 95% CI, 65.3-73.2; RCD, 52.2 hours, 95% CI, 48.2-56.1; p < 0.001). Surgical residents obtained fewer hours of sleep per day (SURG, 5.9 hours, 95% CI, 5.5-6.3; MED, 6.9 hours, 95% CI, 6.5-7.3; RCD, 6.8 hours, 95% CI, 5.6-7.2; p < 0.001). Nearly two-thirds of participants (61%) scored high burnout on the Maslach depersonalisation subscore. Total steps per day and well-being, burnout and job satisfaction were comparable between groups. Total hours worked, daily sleep and steps per day did not predict burnout or well-being. Work hours and average daily sleep did not affect burnout. Physical activity did not prevent burnout. Work hour restrictions may lead to increased sleep but may not affect resident burnout or well-being.

Laser tracker calibration procedure at Central Office of Measures

A laser tracker is a portable coordinate measuring system that tracks a moving target reflector and measures the position of the target in spherical coordinates the establishment of an ASME B89.4.19 laser tracker verification facility at Central Office of Measures, Poland is introduced. The set of laser wavelength calibration, environmental sensors calibration, ranging tests, length measurement system tests and two-face system tests were performed to assess the performance of instruments. For length measurement errors the Brunson Instrument Company KinAiry laser tracker evaluation system was used. Example results from the tests were presented.

B.01 Impact of work-hours and sleep on well-being and burnout for physicians-in-training: the prospective RATE Study

Background: Wearable activity trackers are an innovative tool for measuring sleep and physical activity. The Resident Activity Tracker Evaluation (RATE) is a prospective observational study evaluating the impact of work-hours, sleep, and physical activity on resident well-being, burnout, and job satisfaction. Methods: Residents were recruited from: 1. general surgery and orthopedics (SURG), 2. internal medicine and neurology (MED) and 3. anesthesia and radiology (RCD). Groups 1 and 2 do not enforce on-call duration restrictions and group 3 had 12-hour restricted-call durations (RCD). Participants wore FitBit activity trackers for 14 days and completed four validated surveys assessing self-reported health, sleepiness, burnout, and job satisfaction. Results: Fifty-nine residents completed the study. 778 days of activity and 244 on-call periods were tracked. Surgical residents worked 24 more hours per week than non-surgical residents (84.3 vs 60.7). Surgical residents had 7 less hours of sleep per week and reported significantly higher Epworth Sleepiness scores. Nearly two-thirds of participants (61%) scored high burnout on the Maslach depersonalization subscore. Total steps per day and self-reported well-being, burnout, and job satisfaction were comparable between the groups. Conclusions: Despite a positive correlation between work-hours and sleepiness, burnout and well-being were similar among residents. Physical activity did not prevent burnout. These findings are relevant to work-hours policies.

A method for performance diagnosis and evaluation of video trackers

Several measures for evaluating multi-target video trackers exist that generally aim at providing ‘end performance.’ End performance is important particularly for ranking and comparing trackers. However, for a deeper insight into trackers’ performance it would also be desirable to analyze key contributory factors (false positives, false negatives, ID changes) that (implicitly or explicitly) lead to the attainment of a certain end performance. Specifically, this paper proposes a new approach to enable a diagnosis of the performance of multi-target trackers as well as providing a means to determine the end performance to still enable their comparison in a video sequence. Diagnosis involves analyzing probability density functions of false positives, false negatives and ID changes of trackers in a sequence. End performance is obtained in terms of the extracted performance scores related to false positives, false negatives and ID changes. In the experiments, we used four state-of-the-art trackers on challenging real-world public datasets to show the effectiveness of the proposed approach.

A Novel Performance Evaluation Methodology for Single-Target Trackers.

This paper addresses the problem of single-target tracker performance evaluation. We consider the performance measures, the dataset and the evaluation system to be the most important components of tracker evaluation and propose requirements for each of them. The requirements are the basis of a new evaluation methodology that aims at a simple and easily interpretable tracker comparison. The ranking-based methodology addresses tracker equivalence in terms of statistical significance and practical differences. A fully-annotated dataset with per-frame annotations with several visual attributes is introduced. The diversity of its visual properties is maximized in a novel way by clustering a large number of videos according to their visual attributes. This makes it the most sophistically constructed and annotated dataset to date. A multi-platform evaluation system allowing easy integration of third-party trackers is presented as well. The proposed evaluation methodology was tested on the VOT2014 challenge on the new dataset and 38 trackers, making it the largest benchmark to date. Most of the tested trackers are indeed state-of-the-art since they outperform the standard baselines, resulting in a highly-challenging benchmark. An exhaustive analysis of the dataset from the perspective of tracking difficulty is carried out. To facilitate tracker comparison a new performance visualization technique is proposed.

Configuration Optimization of Laser Tracker Stations for Large-scale Components in Non-uniform Temperature Field Using Monte-carlo Method

The laser trackers play an important role in fabrication and assembly of aircraft, spacecraft and other large-scale products, and optimization of laser tracker configuration station is one of the core issues. When laser trackers are used on the shop floor, environment factors, especially non-uniform temperature can induce considerable measurement uncertainty. Aiming to reduce measurement uncertainty of laser tracker, optimization of laser tracker configuration station for large-scale components in non-uniform temperature field using the Monte-Carlo method is proposed. Firstly, an improved mathematical model for measurement uncertainty of laser tracker is established. This mathematical model is more applicable to measurement uncertainty evaluation of laser tracker because it takes main measurement uncertainty sources (mechanism system and non-uniform temperature) into account. Secondly, the Monte-Carlo method is used to evaluate measurement uncertainties of laser tracker in different configuration stations. Based on the measurement mathematical model, a serial of simulation data can be generated to evaluate the uncertainty of measurement task. Through iterations, the optimal configuration station of laser tracker, in which measurement uncertainty is minimal, can be obtained. Finally, an example of measurement scenario about measuring the inspection jig of an aircraft door demonstrates that the proposed method is feasible and effective. The proposed approach to identify the optimal position of laser tracker is intuitionistic and efficient.

Visual Object Tracking Performance Measures Revisited.

The problem of visual tracking evaluation is sporting a large variety of performance measures, and largely suffers from lack of consensus about which measures should be used in experiments. This makes the cross-paper tracker comparison difficult. Furthermore, as some measures may be less effective than others, the tracking results may be skewed or biased toward particular tracking aspects. In this paper, we revisit the popular performance measures and tracker performance visualizations and analyze them theoretically and experimentally. We show that several measures are equivalent from the point of information they provide for tracker comparison and, crucially, that some are more brittle than the others. Based on our analysis, we narrow down the set of potential measures to only two complementary ones, describing accuracy and robustness, thus pushing toward homogenization of the tracker evaluation methodology. These two measures can be intuitively interpreted and visualized and have been employed by the recent visual object tracking challenges as the foundation for the evaluation methodology.

Determining Observer Reliability in Counts of River Otter Tracks

ABSTRACT In many research projects, reliability of collected data is dependent on reliability of field observers. However, it is uncommon for observer reliability to be either measured or reported in wildlife research. We tested whether observer skill affected outcomes of a northern river otter (Lontra canadensis) track survey conducted by the Texas Parks and Wildlife Department. Observers recorded presence of tracks at bridge sites (n = 250) throughout a 27‐county region in east Texas, USA. Logistic regression indicated that observers were significantly associated with frequency of reported otter tracks. Because observers were not assigned to bridges at random, we tested and found associations between the bridges surveyed by each observer (SURVEY ROUTE) and habitat variables (WATERSHED, VEGETATION‐TYPE, WATER‐TYPE, BRIDGE‐AREA) that may have influenced otter presence and probability of detection. A standardized tracker evaluation procedure indicated that experienced observers (n = 7) misidentified 37% of otter tracks. Additionally, 26% of tracks from species determined to be “otter‐like” were misidentified as otter tracks. We recommend that observer skill in identification of animal tracks and other indirect signs be measured to detect and reduce observer errors in wildlife monitoring.