3D Tracking System Research Articles

Cyber Mobility Mirror: A Deep Learning-Based Real-World Object Perception Platform Using Roadside LiDAR

Object perception plays a fundamental role in Cooperative Driving Automation (CDA) which is regarded as a revolutionary promoter for next-generation transportation systems. However, the vehicle-based perception may suffer from the limited sensing range and occlusion as well as low penetration rates in connectivity. In this paper, we propose <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Cyber Mobility Mirror</i> ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">CMM</i> ), a next-generation real-world object perception system for 3D object detection, tracking, localization, and reconstruction, to explore the potential of roadside sensors for enabling CDA in the real world. The CMM system consists of six main components: i) the data pre-processor to retrieve and preprocess the raw data; ii) the roadside 3D object detector to generate 3D detection results; iii) the multi-object tracker to identify detected objects; iv) the global locator to generate geo-localization information; v) the mobile-edge-cloud-based communicator to transmit perception information to equipped vehicles, and vi) the onboard advisor to reconstruct and display the real-time traffic conditions. An automatic perception evaluation approach is proposed to support the assessment of data-driven models without human-labeling requirements and a CMM field-operational system is deployed at a real-world intersection to assess the performance of the CMM. Results from field tests demonstrate that our CMM prototype system can achieve 96.99% precision and 83.62% recall for detection and 73.55% ID-recall for tracking. High-fidelity real-time traffic conditions (at the object level) can be geo-localized with a root-mean-square error (RMSE) of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$0.69m$</tex-math> </inline-formula> and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$0.33m$</tex-math> </inline-formula> for lateral and longitudinal direction, respectively, and displayed on the GUI of the equipped vehicle with a frequency of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$3-4 Hz$</tex-math> </inline-formula> .

Self-Calibrated Multi-Sensor Wearable for Hand Tracking and Modeling.

We present a multi-sensor system for consistent 3D hand pose tracking and modeling, which leverages the advantages of both wearable and optical sensors. Specifically, we employ a stretch-sensing soft glove and three IMUs in combination with an RGB-D camera. Different sensor modalities are fused based on the availability and confidence estimation, enabling seamless hand tracking in challenging environments with partial or even complete occlusion. To maximize the accuracy while maintaining a high ease-of-use, we propose an automated user calibration that uses the RGB-D camera data to refine both the glove mapping model and the multi-IMU system parameters. Extensive experiments show that our setup outperforms the wearable-only approaches when the hand is in the field-of-view and outplays the camera-only methods when the hand is occluded.

A Multiple Video Camera System for 3D Tracking of Farmed Fry in an Aquaculture Tank

A Multiple Video Camera System for 3D Tracking of Farmed Fry in an Aquaculture Tank

Infrared reflection system for indoor 3D tracking of fish

Many fish rearing infrastructures are already equipped with human-operated camera systems for fish behavior monitoring, e.g. for stopping the feeding system when the fish is satiated or for monitoring of fish behavior abnormalities caused by poor water quality or diseases. The novel infrared reflection (IREF) system for indoor 3D tracking of fish demonstrated in the current study allows for automation of fish behavior monitoring, reducing the system running costs by eliminating the need for continuous human monitoring and increasing the behavioral analysis accuracy by excluding the human subjectivity factor.The operating principle of this system is based on the effect of strong absorption of near infrared (NIR) range light by water, thus allowing estimation of fish distance based on the corresponding fish object brightness on the camera image. The use of NIR illuminator as a part of the IREF system allows fish behavior monitoring in the dark so as not to affect fish circadian rhythm. A system evaluation under aquaculture facility conditions with Atlantic salmon (Salmo salar) using flow-through water in tanks, showed the mean depth estimation error was equal to 5.3±4.0 (SD) cm. The physiological variations among conspecific individual fish introduced the mean depth estimation error of 1.6±1.3 (SD) cm. The advantages of the IREF system over well-known stereo vision systems are lower hardware cost and less computationally intensive 3D coordinates estimation algorithm, while the disadvantage is lower accuracy that is nevertheless acceptable for most applications of aquaculture fish monitoring.

Automatic detection and tracking of pedestrians from a moving stereo rig

We report on a stereo system for 3D detection and tracking of pedestrians in urban traffic scenes. The system is built around a probabilistic environment model which fuses evidence from dense 3D reconstruction and image-based pedestrian detection into a consistent interpretation of the observed scene, and a multi-hypothesis tracker to reconstruct the pedestrians’ trajectories in 3D coordinates over time. Experiments on real stereo sequences recorded in busy inner-city scenarios are presented, in which the system achieves promising results.

Fully Automatic Real-Time 3D Object Tracking using Active Contour and Appearance Models

This paper presents an efficient, robust and fully automatic real-time system for 3D object pose tracking in image sequences. The developed application integrates two main components: an extended and optimized implementa- tion of a state-of-the-art local curve fitting algorithm, and a robust global re-initialization module based on point feature matching. By combining the two main components, together with a trajectory coherence and image cross-correlation check modules, our system achieves full automatic and robust object tracking in real world situations, requiring very small manual intervention from the user. The developed application relies upon a few standard libraries available on most platforms, and runs at video frame rate on a PC with standard hardware equipment.

Smart Servoing of a Controllable Range Sensor for Target Tracking: Application to Pedestrians

This paper presents a system for 3D target tracking. A controllable range sensor is servoid in order to focus the attention of the system on the target. The 3D-sensor is a laser telemeter based range finder of which the laser beam is deflected by two flat mirrors. The deviations of the mirrors are fully programmable and can be dynamically controlled by an application program. The data from the sensor are passed to a Kalman filter which predicts the next location of the target. The predicted location of the target allows to define an area which is scanned by the laser beam to obtain a 3D image. This system is applied to pedestrian tracking in a road environment.