Markerless Tracking Algorithm Research Articles

Monocular Visual SLAM for Markerless Tracking Algorithm to Augmented Reality

Augmented Reality (AR) tries to seamlessly integrate virtual content into the real world of the user. Ideally, the virtual content would behave exactly like real objects. This necessitates a correct and precise estimation of the user’s viewpoint (or that of a camera) with regard to the virtual content’s coordinate system. Therefore, the real-time establishment of 3-dimension (3D) maps in real scenes is particularly important for augmented reality technology. So in this paper, we integrate Simultaneous Localization and Mapping (SLAM) technology into augmented reality. Our research is to implement an augmented reality system without markers using the ORB-SLAM2 framework algorithm. In this paper we propose an improved method for Oriented FAST and Rotated BRIEF (ORB) feature extraction and optimized key frame selection, as well as the use of the Progressive Sample Consensus (PROSAC) algorithm for planar estimation of augmented reality implementations, thus solving the problem of increased system runtime because of the loss of large amounts of texture information in images. In this paper, we get better results by comparing experiments and data analysis. However, there are some improved methods of PROSAC algorithm which are more suitable for the detection of plane feature points.

American society of biomechanics early career achievement award 2020: Toward portable and modular biomechanics labs: How video and IMU fusion will change gait analysis

The field of biomechanics is at a turning point, with marker-based motion capture set to be replaced by portable and inexpensive hardware, rapidly improving markerless tracking algorithms, and open datasets that will turn these new technologies into field-wide team projects. Despite progress, several challenges inhibit both inertial and vision-based motion tracking from reaching the high accuracies that many biomechanics applications require. Their complementary strengths, however, could be harnessed toward better solutions than those offered by either modality alone. The drift from inertial measurement units (IMUs) could be corrected by video data, while occlusions in videos could be corrected by inertial data. To expedite progress in this direction, we have collected the CMU Panoptic Dataset 2.0, which contains 86 subjects captured with 140 VGA cameras, 31 HD cameras, and 15 IMUs, performing on average 6.5 min of activities, including range of motion activities and tasks of daily living. To estimate ground-truth kinematics, we imposed simultaneous consistency with the video and IMU data. Three-dimensional joint centers were first computed by geometrically triangulating proposals from a convolutional neural network applied to each video independently. A statistical meshed model parametrized in terms of body shape and pose was then fit through a top-down optimization approach that enforced consistency with both the video-based joint centers and IMU data. As proof of concept, we used this dataset to benchmark pose estimation from a sparse set of sensors, showing that incorporation of complementary modalities is a promising frontier that can be further strengthened through physics-informed frameworks.

Automated MV markerless tumor tracking for VMAT

Tumor tracking during radiotherapy treatment can improve dose accuracy, conformity and sparing of healthy tissue. Many methods have been introduced to tackle this challenge utilizing multiple imaging modalities, including a template matching based approach using the megavoltage (MV) on-board portal imager demonstrated on 3D conformal treatments. However, the complexity of treatments is evolving with the introduction of VMAT and IMRT, and successful motion management is becoming more important due to a trend towards hypofractionation.We have developed a markerless lung tumor tracking algorithm, utilizing the electronic portal imager (EPID) of the treatment machine. The algorithm has been specifically adapted to track during complex treatment deliveries with gantry and MLC motion. The core of the algorithm is an adaptive template matching method that relies on template stability metrics and local relative orientations to perform multiple feature tracking simultaneously. Only a single image is required to initialize the algorithm and features are automatically added, modified or removed in response to the input images. This algorithm was evaluated against images collected during VMAT arcs of a dynamic thorax phantom. Dynamic phantom images were collected during radiation delivery for multiple lung SBRT breathing traces and an example patient data set. The tracking error was 1.34 mm for the phantom data and 0.68 mm for the patient data.A multi-region, markerless tracking algorithm has been developed, capable of tracking multiple features simultaneously without requiring any other a priori information. This novel approach delivers robust target localization during complex treatment delivery. The reported tracking error is similar to previous reports for 3D conformal treatments.

Real-time tumor tracking using fluoroscopic imaging with deep neural network analysis

PurposeTo improve respiratory gating accuracy and treatment throughput, we developed a fluoroscopic markerless tumor tracking algorithm based on a deep neural network (DNN). MethodsIn the learning stage, target positions were projected onto digitally reconstructed radiography (DRR) images from four-dimensional computed tomography (4DCT). DRR images were cropped into subimages of the target or surrounding regions to build a network that takes input of the image pattern of subimages and produces a target probability map (TPM) for estimating the target position. Using multiple subimages, a DNN was trained to generate a TPM based on the target position projected onto the DRRs. In the tracking stage, the network takes in the subimages cropped from fluoroscopic images at the same position of the subimages on the DRRs and produces TPMs, which are used to estimate target positions. We integrated the lateral correction to modify an estimated target position by using a linear regression model. We tracked five lung and five liver cases, and calculated tracking accuracy (Euclidian distance in 3D space) by subtracting the estimated position from the reference. ResultsTracking accuracy averaged over all patients was 1.64 ± 0.73 mm. Accuracy for liver cases (1.37 ± 0.81 mm) was better than that for lung cases (1.90 ± 0.65 mm). Computation time was <40 ms for a pair of fluoroscopic images. ConclusionsOur markerless tracking algorithm successfully estimated tumor positions. We believe our results will provide useful information to advance tumor tracking technology.

모바일 증강현실 기반의 마커리스 추적 알고리즘 성능 연구

증강현실이란 실제 환경에 가상으로 생성된 정보를 실시간으로 증강하고 사용자가 그 정보들과 상호작용할 수 있도록 함으로써, 정보의 활용을 극대화하는 차세대 정보처리 기술이다. 본 논문에서는 모바일 환경에서 증강현실 시스템을 구현하기 위한 물체 추적 방안으로 마커를 사용하지 않는 마커리스 추적 알고리즘을 연구하였다. 마커리스 방식의 증강현실은 마커를 따로 부착하지 않아도 되고 위치의 제약이 없어서 사용자가 증강현실 기술을 사용하기 편리하다는 장점이 있다. 본 논문에서는 마커리스 추적을 위해 특징점 추출 기반의 SURF 알고리즘을 사용하였다. SURF 알고리즘은 다른 특징점 추출 기반 알고리즘보다 연산량이 적어 PC 환경보다 비교적 낮은 하드웨어 성능을 가지고 있어 모바일 기기에도 사용할 수 있다. 그러나 SURF 알고리즘은 모바일 기기에 적합한 최적화 작업이 되어있지 않다. 그러므로 본 논문에서는 모바일 기기에 적합한 추적을 위해 SURF 알고리즘을 여러 환경에서 실험하여 성능을 비교하고, 최적화 방안을 연구하였다. Augmented reality (AR) is augmented virtual information on the real world with real-time. And user can interact with information. In this paper, Marker-less tracking algorithm has been studied, for implement the augmented reality system on a mobile environment. In marker-less augmented reality, users do not need to attach the markers, and constrained the location. So, it's convenient to use. For marker-less tracking, I use the SURF algorithm based on feature point extraction in this paper. The SURF algorithm can be used on mobile devices because of the computational complexity is low. However, the SURF algorithm optimization work is not suitable for mobile devices. Therefore, in this paper, in order to the suitable tracking in mobile devices, the SURF algorithm was tested in a variety of environments. And ways to optimize has been studied.

Static and Dynamic Error of a Biplanar Videoradiography System Using Marker-Based and Markerless Tracking Techniques

The use of biplanar videoradiography technology has become increasingly popular for evaluating joint function in vivo. Two fundamentally different methods are currently employed to reconstruct 3D bone motions captured using this technology. Marker-based tracking requires at least three radio-opaque markers to be implanted in the bone of interest. Markerless tracking makes use of algorithms designed to match 3D bone shapes to biplanar videoradiography data. In order to reliably quantify in vivo bone motion, the systematic error of these tracking techniques should be evaluated. Herein, we present new markerless tracking software that makes use of modern GPU technology, describe a versatile method for quantifying the systematic error of a biplanar videoradiography motion capture system using independent gold standard instrumentation, and evaluate the systematic error of the W.M. Keck XROMM Facility's biplanar videoradiography system using both marker-based and markerless tracking algorithms under static and dynamic motion conditions. A polycarbonate flag embedded with 12 radio-opaque markers was used to evaluate the systematic error of the marker-based tracking algorithm. Three human cadaveric bones (distal femur, distal radius, and distal ulna) were used to evaluate the systematic error of the markerless tracking algorithm. The systematic error was evaluated by comparing motions to independent gold standard instrumentation. Static motions were compared to high accuracy linear and rotary stages while dynamic motions were compared to a high accuracy angular displacement transducer. Marker-based tracking was shown to effectively track motion to within 0.1 mm and 0.1 deg under static and dynamic conditions. Furthermore, the presented results indicate that markerless tracking can be used to effectively track rapid bone motions to within 0.15 deg for the distal aspects of the femur, radius, and ulna. Both marker-based and markerless tracking techniques were in excellent agreement with the gold standard instrumentation for both static and dynamic testing protocols. Future research will employ these techniques to quantify in vivo joint motion for high-speed upper and lower extremity impacts such as jumping, landing, and hammering.

TU‐E‐204B‐05: Feasibility of Markerless 3D Tumor Trajectory Tracking in CBCT Projections Using Digital Subtraction Method

Purpose: To evaluate feasibility of markerless 3D tumor trajectory tracking in CBCT projections using a digital subtraction technique for tumor enhancement in low contrast lesions. Method and Materials: 3D tumor position was estimated in CBCT measured projections (MP) by registering digital subtraction projections with templates. A fan beam 4DCT scan served as the planning CT (pRCCT), from which two sets of DRRs were generated using an in‐house GPU‐based algorithm. Tumor templates (TT) were generated by masking the GTV+0.5cm and projecting at the MP view angles for each 4D phase image. Subtraction templates (ST) were generated by first removing the GTV+0.5cm from the pRCCT, positioning the pRCCT at the pose of the patient during CBCT acquisition, and projecting at the MP angles for each 4D phase image. The ST was subtracted from the corresponding MP to create subtraction projections (SP). The 2D tumor position in each SP was calculated by matching with the corresponding TT using a robust correlation metric, and transformed to 3D positions by backprojection with a prior depth estimate. 3D position error was assessed by comparing to 3D positions of implanted markers, estimated with a separate algorithm, in a clinical scan. Results: The mean absolute error for the conventional method was 2.1mm, 2.5mm, and 2.9mm in mediolateral, anterior‐posterior, and superior‐inferior. The 90% error level was 4.1mm, 6.4mm, and 4.5mm. For the digital subtraction method, the mean errors were 0.8mm, 0.5mm, and 0.6mm and 90% errors were 1.4mm, 0.9mm, and 1mm. Conclusions: Digital subtraction is feasible for tumor enhancement to reduce error in tumor trajectory measurement. A framework for validation of markerless tracking algorithms using implanted markers was developed. Supported by NIH R01CA116249.

Shape-From-Silhouette Across Time Part II: Applications to Human Modeling and Markerless Motion Tracking

In Part I of this paper we developed the theory and algorithms for performing Shape-From-Silhouette (SFS) across time. In this second part, we show how our temporal SFS algorithms can be used in the applications of human modeling and markerless motion tracking. First we build a system to acquire human kinematic models consisting of precise shape (constructed using the temporal SFS algorithm for rigid objects), joint locations, and body part segmentation (estimated using the temporal SFS algorithm for articulated objects). Once the kinematic models have been built, we show how they can be used to track the motion of the person in new video sequences. This marker-less tracking algorithm is based on the Visual Hull alignment algorithm used in both temporal SFS algorithms and utilizes both geometric (silhouette) and photometric (color) information.