Marker-based Tracking Research Articles

Kinematic synergies show good consistency when extracted with a low-cost markerless device and a marker-based motion tracking system

Recently, markerless tracking systems, such as RGB-Depth cameras, have spread to overcome some of the limitations of the gold standard marker-based tracking systems. Although these systems are valuable substitutes for human motion analysis, as they guarantee higher flexibility, faster setup time and lower costs, their tracking accuracy is lower with respect to marker-based systems. Many studies quantified the error made by markerless systems in terms of body segment length estimation, articular angles, and biomechanics, concluding that they are appropriate for many clinical applications related to motion analysis. We propose an innovative approach to compare a markerless tracking system (Kinect V2) with a gold standard marker-based system (Vicon), based on motor control assessment. We quantified kinematic synergies from the tracking data of fifteen participants performing multi-directional upper limb movements. Kinematic synergy analysis decomposes the kinematic data into a reduced set of motor primitives that describe how the central nervous system coordinates motion at spatial and temporal level. Synergies were extracted with the recently released mixed-matrix factorization algorithm. Four synergies were extracted from both marker-based and markerless datasets and synergies were grouped in 6 clusters for each dataset. Cosine similarity in each cluster was ⩾0.60 in both systems, showing good consistency of synergies. Good matching was found between synergies extracted from markerless and from marker-based data, with a cosine similarity between matched synergies ⩾0.60 in 5 out 6 synergies. These results showed that the markerless sensor can be feasible for kinematic synergy analysis for gross movements, as it correctly estimates the number of synergies and in most cases also their spatial and temporal organization.

Visual control for robotic 3D printing on a moving platform

In recent years, there has been significant progress in developing specialized 3D printing techniques that cater to various demanding applications. However, the current state of this technology is challenged when it comes to complex in situ printing scenarios, which require a controlled printing platform. The lack of a stable printing platform is a fundamental limitation of its use in in situ applications. To address this issue, we present a novel platform-independent 3D fabrication process that enables printing on platforms with non-cooperative movement. The process overcomes the challenge of high-speed tracking, motion compensation, and real-time printing by developing a closed-loop visual feedback-controlled robotic printing process. The proposed process incorporates a marker-based visual detection and tracking controller setup, which is discussed in detail. The algorithm consists of two loops running asynchronously: a high-speed inner control loop and an outer measurement loop. This setup enables precise and accurate tracking of the printing platform, compensating for any disturbances during the printing process. Our experimental results demonstrate the successful printing of simple linear geometries, even with low-disturbing platform velocities. Moreover, the tracking controllers' ability to handle measurement occlusion is validated, showing the proposed process's robustness and effectiveness. Our work provides a significant step towards enabling 3D printing in complex in situ printing scenarios.

Impact of Obesity on Foot Kinematics: Greater Arch Compression and Metatarsophalangeal Joint Dorsiflexion despite Similar Joint Coupling Ratios

This study investigates the sagittal plane dynamics of the foot, particularly the metatarsophalangeal (MTP) joint and medial longitudinal arch (MLA) movements, in relation to obesity and foot health. The kinematics of the MTP and arch joints were measured in 17 individuals with class 2–3 obesity (BMI > 35 kg/m²) and 10 normal-weight individuals (BMI ≤ 24.9 kg/m²) using marker-based tracking. Analysis was conducted during heel lifting while seated and during walking at self-selected speeds. The results indicated that obese participants exhibited 20.92% greater MTP joint dorsiflexion at the end of the push-off phase and 19.84% greater MLA compression during the stance phase compared to normal-weight controls. However, no significant differences were found in the kinematic joint coupling ratio. While these findings reveal the different biomechanical behaviors of the MTP joint and MLA in obese compared to normal-weight individuals, it is important to interpret the implications of these differences with caution. This study identifies specific biomechanical variations that could be further explored to understand their potential impact on foot health in obese populations.

Real-Time Assessment of Rodent Engagement Using ArUco Markers: A Scalable and Accessible Approach for Scoring Behavior in a Nose-Poking Go/No-Go Task.

In the field of behavioral neuroscience, the classification and scoring of animal behavior play pivotal roles in the quantification and interpretation of complex behaviors displayed by animals. Traditional methods have relied on video examination by investigators, which is labor-intensive and susceptible to bias. To address these challenges, research efforts have focused on computational methods and image-processing algorithms for automated behavioral classification. Two primary approaches have emerged: marker- and markerless-based tracking systems. In this study, we showcase the utility of "Augmented Reality University of Cordoba" (ArUco) markers as a marker-based tracking approach for assessing rat engagement during a nose-poking go/no-go behavioral task. In addition, we introduce a two-state engagement model based on ArUco marker tracking data that can be analyzed with a rectangular kernel convolution to identify critical transition points between states of engagement and distraction. In this study, we hypothesized that ArUco markers could be utilized to accurately estimate animal engagement in a nose-poking go/no-go behavioral task, enabling the computation of optimal task durations for behavioral testing. Here, we present the performance of our ArUco tracking program, demonstrating a classification accuracy of 98% that was validated against the manual curation of video data. Furthermore, our convolution analysis revealed that, on average, our animals became disengaged with the behavioral task at ∼75 min, providing a quantitative basis for limiting experimental session durations. Overall, our approach offers a scalable, efficient, and accessible solution for automated scoring of rodent engagement during behavioral data collection.

Pengembangan Aplikasi Augmented Reality sebagai Media Pembelajaran Anatomi Tubuh Manusia Berbasis Android

Augmented reality (AR) provides intriguing opportunities to enhance the learning experience by merging the real world with interactive virtual elements. This research aims to develop an interactive learning application using augmented reality technology to introduce human body anatomy in a more engaging manner. The development method follows the waterfall system with stages of needs analysis, design, implementation, and testing. The researcher employs UML models, flowcharts, and wireframes in the system analysis and design. The application is built on the Unity platform with AR plugins such as Vuforia and ARCore. The AR model utilized in this development involves marker-based tracking and markerless tracking. Testing is conducted to measure the success level of the application, employing black box testing methods to ensure the application's usability without issues. The outcome of this research is an Android-based human body anatomy learning application utilizing augmented reality technology, with the expectation that it will facilitate a better understanding of the human anatomy system.

Overview of Radar-Based Gait Parameter Estimation Techniques for Fall Risk Assessment.

Current methods for fall risk assessment rely on Quantitative Gait Analysis (QGA) using costly optical tracking systems, which are often only available at specialized laboratories that may not be easily accessible to rural communities. Radar placed in a home or assisted living facility can acquire continuous ambulatory recordings over extended durations of a subject's natural gait and activity. Thus, radar-based QGA has the potential to capture day-to-day variations in gait, is time efficient and removes the burden for the subject to come to a clinic, providing a more realistic picture of older adults' mobility. Although there has been research on gait-related health monitoring, most of this work focuses on classification-based methods, while only a few consider gait parameter estimation. On the one hand, metrics that are accurately and easily computable from radar data have not been demonstrated to have an established correlation with fall risk or other medical conditions; on the other hand, the accuracy of radar-based estimates of gait parameters that are well-accepted by the medical community as indicators of fall risk have not been adequately validated. This paper provides an overview of emerging radar-based techniques for gait parameter estimation, especially with emphasis on those relevant to fall risk. A pilot study that compares the accuracy of estimating gait parameters from different radar data representations - in particular, the micro-Doppler signature and skeletal point estimates - is conducted based on validation against an 8-camera, marker-based optical tracking system. The results of pilot study are discussed to assess the current state-of-the-art in radar-based QGA and potential directions for future research that can improve radar-based gait parameter estimation accuracy.

Accuracy of routine external ventricular drain placement following a mixed reality-guided twist-drill craniostomy.

The traditional freehand placement of an external ventricular drain (EVD) relies on empirical craniometric landmarks to guide the craniostomy and subsequent passage of the EVD catheter. The diameter and trajectory of the craniostomy physically limit the possible trajectories that can be achieved during the passage of the catheter. In this study, the authors implemented a mixed reality-guided craniostomy procedure to evaluate the benefit of an optimally drilled craniostomy to the accurate placement of the catheter. Optical marker-based tracking using an OptiTrack system was used to register the brain ventricular hologram and drilling guidance for craniostomy using a HoloLens 2 mixed reality headset. A patient-specific 3D-printed skull phantom embedded with intracranial camera sensors was developed to automatically calculate the EVD accuracy for evaluation. User trials consisted of one blind and one mixed reality-assisted craniostomy followed by a routine, unguided EVD catheter placement for each of two different drill bit sizes. A total of 49 participants were included in the study (mean age 23.4 years, 59.2% female). The mean distance from the catheter target improved from 18.6 ± 12.5 mm to 12.7 ± 11.3 mm (p = 0.0008) using mixed reality guidance for trials with a large drill bit and from 19.3 ± 12.7 mm to 10.1 ± 8.4 mm with a small drill bit (p < 0.0001). Accuracy using mixed reality was improved using a smaller diameter drill bit compared with a larger bit (p = 0.039). Overall, the majority of the participants were positive about the helpfulness of mixed reality guidance and the overall mixed reality experience. Appropriate indications and use cases for the application of mixed reality guidance to neurosurgical procedures remain an area of active inquiry. While prior studies have demonstrated the benefit of mixed reality-guided catheter placement using predrilled craniostomies, the authors demonstrate that real-time quantitative and visual feedback of a mixed reality-guided craniostomy procedure can independently improve procedural accuracy and represents an important tool for trainee education and eventual clinical implementation.

Aplikasi Mobile Augmented Reality untuk Mendukung Pengenalan Aksara Sunda

Currently, the Sundanese script is slowly being ignored by the community, especially children, because of a lack of knowledge about the recognition of the letter. This script is only taught in schools with conventional models, thus making some children feel bored and less interested in learning Sundanese script. Augmented reality (AR) technology can now be utilized in the field of education. The purpose of this study is to build an AR application of Sundanese script using the marker-based tracking method to attract the interest of the community and children in learning Sundanese script. This application is made using Unity software. This research is development research using the Software Development Life Cycle (SDLC) Waterfall model. The stages carried out are the stages of analysis, design, implementation, and finally testing. Our findings are Sundanese AR applications using the marker-based tracking method. In this application, there are features like Main, Marker, Guide, History, and Quiz. The results of black box testing show in all features are working properly. Eligibility results with questionnaire instruments obtained an average percentage of 88.9% from 182 respondents who fell into the very feasible category. So, from previous research, the application was successfully developed with the Sundanese script Ngalagena, Swara, Angka, Rarangken and the history of Sundanese script and practice questions.

Penerapan Teknologi Augmented Reality sebagai Media Pembelajaran Interaktif Tata Surya

Learning the solar system through books has been a major choice over time. This learning is considered monotonous because of the limited interaction of students with the delivery of material. The book only presents text and static images of the solar system planets that limit their safety. The research is aimed at building an application as an interactive learning medium in the solar system that utilizes augmented reality technology to overcome restrictions on books. Using the Multimedia Development Life Cycle (MDLC) method as a systematic framework that has six stages, namely concept, design, material collection, assembly, testing, and distribution, The research has successfully built an augmented reality solar system application using Unity 3D on Android devices. The application uses a marker-based tracking method, where there is an image card that will be identified by the camera and then processed to display 3D Solar System planet objects, matter, and voice elements on the smartphone screen. Tested applications using black box testing techniques on application features and image markers show that all systems are running well, so applications can be distributed to Tlacap State SD. Through this built-in Solar System Augmented Reality application, learning becomes more interactive, and students' understanding of the planets of the Solar System can be enhanced.

Optical mapping of cardiac electromechanics in beating in vivo hearts

Optical mapping has been widely used in the study of cardiac electrophysiology in motion-arrested, ex vivo heart preparations. Recent developments in motion artifact mitigation techniques have made it possible to optically map beating ex vivo hearts, enabling the study of cardiac electromechanics using optical mapping. However, the ex vivo setting imposes limitations on optical mapping such as altered metabolic states, oversimplified mechanical loads, and the absence of neurohormonal regulation. In this study, we demonstrate optical electromechanical mapping in an in vivo heart preparation. Swine hearts were exposed via median sternotomy. Voltage-sensitive dye, either di-4-ANEQ(F)PTEA or di-5-ANEQ(F)PTEA, was injected into the left anterior descending artery. Fluorescence was excited by alternating green and amber light for excitation ratiometry. Cardiac motion during sinus and paced rhythm was tracked using a marker-based method. Motion tracking and excitation ratiometry successfully corrected most motion artifact in the membrane potential signal. Marker-based motion tracking also allowed simultaneous measurement of epicardial deformation. Reconstructed membrane potential and mechanical deformation measurements were validated using monophasic action potentials and sonomicrometry, respectively. Di-5-ANEQ(F)PTEA produced longer working time and higher signal/noise ratio than di-4-ANEQ(F)PTEA. In addition, we demonstrate potential applications of the new optical mapping system including electromechanical mapping during vagal nerve stimulation, fibrillation/defibrillation. and acute regional ischemia. In conclusion, although some technical limitations remain, optical mapping experiments that simultaneously image electrical and mechanical function can be conducted in beating, in vivo hearts.

Opportunities and challenges in applying reinforcement learning to robotic manipulation: An industrial case study

As manufacturing moves towards a more agile paradigm, industrial robots are expected to perform more complex tasks in less structured environments, complicating the use of traditional automation techniques and leading to growing interest in data-driven methods, like reinforcement learning (RL). In this work, we explore the process of applying RL to enable automation of a challenging industrial manipulation task. We focus on wire harness installation as a motivating example, which presents challenges for traditional automation due to the nonlinear dynamics of the deformable harness. A physical system was developed involving a three-terminal harness manipulated by a 6-DOF UR5 robot, with control enabled through a ROS interface. Modifications were made to the harness to enable simplified grasping and marker-based visual tracking. We detail the development of an RL formulation of the problem, subject to practical constraints on control and sensing motivated by the physical system. We develop a simulator and a basic scripted policy with which to safely generate a data-set of high-quality behaviors, then apply a state-of-the-art model-free offline RL algorithm, TD3 + BC, to learn a policy to serve as a safe starting point on the physical system. Despite extensive tuning, we find that the algorithm fails to achieve acceptable performance. We propose three failure modalities to explain the learning performance, related to control frequency, task symmetry arising from problem simplifications, and unexpected policy complexity, and discuss opportunities for future applications.

Approaches for Hybrid Coregistration of Marker-Based and Markerless Coordinates Describing Complex Body/Object Interactions.

Full-body motion capture is essential for the study of body movement. Video-based, markerless, mocap systems are, in some cases, replacing marker-based systems, but hybrid systems are less explored. We develop methods for coregistration between 2D video and 3D marker positions when precise spatial relationships are not known a priori. We illustrate these methods on three-ball cascade juggling in which it was not possible to use marker-based tracking of the balls, and no tracking of the hands was possible due to occlusion. Using recorded video and motion capture, we aimed to transform 2D ball coordinates into 3D body space as well as recover details of hand motion. We proposed four linear coregistration methods that differ in how they optimize ball-motion constraints during hold and flight phases, using an initial estimate of hand position based on arm and wrist markers. We found that minimizing the error between ball and hand estimate was globally suboptimal, distorting ball flight trajectories. The best-performing method used gravitational constraints to transform vertical coordinates and ball-hold constraints to transform lateral coordinates. This method enabled an accurate description of ball flight as well as a reconstruction of wrist movements. We discuss these findings in the broader context of video/motion capture coregistration.

Population-based asymmetric margins for moving targets in real-time tumor tracking.

Both geometric and dosimetric components are commonly considered when determining the margin for planning target volume (PTV). As dose distribution is shaped by controlling beam aperture in peripheral dose prescription and dose-escalated simultaneously integrated boost techniques, adjusting the margin by incorporating the variable dosimetric component into the PTV margin is inappropriate; therefore, geometric components should be accurately estimated for margin calculations. We introduced an asymmetric margin-calculation theory using the guide to the expression of uncertainty in measurement (GUM) and intra-fractional motion. The margins in fiducial marker-based real-time tumor tracking (RTTT) for lung, liver, and pancreatic cancers were calculated and were then evaluated using Monte Carlo (MC) simulations. A total of 74705, 73235, and 164968 sets of intra- and inter-fractional positional data were analyzed for 48 lung, 48 liver, and 25 pancreatic cancer patients, respectively, in RTTT clinical trials. The 2.5th and 97.5th percentiles of the positional error were considered representative values of each fraction of the disease site. The population-based statistics of the probability distributions of these representative positional errors (PD-RPEs) were calculated in six directions. A margin covering 95% of the population was calculated using the proposed formula. The content rate in which the clinical target volume (CTV) was included in the PTV was calculated through MC simulations using the PD-RPEs. The margins required for RTTT were at most 6.2, 4.6, and 3.9mm for lung, liver, and pancreatic cancer, respectively. MC simulations revealed that the median content rates using the proposed margins satisfied 95% for lung and liver cancers and 93% for pancreatic cancer, closer to the expected rates than the margins according to van Herk's formula. Our proposed formula based on the GUM and motion probability distributions (MPD) accurately calculated the practical margin size for fiducial marker-based RTTT. This was verified through MC simulations.

PENGEMBANGAN APLIKASI AUGMENTED REALITY SEBAGAI MEDIA PEMBELAJARAN UNTUK MENGENAL TATA SURYA PADA SD ISLAMIC VILLAGE

Educators must adapt their teaching methods during the COVID-19 pandemic to ensure effective and captivating learning experiences that incorporate interactivity, challenges, motivation, and allow students to nurture their creativity and independence based on their individual talents and interests. At Islamic Village Elementary School, it is especially important to utilize engaging learning media since elementary students have a natural enthusiasm for new concepts. When selecting learning media, factors such as learning objectives, effectiveness, user-friendliness, and flexibility should be taken into consideration. Augmented Reality (AR) applications have emerged as an exciting advancement in learning media, particularly in helping students comprehend Natural Science subjects. Thus, an AR application has been developed as a learning tool for Natural Science at Islamic Village Elementary School, utilizing the Marker-Based Tracking AR method. The research aims to facilitate students' learning and understanding of the material while making the learning process more captivating and less monotonous. Within the application, objects are visually presented in 3D using animations, sound, and appealing colors. The research methodology encompasses literature review, field study, and sample calculations. The outcome of this research is a mobile application that serves as a supportive tool to enhance the learning experience.

APLIKASI PENGENALAN OBJEK WISATA KOTA BATAM MENGGUNAKAN MARKER AUGMENTED REALITY

Batam City is an island in the north of Indonesia because it is at the crossroads between countries such as Singapore, Malaysia, Vietnam, Cambodia, and Thailand, so Batam City is a business and tourist destination. To increase domestic and international tourist visits, the Batam City Government has carried out promotions in various ways such as promotions through the Batam City Culture and Tourism Office website, social media. But there is no application to promote tourist attractions in Batam City, it is necessary to build an application for recognizing tourist objects in Batam City which includes visualization of tourist objects in 3D and information about tourist objects to make it easier for tourists to choose tourist objects according to their wishes. This research developed an application to recognize tourism objects in Batam City using augmented reality markers with the application of the SDLC (System Development Life Cycle) Waterfall model method. The augmented reality (AR) technology tracking method used in this study is marker-based tracking. Based on the results of functional testing, shows that the application functions properly. The results of the user satisfaction test gave a good value so the results found that the application was very well received by users.

Augmented Reality-Assisted versus Freehand Ventriculostomy in a Head Model.

Ventriculostomy (VST) is a frequent neurosurgical procedure. Freehand catheter placement represents the standard current practice. However, multiple attempts are often required. We present augmented reality (AR) headset guided VST with in-house developed head models. We conducted a proof of concept study in which we tested AR-guided as well as freehand VST. Repeated AR punctures were conducted to investigate if a learning curve can be derived. Five custom-made 3D-printed head models, each holding an anatomically different ventricular system, were filled with agarose gel. Eleven surgeons placed two AR-guided as well as two freehand ventricular drains per head. A subgroup of four surgeons did a total of three series of AR-guided punctures each to test for a learning curve. A Microsoft HoloLens served as the hardware platform. The marker-based tracking did not require rigid head fixation. Catheter tip position was evaluated in computed tomography scans. Marker-tracking, image segmentation, and holographic display worked satisfactorily. In freehand VST, a success rate of 72.7% was achieved, which was higher than under AR guidance (68.2%, difference not statistically significant). Repeated AR-guided punctures increased the success rate from 65 to 95%. We assume a steep learning curve as repeated AR-guided punctures led to an increase in successful attempts. Overall user experience showed positive feedback. We achieved promising results that encourage the continued development and technical improvement. However, several more developmental steps have to be taken before an application in humans can be considered. In the future, AR headset-based holograms have the potential to serve as a compact navigational help inside and outside the operating room.

Localisation of Ultrasonic NDT Data Using Hybrid Tracking of Component and Probe

Knowledge about the location of a defect is essential for damage assessment. In terms of a digitalised maintenance, inspection data is combined with position information. The presented approach regards the manual ultrasonic inspection, where the ultrasonic probe and the inspected component are both hand-held. By using markerless tracking technologies, it is possible to track the component without any markers. The ultrasonic probe is tracked by a more stable marker-based tracking technology. This results in a hybrid tracking system, which allows a referencing of the non-destructive testing (NDT) data directly to the local coordinate system of the 3D model that corresponds to the inspected component. Transferring this approach to other manual inspection technologies allows for a superimposition of recorded NDT data without any postprocessing or transformation. A better damage assessment is thus enabled. The inspection system, the inspection tool calibration and the camera registration process are described and analysed in detail. This work is focused on the analysis of the system accuracy, which is realised by using a reference body.

Introduction to Characteristics of Authenticity in Rupiah Based on Augmented Reality

This research is developing an application introduction to the characteristics of the authenticity of the rupiah by applying a system of Augmented Reality (AR) on android mobile devices. The Act of forgery of currency has a broad impact and is very detrimental to society. Congressional education applications introduction to the characteristics of the authenticity of the rupiah using augmented reality technology by applying techniques of marker-based tracking. This research made RUPIAR with the Multimedia Development Life Cycle (MDLC) method. Based on the testing that has been done, the results of the alpha testing are already functionally fit. From beta testing, test results from User Acceptance of functionality Test (UAT) obtained the value of 89.68%, declared eligible for use with the interpretation "Good", which means the application is worthy of being used and can be developed.

An accurate scapula registration process in shoulder arthroplasty using mixed reality.

Arthroplasty surgery can be challenging because of limited exposure of the bones. However, in shoulder arthroplasty for example, a good positioning of the glenoid component is essential to mitigate risks of revision surgeries. To improve the procedure's outcomes, mixed reality can be used as a guidance system relying on a fundamental registration step between the patient's bone and its corresponding 3D model. We present a complete registration workflow for shoulder arthroplasty using Hololens 2 Head Mounted Display. We rely on acquisitions made thanks to our marker-based tracking system, an improved Iterative Closest Point algorithm and verification steps. Our accuracy targets are 1.5 ± 1.5mm for the glenoid guidewire entry point on both antero-posterior and supero-inferior axes, and 1.5 ± 1[Formula: see text] for inclination and version. The overall process must last less than 5min. We have evaluated our process on a cohort of 13 3D printed glenoid bones of all types, showing an average accuracy of 0.84 ± 0.58mm on the antero-posterior axis and 0.49 ± 0.41mm on the supero-inferior one for the entry point. As for inclination and version, we have 0.89±0.6[Formula: see text] and 0.97±0.8[Formula: see text], respectively. The mean process time is about 1min 24s. We have developed a complete, embedded registration workflow and a verification protocol to evaluate our accuracy. Our results are promising for the improvement of the glenoid guidewire placement. Moreover, everything is performed in the field of view of the surgeon, which allows them to fully concentrate on the surgical site.

Estimation of Contact Regions Between Hands and Objects During Human Multi-Digit Grasping.

To grasp an object successfully, we must select appropriate contact regions for our hands on the surface of the object. However, identifying such regions is challenging. This paper describes a workflow to estimate the contact regions from marker-based tracking data. Participants grasp real objects, while we track the 3D position of both the objects and the hand, including the fingers' joints. We first determine the joint Euler angles from a selection of tracked markers positioned on the back of the hand. Then, we use state-of-the-art hand mesh reconstruction algorithms to generate a mesh model of the participant's hand in the current pose and the 3D position. Using objects that were either 3D printed or 3D scanned-and are, thus, available as both real objects and mesh data-allows the hand and object meshes to be co-registered. In turn, this allows the estimation of approximate contact regions by calculating the intersections between the hand mesh and the co-registered 3D object mesh. The method may be used to estimate where and how humans grasp objects under a variety of conditions. Therefore, the method could be of interest to researchers studying visual and haptic perception, motor control, human-computer interaction in virtual and augmented reality, and robotics.