Fiducial Marker Detection Research Articles

Vispro improves imaging analysis for Visium spatial transcriptomics.

Spatial transcriptomics (ST) is a cutting-edge technology that enables the comprehensive analysis of gene expression while preserving the spatial context of tissues. Within this framework, histological images serve a crucial role, providing spatially cohesive information that is often challenging to capture through gene expression alone. However, the imaging process in ST data presents inherent challenges, such as variability in image quality, artifacts from fiducial markers, and difficulty in distinguishing tissue regions. These challenges can significantly impair the accuracy and effectiveness of downstream analyses. To address these limitations, we developed Vispro, an end-to-end, fully automated image processing tool tailored for ST data. Vispro integrates four key modules: fiducial marker detection, marker removal and image restoration, tissue region detection, and segmentation of disconnected tissue areas. These modules systematically enhance the quality of ST images, reducing artifacts and improving tissue segmentation accuracy. Furthermore, by improving the integrity of image data, we demonstrate that Vispro facilitates downstream analyses such as cell segmentation and image registration, empowering researchers to extract more detailed and accurate biological insights from complex tissue architectures.

DeepArUco++: Improved detection of square fiducial markers in challenging lighting conditions

Fiducial markers are a computer vision tool used for object pose estimation and detection. These markers are highly useful in fields such as industry, medicine and logistics. However, optimal lighting conditions are not always available, and other factors such as blur or sensor noise can affect image quality. Classical computer vision techniques that precisely locate and decode fiducial markers often fail under difficult illumination conditions (e.g. extreme variations of lighting within the same frame). Hence, we propose DeepArUco++, a deep learning-based framework that leverages the robustness of Convolutional Neural Networks to perform marker detection and decoding in challenging lighting conditions. The framework is based on a pipeline using different Neural Network models at each step, namely marker detection, corner refinement and marker decoding. Additionally, we propose a simple method for generating synthetic data for training the different models that compose the proposed pipeline, and we present a second, real-life dataset of ArUco markers in challenging lighting conditions used to evaluate our system. The developed method outperforms other state-of-the-art methods in such tasks and remains competitive even when testing on the datasets used to develop those methods. Code available in GitHub: https://github.com/AVAuco/deeparuco/.

Multi-sensor fusion for robust indoor localization of industrial UAVs using particle filter

Abstract Robotic platforms including Unmanned Aerial Vehicles (UAVs) require an accurate and reliable source of position information, especially in indoor environments where GNSS cannot be used. This is typically accomplished by using multiple independent position sensors. This paper presents a UAV position estimation mechanism based on a particle filter, that combines information from visual odometry cameras and visual detection of fiducial markers. The article proposes very compact, lightweight and robust method for indoor localization, that can run with high frequency on the UAV’s onboard computer. The filter is implemented such that it can seamlessly handle sensor failures and disconnections. Moreover, the filter can be extended to include inputs from additional sensors. The implemented approach is validated on data from real-life UAV test flights, where average position error under 0.4 m was achieved.

Automated fiducial-based alignment of cryo-electron tomography tilt series in Dynamo

With the advent of modern technologies for cryo-electron tomography (cryo-ET), high-quality tilt series are more rapidly acquired than processed and analyzed. Thus, a robust and fast-automated alignment for batch processing in cryo-ET is needed. While different software packages have made available several approaches for automated marker-based alignment of tilt series, manual user intervention remains necessary for many datasets, thus preventing high-throughput tomography. We have developed a MATLAB-based framework integrated into the Dynamo software package for automatic detection of fiducial markers that generates a robust alignment model with minimal input parameters. This approach allows high-throughput, unsupervised volume reconstruction. This new module extends Dynamo with a large repertory of tools for tomographic alignment and reconstruction, as well as specific visualization browsers to rapidly assess the biological relevance of the dataset. Our approach has been successfully tested on a broad range of datasets that include diverse biological samples and cryo-ET modalities.

MarkerDetector: A method for robust fiducial marker detection in electron micrographs using wavelet-based template

Fiducial marker detection in electron micrographs becomes an important and challenging task with the development of large-field electron microscopy. The fiducial marker detection plays an important role in several steps during the process of electron micrographs, such as the alignment and parameter calibrations. However, limited by the conditions of low signal-to-noise ratio (SNR) in the electron micrographs, the performance of fiducial marker detection is severely affected. In this work, we propose the MarkerDetector, a novel algorithm for detecting fiducial markers in electron micrographs. The proposed MarkerDetector is built upon the following contributions: Firstly, a wavelet-based template generation algorithm is devised in MarkerDetector. By adopting a shape-based criterion, a high-quality template can be obtained. Secondly, a robust marker determination strategy is devised by utilizing statistic-based filtering, which can guarantee the correctness of the detected fiducial markers. The average running time of our algorithm is 1.67 seconds with promising accuracy, indicating its practical feasibility for applications in electron micrographs.

Multiple Projector Camera Calibration by Fiducial Marker Detection

Multiple Projector Camera Calibration by Fiducial Marker Detection

Fiducial marker recovery and detection from severely truncated data in navigation-assisted spine surgery.

Fiducial markers are commonly used in navigation-assisted minimally invasive spine surgery and they help transfer image coordinates into real-world coordinates. In practice, these markers might be located outside the field-of-view (FOV) of C-arm cone-beam computed tomography (CBCT) systems used in intraoperative surgeries, due to the limited detector sizes. As a consequence, reconstructed markers in CBCT volumes suffer from artifacts and have distorted shapes, which sets an obstacle fornavigation. In this work, we propose two fiducial marker detection methods: direct detection from distorted markers (direct method) and detection after marker recovery (recovery method). For direct detection from distorted markers in reconstructed volumes, an efficient automatic marker detection method using two neural networks and a conventional circle detection algorithm is proposed. For marker recovery, a task-specific data preparation strategy is proposed to recover markers from severely truncated data. Afterwards, a conventional marker detection algorithm is applied for position detection. The networks in both methods are trained based on simulated data. For the direct method, 6800 images and 10000 images are generated, respectively, to train the U-Net and ResNet50. For the recovery method, the training set includes 1360 images for FBPConvNet and Pix2pixGAN. The simulated data set with 166 markers and four cadaver cases with real fiducials are used forevaluation. The two methods are evaluated on simulated data and real cadaver data. The direct method achieves 100% detection rates within 1mm detection error on simulated data with normal truncation and simulated data with heavier noise, but only detect 94.6% markers in extremely severe truncation case. The recovery method detects all the markers successfully in three test data sets and around 95% markers are detected within 0.5mm error. For real cadaver data, both methods achieve 100% marker detection rates with mean registration error below 0.2mm. Our experiments demonstrate that the direct method is capable of detecting distorted markers accurately and the recovery method with the task-specific data preparation strategy has high robustness and generalizability on various data sets. The task-specific data preparation is able to reconstruct structures of interest outside the FOV from severely truncated data better than conventional datapreparation.

A feasibility study on the development and use of a deep learning model to automate real-time monitoring of tumor position and assessment of interfraction fiducial marker migration in prostate radiotherapy patients * *Deep Learning Fiducial Marker Localisation.

Purpose. The aim of this study was to assess the feasibility of the development and training of a deep learning object detection model for automating the assessment of fiducial marker migration and tracking of the prostate in radiotherapy patients. Methods and Materials. A fiducial marker detection model was trained on the YOLO v2 detection framework using approximately 20,000 pelvis kV projection images with fiducial markers labelled. The ability of the trained model to detect marker positions was validated by tracking the motion of markers in a respiratory phantom and comparing detection data with the expected displacement from a reference position. Marker migration was then assessed in 14 prostate radiotherapy patients using the detector for comparison with previously conducted studies. This was done by determining variations in intermarker distance between the first and subsequent fractions in each patient. Results. On completion of training, a detection model was developed that operated at a 96% detection efficacy and with a root mean square error of 0.3 pixels. By determining the displacement from a reference position in a respiratory phantom, experimentally and with the detector it was found that the detector was able to compute displacements with a mean accuracy of 97.8% when compared to the actual values. Interfraction marker migration was measured in 14 patients and the average and maximum standard deviation marker migration were found to be and respectively. Conclusion. This study demonstrates the benefits of pairing deep learning object detection, and image-guided radiotherapy and how a workflow to automate the assessment of organ motion and seed migration during prostate radiotherapy can be developed. The high detection efficacy and low error make evident the advantages of using a pre-trained model to automate the assessment of the target volume positional variation and the migration of fiducial markers between fractions.

Hybrid 3D T1-weighted gradient-echo sequence for fiducial marker detection and tumor delineation via magnetic resonance imaging in liver stereotactic body radiation therapy

PurposeGold fiducial markers are used to guide liver stereotactic body radiation therapy (SBRT) and are hard to detect by magnetic resonance imaging (MRI). In this study, the parameters of the three-dimensional T1-weighted turbo gradient-echo (3D T1W-GRE) sequence were optimized for gold marker detection without degrading tumor delineation. MethodsCustom-made phantoms mimicking tumor and normal liver parenchyma were prepared and embedded with a gold marker. The 3D T1W-GRE was scanned by varying echo time (TE), bandwidth (BW), flip angle (FA), and base matrix size. The signal-to-noise ratio (SNR), contrast ratio (CR), and relative standard deviation (RSD) of the signal intensity in the area including the gold marker were evaluated, and the parameters were optimized accordingly. The modified 3D T1W-GRE (called HYBRID) was compared with the conventional T1W-GRE- and T2*-sequences in both phantom and clinical studies. In the clinical study of six patients with primary liver tumors, two observers visually assessed marker detection, tumor delineation, and overall image quality on a four-point scale. ResultsIn the phantom study, HYBRID showed significantly higher SNR and RSD than those of conventional T1W-GRE (P < 0.001). In the clinical study, HYBRID yielded significantly higher scores than conventional T1W-GRE did in terms of marker detection (P < 0.001). The scores of both sequences were not statistically different in terms of tumor delineation and overall image quality (P = 0.56 and P = 0.32). ConclusionsThe proposed HYBRID sequence improved gold fiducial marker detection without degrading tumor delineation in MRI for SBRT of primary liver tumor.

DeepTag: A General Framework for Fiducial Marker Design and Detection.

A fiducial marker system usually consists of markers, a detection algorithm, and a coding system. The appearance of markers and the detection robustness are generally limited by the existing detection algorithms, which are hand-crafted with traditional low-level image processing techniques. Furthermore, a sophisticatedly designed coding system is required to overcome the shortcomings of both markers and detection algorithms. To improve the flexibility and robustness in various applications, we propose a general deep learning based framework, DeepTag, for fiducial marker design and detection. DeepTag not only supports detection of a wide variety of existing marker families, but also makes it possible to design new marker families with customized local patterns. Moreover, we propose an effective procedure to synthesize training data on the fly without manual annotations. Thus, DeepTag can easily adapt to existing and newly-designed marker families. To validate DeepTag and existing methods, beside existing datasets, we further collect a new large and challenging dataset where markers are placed in different view distances and angles. Experiments show that DeepTag well supports different marker families and greatly outperforms the existing methods in terms of both detection robustness and pose accuracy. Both code and dataset are available at https://herohuyongtao.github.io/research/publications/deep-tag/.

Visual Navigation System for Autonomous Drone using Fiducial Marker Detection

Drones have been quickly developing for civilian applications in recent years. Because of the nonlinearity of the mathematical drone model, and the importance of precise navigation to avoid possible dangers, it is necessary to establish an algorithm to localize the drone simultaneously and maneuver it to the desired destination. This paper presents a visual-based multi-stage error tolerance navigation algorithm of an autonomous drone by a tag-based fiducial marker detection in finding its target. Dynamic and kinematic models of the drone were developed by Newton-Euler. The position and orientation of the drone, related to the tag, are determined by AprilTag, which is used as feedback in a closed-loop control system with an Adjustable Proportional-Integral-Derivative (APID) controller. Parameters of the controller are tuned based on steady-State error, which is defined as the distance of the drone from the desired point. The sequence of path trajectory, that drone follows to reach the desired point, is defined as a navigation algorithm. A model of the drone was simulated in a virtual outdoor to mimic hovering in complex obstacles environment. The results present satisfactory performance of the navigation system programmed by the APID controller in comparison with the conventional Proportional-Integral-Derivative (PID) controller. It can be ascertained that the proposed navigation system based on a tag marker in the closed-loop control system is applicable to maneuvering the drone autonomously and useful for various industrial tasks in indoor/outdoor environments.

Thermoelasticity and ArUco marker-based model validation of polymer structure: application to the San Giorgio’s bridge inspection robot

&lt;p&gt;Experimental procedures are often involved in the numerical models validation. To define the behaviour of a structure, its underlying dynamics and stress distributions are generally investigated. In this research, a multi-instrumental and multi-spectral method is proposed in order to validate the numerical model of the Inspection Robot mounted on the new San Giorgio's Bridge on the Polcevera river. An infrared thermoelasticity-based approach is used to measure stress-concentration factors and, additionally, an innovative methodology is implemented to define the natural frequencies of the Robot Inspection structure, based on the detection of ArUco fiducial markers. Established impact hammer procedure is also performed for the validation of the results.&lt;/p&gt;

Space Vision

This paper presents a novel mobile serious game, "Space Vision", which uses a hidden-object mechanic with fiducial marker detection to gamify a clinical test of visual acuity - a key marker of childhood eye disease. For Space Vision to become a credible clinical tool that can facilitate the screening and home-monitoring of children's visual acuity, it must be able to sustain player engagement over the extended durations required to detect vision abnormalities. Hence, we pay particular attention to developing effective game characters - a crucial aspect of children's game design. Using an early prototype with 13 school children (aged 5-6 years), we investigate player experiences through a series of evaluation sessions, involving a single one-to-one observational playtest, semi-structured interview, and ideation activity with each child. Thematic analysis of our session video recordings, written observations, and ideation artefacts, found that future game iterations must: increase resonance between players and game characters by providing aesthetic and behavioural customisation, embrace more anthropomorphic styling, better imbue narratives pertaining to children's life scripts, and feature fantasy character powers as a form of self-expression. Meanwhile, greater physical device support and digital incentivisation of body posture and head position are key to improving the reliability of the visual acuity measurement.

Automated fiducial marker detection and localization in volumetric computed tomography images: a three-step hybrid approach with deep learning.

.Purpose: Automating fiducial detection and localization in the patient’s pre-operative images can lead to better registration accuracy, reduced human errors, and shorter intervention time. Most current approaches are optimized for a single marker type, mainly spherical adhesive markers. A fully automated algorithm is proposed and evaluated for screw and spherical titanium fiducials, typically used in high-accurate frameless surgical navigation.Approach: The algorithm builds on previous approaches with morphological functions and pose estimation algorithms. A 3D convolutional neural network (CNN) is proposed for the fiducial classification task and evaluated for both traditional closed-set and emerging open-set classifiers. A proposed digital ground-truth experiment, with cone-beam computed tomography (CBCT) imaging software, is performed to determine the localization accuracy of the algorithm. The localized fiducial positions in the CBCT images by the presented algorithm were compared to the actual known positions in the virtual phantom models. The difference represents the fiducial localization error (FLE).Results: A total of 241 screws, 151 spherical fiducials, and 1550 other structures are identified with the best true positive rate 95.9% for screw and 99.3% for spherical fiducials at 8.7% and 3.4% false positive rate, respectively. The best achieved FLE mean and its standard deviation for a screw and spherical marker are 58 (14) and , respectively.Conclusions: Accurate marker detection and localization were achieved, with spherical fiducials being superior to screws. Large marker volume and smaller voxel size yield significantly smaller FLEs. Attenuating noise by mesh smoothing has a minor effect on FLE. Future work will focus on expanding the CNN for image segmentation.

LiDARTag: A Real-Time Fiducial Tag System for Point Clouds

Image-based fiducial markers are useful in problems such as object tracking in cluttered or textureless environments, camera (and multi-sensor) calibration tasks, and vision-based simultaneous localization and mapping (SLAM). The state-of-the-art fiducial marker detection algorithms rely on the consistency of the ambient lighting. This paper introduces LiDARTag, a novel fiducial tag design and detection algorithm suitable for light detection and ranging (LiDAR) point clouds. The proposed method runs in real-time and can process data at 100 Hz, which is faster than the currently available LiDAR sensor frequencies. Because of the LiDAR sensors' nature, rapidly changing ambient lighting will not affect the detection of a LiDARTag; hence, the proposed fiducial marker can operate in a completely dark environment. In addition, the LiDARTag nicely complements and is compatible with existing visual fiducial markers, such as AprilTags, allowing for efficient multi-sensor fusion and calibration tasks. We further propose a concept of minimizing a fitting error between a point cloud and the marker's template to estimate the marker's pose. The proposed method achieves millimeter error in translation and a few degrees in rotation. Due to LiDAR returns' sparsity, the point cloud is lifted to a continuous function in a reproducing kernel Hilbert space where the inner product can be used to determine a marker's ID. The experimental results, verified by a motion capture system, confirm that the proposed method can reliably provide a tag's pose and unique ID code. The rejection of false positives is validated on the Google Cartographer indoor dataset and the Honda H3D outdoor dataset. All implementations are coded in C++ and are available at: https://github.com/UMich-BipedLab/LiDARTag.

Hybrid Spine Simulator Prototype for X-ray Free Pedicle Screws Fixation Training

Simulation for surgical training is increasingly being considered a valuable addition to traditional teaching methods. 3D-printed physical simulators can be used for preoperative planning and rehearsal in spine surgery to improve surgical workflows and postoperative patient outcomes. This paper proposes an innovative strategy to build a hybrid simulation platform for training of pedicle screws fixation: the proposed method combines 3D-printed patient-specific spine models with augmented reality functionalities and virtual X-ray visualization, thus avoiding any exposure to harmful radiation during the simulation. Software functionalities are implemented by using a low-cost tracking strategy based on fiducial marker detection. Quantitative tests demonstrate the accuracy of the method to track the vertebral model and surgical tools, and to coherently visualize them in either the augmented reality or virtual fluoroscopic modalities. The obtained results encourage further research and clinical validation towards the use of the simulator as an effective tool for training in pedicle screws insertion in lumbar vertebrae.

Prospective dual-surrogate validation study of periodic imaging during treatment for accurately monitoring intrafraction motion of prostate cancer patients

Background and purposeThe goal of this prospective study is to validate the use of periodic imaging during treatment with a fiducial marker detection algorithm using radiofrequency transponders for prostate cancer patients undergoing treatment for radiation therapy. Materials and methodsTen male patients were enrolled in this study and treated for prostate cancer with implanted electromagnetic monitoring beacons. We evaluated the accuracy and limitations of Intrafraction Motion Review (IMR) by comparing the known locations of the beacons using the electromagnetic monitoring system to the position data reported from IMR images. ResultsA total of 4054 images were taken during treatment. The difference in vector magnitude of the two methods is centered around zero (mean: 0.03 cm, SD: 0.16 cm) and Lin’s Concordance Correlation Coefficient (CCC) is 0.99 (95% CI: 0.98, 1) overall. The Euclidean distance between the two methods was close to zero (median: 0.09 cm, IQR: 0.06, 0.14 cm). The difference in distance between any two markers was centered around zero (mean: 0.01 cm, SD: 0.12 cm) and Lin’s CCC is 0.97 (95% CI: 0.96, 0.98) overall. ConclusionThe accuracy of the algorithm for detected markers within the 2D images is comparable to electromagnetic monitoring for fiducial identification when detected. IMR could provide an alternate solution for patients with contraindications of use of an electromagnetic monitoring system and a cost effective alternative to the acquisition of an additional system for patient monitoring, but does not provide data for pre-treatment set-up verification and real-time 3D positioning during treatment.

Detection of Binary Square Fiducial Markers Using an Event Camera

Event cameras are a new type of image sensors that output changes in light intensity (events) instead of absolute intensity values. They have a very high temporal resolution and a high dynamic range. In this paper, we propose a method to detect and decode binary square markers using an event camera. We detect the edges of the markers by detecting line segments in an image created from events in the current packet. The line segments are combined to form marker candidates. The bit value of marker cells is decoded using the events on their borders. To the best of our knowledge, no other approach exists for detecting square binary markers directly from an event camera using only the CPU unit in real-time. Experimental results show that the performance of our proposal is much superior to the one from the RGB ArUco marker detector. The proposed method can achieve the real-time performance on a single CPU thread.

Simple Optical Coordinate Measuring System, Based on Fiducial Markers Detection, and its Accuracy Assessment

Simple Optical Coordinate Measuring System, Based on Fiducial Markers Detection, and its Accuracy Assessment

A generative adversarial network-based (GAN-based) architecture for automatic fiducial marker detection in prostate MRI-only radiotherapy simulation images.

Clinical sites utilizing magnetic resonance imaging (MRI)-only simulation for prostate radiotherapy planning typically use fiducial markers for pretreatment patient positioning and alignment. Fiducial markers appear as small signal voids in MRI images and are often difficult to discern. Existing clinical methods for fiducial marker localization require multiple MRI sequences and/or manual interaction and specialized expertise. In this study, we develop a robust method for automatic fiducial marker detection in prostate MRI simulation images and quantify the pretreatment alignment accuracy using automatically detected fiducial markers in MRI. In this study, a deep learning-based algorithm was used to convert MRI images into labeled fiducial marker volumes. Seventy-seven prostate cancer patients who received marker implantation prior to MRI and CT simulation imaging were selected for this study. Multiple-Echo T1 -VIBE MRI images were acquired, and images were stratified (at the patient level) based on the presence of intraprostatic calcifications. Ground truth (GT) contours were defined by an expert on MRI using CT images. Training was done using the pix2pix generative adversarial network (GAN) image-to-image translation package and model testing was done using fivefold cross validation. For performance comparison, an experienced medical dosimetrist and a medical physicist each manually contoured fiducial markers in MRI images. The percent of correct detections and F1 classification scores are reported for markers detected using the automatic detection algorithm and human observers. The patient positioning errors were quantified by calculating the target registration errors (TREs) from fiducial marker driven rigid registration between MRI and CBCT images. Target registration errors were quantified for fiducial marker contours defined on MRI by the automatic detection algorithm and the two expert human observers. Ninety-six percent of implanted fiducial markers were correctly identified using the automatic detection algorithm. Two expert raters correctly identified 97% and 96% of fiducial markers, respectively. The F1 classification score was 0.68, 0.75, and 0.72 for the automatic detection algorithm and two human raters, respectively. The main source of false discoveries was intraprostatic calcifications. The mean TRE differences between alignments from automatic detection algorithm and human detected markers and GT were <1mm. We have developed a deep learning-based approach to automatically detect fiducial markers in MRI-only simulation images in a clinically representative patient cohort. The automatic detection algorithm-predicted markers can allow for patient setup with similar accuracy to independent human observers.