Augmented Reality Neuronavigation Research Articles

Validation of real-time inside-out tracking and depth realization technologies for augmented reality-based neuronavigation.

Concomitant with the significant advances in computing technology, the utilization of augmented reality-based navigation in clinical applications is being actively researched. In this light, we developed novel object tracking and depth realization technologies to apply augmented reality-based neuronavigation to brain surgery. We developed real-time inside-out tracking based on visual inertial odometry and a visual inertial simultaneous localization and mapping algorithm. The cube quick response marker and depth data obtained from light detection and ranging sensors are used for continuous tracking. For depth realization, order-independent transparency, clipping, and annotation and measurement functions were developed. In this study, the augmented reality model of a brain tumor patient was applied to its life-size three-dimensional (3D) printed model. Using real-time inside-out tracking, we confirmed that the augmented reality model remained consistent with the 3D printed patient model without flutter, regardless of the movement of the visualization device. The coordination accuracy during real-time inside-out tracking was also validated. The average movement error of the X and Y axes was 0.34 ± 0.21 and 0.04 ± 0.08mm, respectively. Further, the application of order-independent transparency with multilayer alpha blending and filtered alpha compositing improved the perception of overlapping internal brain structures. Clipping, and annotation and measurement functions were also developed to aid depth perception and worked perfectly during real-time coordination. We named this system METAMEDIP navigation. The results validate the efficacy of the real-time inside-out tracking and depth realization technology. With these novel technologies developed for continuous tracking and depth perception in augmented reality environments, we are able to overcome the critical obstacles in the development of clinically applicable augmented reality neuronavigation.

Augmented Reality Neuronavigation for En Bloc Resection of Spinal Column Lesions

Primary tumors involving the spine are relatively rare but represent surgically challenging procedures with high patient morbidity. En bloc resection of these tumors necessitates large exposures, wide tumor margins, and poses risks to functionally relevant anatomical structures. Augmented reality neuronavigation (ARNV) represents a paradigm shift in neuronavigation, allowing on-demand visualization of 3D navigation data in real-time directly in line with the operative field. Here, we describe the first application of ARNV to perform distal sacrococcygectomies for the en bloc removal of sacral and retrorectal lesions involving the coccyx in 2 patients, as well as a thoracic 9-11 laminectomy with costotransversectomy for en bloc removal of a schwannoma in a third patient. In our experience, ARNV allowed our teams to minimize the length of the incision, reduce the extent of bony resection, and enhanced visualization of critical adjacent anatomy. All tumors were resected en bloc, and the patients recovered well postoperatively, with no known complications. Pathologic analysis confirmed the en bloc removal of these lesions with negative margins. We conclude that ARNV is an effective strategy for the precise, en bloc removal of spinal lesions including both sacrococcygeal tumors involving the retrorectal space and thoracic schwannomas.

A Neuronavigation System Using a Mobile Augmented Reality Solution

Image-guided surgery has shown great utility in neurosurgery, especially in allowing for more accurate surgical planning and navigation. The current gold standard for image-guided neurosurgery is neuronavigation, which provides millimetric accuracy on such tasks. However, these approaches often require a complicated setup and have high cost, hindering their potential in low- and middle-income countries. The aim of this study was to develop and evaluate the performance of a mobile-based augmented reality neuronavigation solution under different conditions in a preclinical environment. The application was developed using the Swift programming language and was tested on a replica of a human scalp under variable lighting, with different numbers of registration points and target point position conditions. For each condition, reference points were input into the application, and the target points were computed for 10 iterations. The mean registration error and target error were used to assess the performance of the application. In the best-case scenario, the proposed solution had a mean target error of 2.6 ± 1.6 mm. Our approach provides a viable, low-cost, easy-to-use, portable method for locating points on the scalp surface with an accuracy of 2.6 ± 1.6 mm in the best-case scenario.

Stereotactic co-axial projection imaging for augmented reality neuronavigation: a proof-of-concept study.

Lack of intuitiveness and poor hand-eye coordination present a major technical challenge in neurosurgical navigation. We developed an integrated dexterous stereotactic co-axial projection imaging (sCPI) system featuring orthotopic image projection for augmented reality (AR) neurosurgical navigation. The performance characteristics of the sCPI system, including projection resolution and navigation accuracy, were quantitatively verified. The resolution of the sCPI was tested with a USAF1951 resolution test chart. The stereotactic navigation accuracy of the sCPI was measured using a calibration panel with a 7×7 circle array pattern. In benchtop validation, the navigation accuracy of the sCPI and the BrainLab Kick Navigation Station was compared using a skull phantom with 8 intracranial targets. Finally, we demonstrated the potential clinical application of sCPI through a clinical trial. The resolution test showed that the resolution of the sCPI was 1.3 mm. In a stereotactic navigation accuracy test, the maximum and minimum error of the sCPI was 2.9 and 0.3 mm, and the mean error was 1.5 mm. The stereotactic navigation accuracy test also showed that the navigation error of the sCPI would increase with the pitch and yaw angle, but there was no obvious difference in navigation errors caused by different yaw directions, which meant that the navigation error is unbiased across all directions. The benchtop validation showed that the average navigation errors for the sCPI system and the Kick Navigation Station were 1.4±0.8 and 1.8±0.7 mm, the medians were 1.3 and 1.9 mm, and the average preparation times were 3 min 24 sec and 6 min 8 sec, respectively. The clinical feasibility of sCPI-assisted neurosurgical navigation was demonstrated in a clinical study. In comparison with the BrainLab device, the sCPI system required less time for preoperative preparation and enhanced the clinician experience in intraoperative visualization and navigation. The sCPI technique can be potentially used in many surgical applications for intuitive visualization of medical information and intraoperative guidance of surgical trajectories.

Three-dimensional-printed marker-based augmented reality neuronavigation: a new neuronavigation technique.

The aim of this study was to assess the precision and feasibility of 3D-printed marker-based augmented reality (AR) neurosurgical navigation and its use intraoperatively compared with optical tracking neuronavigation systems (OTNSs). Three-dimensional-printed markers for CT and MRI and intraoperative use were applied with mobile devices using an AR light detection and ranging (LIDAR) camera. The 3D segmentations of intracranial tumors were created with CT and MR images, and preoperative registration of the marker and pathology was performed. A patient-specific, surgeon-facilitated mobile application was developed, and a mobile device camera was used for neuronavigation with high accuracy, ease, and cost-effectiveness. After accuracy values were preliminarily assessed, this technique was used intraoperatively in 8 patients. The mobile device LIDAR camera was found to successfully overlay images of virtual tumor segmentations according to the position of a 3D-printed marker. The targeting error that was measured ranged from 0.5 to 3.5 mm (mean 1.70 ± 1.02 mm, median 1.58 mm). The mean preoperative preparation time was 35.7 ± 5.56 minutes, which is longer than that for routine OTNSs, but the amount of time required for preoperative registration and the placement of the intraoperative marker was very brief compared with other neurosurgical navigation systems (mean 1.02 ± 0.3 minutes). The 3D-printed marker-based AR neuronavigation system was a clinically feasible, highly precise, low-cost, and easy-to-use navigation technique. Three-dimensional segmentation of intracranial tumors was targeted on the brain and was clearly visualized from the skin incision to the end of surgery.

First experience with augmented reality neuronavigation in endoscopic assisted midline skull base pathologies in children

IntroductionEndoscopic skull base approaches are broadly used in modern neurosurgery. The support of neuronavigation can help to effectively target the lesion avoiding complications. In children, endoscopic-assisted skull base surgery in combination with navigation systems becomes even more important because of the morphological variability and rare diseases affecting the sellar and parasellar regions. This paper aims to analyze our first experience on augmented reality navigation in endoscopic skull base surgery in a pediatric case series.Patients and methodsA retrospective review identified seventeen endoscopic-assisted endonasal or transoral procedures performed in an interdisciplinary setting in a period between October 2011 and May 2020. In all the cases, the surgical target was a lesion in the sellar or parasellar region. Clinical conditions, MRI appearance, intraoperative conditions, postoperative MRI, possible complications, and outcomes were analyzed.ResultsThe mean age of our patients was 14.5 ± 2.4 years. The diagnosis varied, but craniopharyngiomas (31.2%) were mostly represented. AR navigation was experienced to be very helpful for effectively targeting the lesion and defining the intraoperative extension of the pathology. In 65% of the oncologic cases, a radical removal was proven in postoperative MRI. The mean follow-up was 89 ± 79 months. There were no deaths in our series. No long-term complications were registered; two cerebrospinal fluid (CSF) fistulas and a secondary abscess required further surgery.ConclusionThe implementation of augmented reality to endoscopic-assisted neuronavigated procedures within the skull base was feasible and did provide relevant information directly in the endoscopic field of view and was experienced to be useful in the pediatric cases, where anatomical variability and rarity of the pathologies make surgery more challenging.

Holographic patient tracking after bed movement for augmented reality neuronavigation using a head-mounted display

BackgroundHolographic neuronavigation has several potential advantages compared to conventional neuronavigation systems. We present the first report of a holographic neuronavigation system with patient-to-image registration and patient tracking with a reference array using an augmented reality head-mounted display (AR-HMD).MethodsThree patients undergoing an intracranial neurosurgical procedure were included in this pilot study. The relevant anatomy was first segmented in 3D and then uploaded as holographic scene in our custom neuronavigation software. Registration was performed using point-based matching using anatomical landmarks. We measured the fiducial registration error (FRE) as the outcome measure for registration accuracy. A custom-made reference array with QR codes was integrated in the neurosurgical setup and used for patient tracking after bed movement.ResultsSix registrations were performed with a mean FRE of 8.5 mm. Patient tracking was achieved with no visual difference between the registration before and after movement.ConclusionsThis first report shows a proof of principle of intraoperative patient tracking using a standalone holographic neuronavigation system. The navigation accuracy should be further optimized to be clinically applicable. However, it is likely that this technology will be incorporated in future neurosurgical workflows because the system improves spatial anatomical understanding for the surgeon.

Gesture-based registration correction using a mobile augmented reality image-guided neurosurgery system.

In image-guided neurosurgery, a registration between the patient and their pre-operative images and the tracking of surgical tools enables GPS-like guidance to the surgeon. However, factors such as brainshift, image distortion, and registration error cause the patient-to-image alignment accuracy to degrade throughout the surgical procedure no longer providing accurate guidance. The authors present a gesture-based method for manual registration correction to extend the usage of augmented reality (AR) neuronavigation systems. The authors’ method, which makes use of the touchscreen capabilities of a tablet on which the AR navigation view is presented, enables surgeons to compensate for the effects of brainshift, misregistration, or tracking errors. They tested their system in a laboratory user study with ten subjects and found that they were able to achieve a median registration RMS error of 3.51 mm on landmarks around the craniotomy of interest. This is comparable to the level of accuracy attainable with previously proposed methods and currently available commercial systems while being simpler and quicker to use. The method could enable surgeons to quickly and easily compensate for most of the observed shift. Further advantages of their method include its ease of use, its small impact on the surgical workflow and its small-time requirement.

Augmented reality in neurosurgery: a systematic review.

Neuronavigation has become an essential neurosurgical tool in pursuing minimal invasiveness and maximal safety, even though it has several technical limitations. Augmented reality (AR) neuronavigation is a significant advance, providing a real-time updated 3D virtual model of anatomical details, overlaid on the real surgical field. Currently, only a few AR systems have been tested in a clinical setting. The aim is to review such devices. We performed a PubMed search of reports restricted to human studies of in vivo applications of AR in any neurosurgical procedure using the search terms "Augmented reality" and "Neurosurgery." Eligibility assessment was performed independently by two reviewers in an unblinded standardized manner. The systems were qualitatively evaluated on the basis of the following: neurosurgical subspecialty of application, pathology of treated lesions and lesion locations, real data source, virtual data source, tracking modality, registration technique, visualization processing, display type, and perception location. Eighteen studies were included during the period 1996 to September 30, 2015. The AR systems were grouped by the real data source: microscope (8), hand- or head-held cameras (4), direct patient view (2), endoscope (1), and X-ray fluoroscopy (1) head-mounted display (1). A total of 195 lesions were treated: 75 (38.46%) were neoplastic, 77 (39.48%) neurovascular, and 1 (0.51%) hydrocephalus, and 42 (21.53%) were undetermined. Current literature confirms that AR is a reliable and versatile tool when performing minimally invasive approaches in a wide range of neurosurgical diseases, although prospective randomized studies are not yet available and technical improvements are needed.

Preliminary Study on the Clinical Application of Augmented Reality Neuronavigation

To develop an augmented reality (AR) neuronavigation system with Web cameras and examine its clinical utility. The utility of the system was evaluated in three patients with brain tumors. One patient had a glioblastoma and two patients had convexity meningiomas. Our navigation system comprised the open-source software 3D Slicer (Brigham and Women's Hospital, Boston, Massachusetts, USA), the infrared optical tracking sensor Polaris (Northern Digital Inc., Waterloo, Canada), and Web cameras. We prepared two different types of Web cameras: a handheld type and a headband type. Optical markers were attached to each Web camera. We used this system for skin incision planning before the operation, during craniotomy, and after dural incision. We were able to overlay these images in all cases. In Case 1, accuracy could not be evaluated because the tumor was not on the surface, though it was generally suitable for the outline of the external ear and the skin. In Cases 2 and 3, the augmented reality error was ∼2 to 3 mm. AR technology was examined with Web cameras in neurosurgical operations. Our results suggest that this technology is clinically useful in neurosurgical procedures, particularly for brain tumors close to the brain surface.