Laser projection augmented reality system for computer-assisted surgery

Abstract

A general-purpose augmented reality apparatus was developed. The system uses rapidly scanned lasers to display information directly onto the patient using the patient as a “projection screen”. The device is also capable of measuring the location of projected laser spots with a 3D camera to ensure that the display is accurate, to apply corrections to the projection path and to assist in registration. The projection system has its own Application Programmer's Interface (API) and is a stand-alone add-on unit to any host computer system. An initial host application was developed to demonstrate the new unit. Fiducials representing vertices along a proposed craniotomy were embedded into a plastic skull and a projection path was calculated. A feedback-based optimization of the plan was performed by comparing the measurement taken by the camera of these coordinates. The optimized plan was projected onto the skull. On average, the projection deviated by approximately 1 mm from the plan. Possible applications include identification of critical anatomical structures, visualization of preplanned paths and targets, and telesurgery or teleconsultation applications.