A Method to Introduce & Evaluate Motion Parallax with Stereo for Medical AR/MR

Abstract

Incorrect depth perception and lack of good evaluation systems are major barriers in clinical translation of augmented and mixed reality AR/MR. Thus, a systematic study of depth cues is necessary. Therefore, in the current paper we present a method to introduce the quantitative depth cue Motion Parallax (MP) in surgical scenes and study its effect on depth perception when combined with binocular disparity. In addition to this we present an innovative virtual tool method to evaluate depth. To introduce MP, we reconstructed the tissue surface using structure from motion technique. Then to get accurate absolute scale of the reconstructed surface stereo-triangulation was used. The simulated tumor was rendered beneath the reconstructed point-cloud by rendering a hole for ‘X-ray’ like vision. The MP was introduced by rotating the entire scene from side-to-side with a tumor-surface-point as pivot for maximum impact. Finally for evaluation, we used a virtual surgical tool rendered using real-time da Vinci surgical API's forward kinematics data. In total, 12 subjects participated in a within-subjects-experiment design to study four cases, i.e., Stereo + MP ( $\text{S}+\text{MP}$ ), Mono + MP ( $\text{M}+\text{MP}$ ), Stereo + No MP $(\text{S}+\text{N}-\text{MP})$ and Mono + No MP $(\text{M}+\text{N}-\text{MP})$ . The subjects significantly overestimated Judged Percentage of True Distance in $\text{M}+\text{MP}$ when compared to $\text{M}+\text{N}-\text{MP}$ (probability $(\text{p})=0.000$ , Number of Samples $(\text{N}))=120)$ and $\text{S}+\text{N}-\text{MP}$ cases ( $\text{p}=0.001, \text{N}= 120$ ). Furthermore, the observed VariableError was less in $\text{S}+\text{MP}$ and $\text{S}+\text{N}-\text{MP}$ cases when compared to $\text{M}+\text{MP}$ and $\text{M}+\text{N}-\text{MP}$ cases. The use of Motion Parallax in console interfaces for surgical robotics showed overestimation of judged distance. But to our knowledge it is the first work studying the effect of motion parallax and stereo in the surgical context. Therefore, its further study is warranted.