Multispectral image alignment using a three channel endoscope in vivo during minimally invasive surgery.

Neil T Clancy,Guang-Zhong Yang,Aimee Di Marco,Danail Stoyanov,James Clark,Daniel S Elson,Vincent Sauvage,David R C James

doi:10.1364/boe.3.002567

Abstract

Sequential multispectral imaging is an acquisition technique that involves collecting images of a target at different wavelengths, to compile a spectrum for each pixel. In surgical applications it suffers from low illumination levels and motion artefacts. A three-channel rigid endoscope system has been developed that allows simultaneous recording of stereoscopic and multispectral images. Salient features on the tissue surface may be tracked during the acquisition in the stereo cameras and, using multiple camera triangulation techniques, this information used to align the multispectral images automatically even though the tissue or camera is moving. This paper describes a detailed validation of the set-up in a controlled experiment before presenting the first in vivo use of the device in a porcine minimally invasive surgical procedure. Multispectral images of the large bowel were acquired and used to extract the relative concentration of haemoglobin in the tissue despite motion due to breathing during the acquisition. Using the stereoscopic information it was also possible to overlay the multispectral information on the reconstructed 3D surface. This experiment demonstrates the ability of this system for measuring blood perfusion changes in the tissue during surgery and its potential use as a platform for other sequential imaging modalities.

Highlights

Sequential imaging techniques are commonly used to detect changes in the spatial distribution of various molecules and biological materials
It was placed in front of the endoscope and images recorded as the position of the chart was changed and the LCTF scanned through the 420-700 nm wavelength range so that the final data set consisted of synchronised images from the multispectral and colour cameras of the object in different orientations
The red, green and blue filter response of the colour stereo cameras was calculated by acquiring images of the reflectance standard through the LCTF and dividing by the reference spectrum to correct for the CCD sensitivity

Summary

Introduction

Sequential imaging techniques are commonly used to detect changes in the spatial distribution of various molecules and biological materials Chromophores such as haemoglobin [1,2,3], melanin and water [4] may be detected using multispectral imaging, the closely related technique of narrowband imaging can allow cancer detection in flexible endoscopy [5,6,7], multispectral fluorescence imaging can indicate molecular targeting [8,9], and areas of differing cell density can be identified using polarisation imaging [10]. The changing spectral features of interest can be followed using a suitable mathematical model of light interaction with the tissue and chromophores of interest These techniques have potential surgical applications in real-time determination of tissue oxygenation, aiding the diagnosis of mesenteric ischaemia. They may prove useful in other applications of multispectral imaging, such as detection of Hirschsprung’s disease [11], transplanted organ viability [12,13], or visualisation of the bile duct [14]

Methods

Results

Conclusion