Abstract
Depth mapping from binocular endoscopy images plays an important role in stereoscopic surgical treatment. Owing to the development of deep convolutional neural networks (CNNs), binocular depth estimation models have achieved many exciting results in the fields of autonomous driving and machine vision. However, the application of these methods to endoscopic imaging is greatly limited by the fact that binocular endoscopic images not only are rare, but also have unsatisfying features such as no texture, no ground truth, bad contrast, and high gloss. Aiming at solving the above-mentioned problems, we have built a precise gastrointestinal environment by the open-source software blender to simulate abundant binocular endoscopy data and proposed a 23-layer deep CNNs method to generate real-time stereo depth mapping. An efficient scale-invariant loss function is introduced in this paper to accommodate the characteristics of endoscope images, which improves the accuracy of achieved depth mapping results. Regarding the considerable training data for typical CNNs, our method requires only a few images ($960\times 720$ resolution) at 45 frames per second on an NVIDIA GTX 1080 GPU module, then the depth mapping information is generated in real-time with satisfactory accuracy. The effectiveness of the developed method is validated by comparing with state-of-the-art methods on processing the same datasets, demonstrating a faster and more accurate performance than other model frames.
Highlights
With the continuous growth of public demands for minimal invasion and accurate operation in surgery, the concept of surgical navigation system with ‘‘fine treatment’’ and ‘‘accurate surgery’’ has become the tendency of future intelligent surgery development [1]
Surgical navigation system is the combination of surgery, computer technology, image processing, and stereoscopic vision to obtain the exact positioning of relevant lesions and dynamic movement orientations of surgical instruments, which assists doctors with binocular images in real-time diagnosis and treatment [2]–[4]
(3) From the perspective of depth estimation in the endoscope environment, the proposed deep learning-based method is used for the first time to achieve an accurate real-time estimation of disparity
Summary
With the continuous growth of public demands for minimal invasion and accurate operation in surgery, the concept of surgical navigation system with ‘‘fine treatment’’ and ‘‘accurate surgery’’ has become the tendency of future intelligent surgery development [1]. Wang et al.: Deep Convolutional Network for Stereo Depth Mapping in Binocular Endoscopy in recent years These methods have considerable advantages in machine vision and autonomous driving, their application in stereo endoscopic images is still facing certain challenges. We have trained a deep learning neural network for obtaining dense and high-accuracy stereo disparity mapping in an endoscopic environment for real-time surgeries. (1) The open-source modeling software(blender) is used to create a 3D gastrointestinal environment simulation model, which can generate numerous binocular endoscopic images with accurate disparity information in a short time. (3) From the perspective of depth estimation in the endoscope environment, the proposed deep learning-based method is used for the first time to achieve an accurate real-time estimation of disparity. The training error of the network can reach as few as 0.41 pixels, and the processing time of a single image is 0.022 s
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
