Multi-resolution Foveated Laparoscope Research Articles

A dual-view multi-resolution laparoscope for safer and more efficient minimally invasive surgery

Minimally invasive surgery (MIS) is limited in safety and efficiency by the hand-held nature and narrow fields of view of traditional laparoscopes. A multi-resolution foveated laparoscope (MRFL) was invented to address these concerns. The MRFL is a stationary dual-view imaging device with optical panning and zooming capabilities. It is designed to simultaneously capture and display a zoomed view and supplemental wide view of the surgical field. Optical zooming and panning capabilities facilitate repositioning of the zoomed view without physically moving the system. Additional MRFL features designed to improve safety and efficiency include its snub-nosed endoscope, tool-tip auto tracking, programmable focus profiles, unique selectable display modalities, foot pedal controls, and independently controlled surgeon and assistant displays. An MRFL prototype was constructed to demonstrate and test these features. Testing of the prototype validates its design architecture and confirms the functionality of its features. The current MRFL prototype functions adequately as a proof of concept, but the system features and performance require further improvement to be practical for clinical use.

High-throughput multi-resolution foveated laparoscope for minimally invasive surgery.

Feasibility and clinical utility of a multi-resolution foveated laparoscope (MRFL) was previously tested in a porcine surgical study. The study revealed several clinical limitations of the system including moisture proofing, working distance, image quality, low light performance, color accuracy, size, and weight. In this paper, we discuss the root causes of these limitations and strategies to correct them, present the design and prototyping of a new high throughput multi resolution foveated laparoscope (HT-MRFL), and demonstrate the HT-MRFL prototype performance in comparison to the MRFL and simulated performance metrics.

Surgeon Assessment of a Novel Multi-Resolution Foveated Laparoscope.

We developed a multi-resolution foveated laparoscope (MRFL) to improve situational awareness in laparoscopic surgery. We assessed surgeon objective task performance and subjective attitudes with MRFL when used for box trainer tasks and porcine surgery. The MRFL simultaneously obtains a wide-angle view and a magnified view. The 2 images are displayed simultaneously. 6 urologists and 2 general surgeons performed box trainer and porcine surgery tasks with the MRFL and a standard laparoscope. Task time, use of display options, and subjective assessments were obtained. Subjectively, surgeons rated situational awareness, depth perception, and instrument interference as comparable between the prototype MRFL and laparoscope for trainer tasks. For porcine surgery, the MRFL was rated as having less interference than the standard laparoscope. The image quality of the MRFL was rated interior to the standard laparoscope. Participants found the different viewing modes useful for different roles and reported that they would likely use the MRFL for conventional laparoscopic and robotic surgery. Objectively, box trainer task time was comparable for 2 of 3 tasks with the remaining task shorter for the standard laparoscope. Porcine nephrectomy and oophorectomy were feasible with the prototype MRFL, although nephrectomy task time was significantly longer than traditional laparoscopy. The MRFL demonstrated feasibility for performing complex surgery. Surgeons had favorable attitudes toward its features and likelihood to use the device if available. Users utilized different view types for different tasks. Longer MRFL task times were attributed to poorer image quality of the prototype.

MP10-04 SURGEON ASSESSMENT OF A NOVEL MULTI-RESOLUTION FOVEATED LAPAROSCOPE

You have accessJournal of UrologySurgical Technology & Simulation: Instrumentation & Technology I (MP10)1 Sep 2021MP10-04 SURGEON ASSESSMENT OF A NOVEL MULTI-RESOLUTION FOVEATED LAPAROSCOPE Mike Nguyen, Jamal Nhabani, Allan Hamilton, Sangyoon Lee, Jeremy Katz, and Hong Hua Mike NguyenMike Nguyen More articles by this author , Jamal NhabaniJamal Nhabani More articles by this author , Allan HamiltonAllan Hamilton More articles by this author , Sangyoon LeeSangyoon Lee More articles by this author , Jeremy KatzJeremy Katz More articles by this author , and Hong HuaHong Hua More articles by this author View All Author Informationhttps://doi.org/10.1097/JU.0000000000001983.04AboutPDF ToolsAdd to favoritesDownload CitationsTrack CitationsPermissionsReprints ShareFacebookLinked InTwitterEmail Abstract INTRODUCTION AND OBJECTIVE: To improve safety in minimally invasive surgery, we developed a multi-resolution foveated laparoscope (MRFL) which can simultaneously obtain and display wide and zoomed in views in order to avoid surgical tunnel vision and improve situational awareness. We assessed surgeon objective task performance and subjective attitudes towards the MRFL for ex vivo box trainer tasks and in vivo porcine surgery. METHODS: The MRFL utilizes two sensors to simultaneously obtain a wide-angle view and obtain and pan a high-resolution magnified view. Options for displaying the two images including side by side view, full screen views, and 3 variations of overlaid images. A pedal changes views and controls retargeting of the zoomed view. 6 urologists and 2 general surgeons performed box trainer and porcine surgery tasks with both a standard laparoscope and the MRFL. Task time, use of display options, and subjective assessments were obtained. RESULTS: For box trainer tasks, task time was shorter for standard laparoscopy than MRFL for internal peg transfer tasks (28.5 vs 56.6 secs, p=0.0055) in one of two sessions but was not different for external peg transfer tasks (69.2 vs 77.2 secs, p=0.52 and 124.2 vs 127.5 secs, p=0.93). For porcine nephrectomy, laparoscopy surgical time was significantly shorter than MRFL (17.5 vs 33.3 min, p=0.049). For the simpler oophorectomy surgical task, time did not differ significantly. As a surgeon, participants were most likely to use side by side view (46.4%) followed by full screen overview or zoomed view (26.6%). Assistants predominately used the side-by-side view (80.1%). Subjectively, surgeons rated situational awareness, depth perception, and interference with other instruments as similar between the systems for the trainer session while the MRFL was rated as having less interference for the animal lab session (2 vs 6.3, p=0.0093). MRFL was rated as having lower image quality than laparoscopy in both the trainer (6.4 vs 8.4, p=0.012) and animal lab sessions (3.5 vs 9.2, p=0.0003). Participants rated having different MRFL views as being useful for different roles (8.1 out of a scale of 10) and being likely to use the MRFL for laparoscopic surgery (8.1) and robotic surgery (8.3). CONCLUSIONS: Ex vivo and in vivo surgeon assessments of the MRFL demonstrated its feasibility for performing complex surgery. Surgeons had favorable attitudes towards its functional features and expressed likelihood to use the device if it were available. Users utilized different view types based on different task needs. Poorer image quality of the prototype MRFL was associated with longer task times. Source of Funding: NIH RO1 EB18921 © 2021 by American Urological Association Education and Research, Inc.FiguresReferencesRelatedDetails Volume 206Issue Supplement 3September 2021Page: e169-e170 Advertisement Copyright & Permissions© 2021 by American Urological Association Education and Research, Inc.MetricsAuthor Information Mike Nguyen More articles by this author Jamal Nhabani More articles by this author Allan Hamilton More articles by this author Sangyoon Lee More articles by this author Jeremy Katz More articles by this author Hong Hua More articles by this author Expand All Advertisement Loading ...

Further Comparison of 4 Display Modes for a Multi-Resolution Foveated Laparoscope.

Background. To overcome field of view and ergonomic limitations of standard laparoscopes, we are developing a multi-resolution foveated laparoscope (MRFL), which can simultaneously obtain both wide- and zoomed-in-view images. To facilitate the effectiveness of our MRFL, we have been investigating various ways of organizing and visualizing dual-view multi-resolution images acquired by the MRFL. In our prior study, we implemented and compared 6 display modes for the MRFL, assuming a typical clinical environment where a standard (but limited) resolution monitor is available. To take full advantage of our MRFL, displays having sufficient screen resolutions might be advantageous. The present study aims to further understand the effects of view configurations through displays with a standard high-definition (HD) resolution and a 4K resolution. In this study, we compare 3 display modes for limited-resolution displays against a new mode for sufficient-resolution displays. Methods. Twenty subjects performed 3 evaluation trials of a touching task with each display mode in an emulated MRFL environment. Various objective measurements including task completion time and the number of collisions, and subjective preference were recorded. Results. The new mode showed a better task completion time than the other modes, while it maintained a low number of collisions similar to the others. Moreover, the majority of participants selected the new mode as their most preferred one. Conclusions. With a sufficient display resolution, the co-registration between the unblocked and unwarped wide context view and the high-resolution zoomed-in view offered by the new mode was highly effective on both task performance and user preference.

Improved multi-resolution foveated laparoscope with real-time digital transverse chromatic correction.

A multi-resolution foveated laparoscope (MRFL) with autofocus and zooming capabilities was previously designed to address the limiting trade-off between spatial resolution and field of view during laparoscopic minimally invasive surgery. The MRFL splits incoming light into two paths enabling simultaneous capture of the full surgical field and a zoomed-in view of the local surgical site. A fully functional prototype was constructed to demonstrate and test the autofocus, zooming capabilities, and clinical utility of this new laparoscope. The test of the prototype in both dry lab and animal models was successful, but it also revealed several major limitations of the prototype. In this paper, we present a brief overview of the aforementioned MRFL prototype design and results, and the shortcomings associated with its optical and mechanical designs. We then present several methods to address the shortcomings of the existing prototype with a modified optical layout and redesigned mechanics. The performances of the new and old system prototypes are comparatively analyzed in accordance with the design goals of the new MRFL. Finally, we present and demonstrate a real-time digital method for correcting transverse chromatic aberration to further improve the overall image quality, which can be adapted to future MRFL systems.

Comparison of six display modes for a multi-resolution foveated laparoscope.

To overcome the field of view and ergonomic limitations of standard laparoscopes, we are developing a multi-resolution foveated laparoscope that can simultaneously obtain both wide- and zoomed-in-view images through a single scope. To facilitate the effective access to the dual views of images with different resolution and field coverage acquired by our laparoscope, six different display modes have been developed. Each of the six display modes has inherent advantages and disadvantages. This study compares the six display modes through a human-subject experiment, which was conducted with an emulated laparoscope using a 4K camera. Twenty-four subjects without medicine background performed three evaluation trials of a touching task with each of the display modes. Various objective measurements including task completion time, the number of collisions, gaze position, and tooltip position, and subjective preference for the display modes were recorded. For all the measurements except for task completion time and moving speed of tooltip, there were statistically significant differences among the display modes. Although the focus plus warped context view mode was selected as one of the least preferred modes, it showed the best task performance. The unblocked wide context view was useful to provide a situational awareness even when it was severely distorted in some of the display modes, and information continuity played an important role in improving task performance. Moreover, the position change of viewing window coupled to the location of region of interest helped improve task performance, by providing an additional cue for spatial awareness.

Optical design and system engineering of a multiresolution foveated laparoscope.

The trade-off between the spatial resolution and field of view is one major limitation of state-of-the-art laparoscopes. In order to address this limitation, we demonstrated a multiresolution foveated laparoscope (MRFL) which is capable of simultaneously capturing both a wide-angle overview for situational awareness and a high-resolution zoomed-in view for accurate surgical operation. In this paper, we focus on presenting the optical design and system engineering process for developing the MRFL prototype. More specifically, the first-order specifications and properties of the optical system are discussed, followed by a detailed discussion on the optical design strategy and procedures of each subsystem. The optical performance of the final system, including diffraction efficiency, tolerance analysis, stray light and ghost image, is fully analyzed. Finally, the prototype assembly process and the final prototype are demonstrated.

Continuously zoom imaging probe for the multi-resolution foveated laparoscope.

In modern minimally invasive surgeries (MIS), standard laparoscopes suffer from the tradeoff between the spatial resolution and field of view (FOV). The inability of simultaneously acquiring high-resolution images for accurate operation and wide-angle overviews for situational awareness limits the efficiency and outcome of the MIS. A dual view multi-resolution foveated laparoscope (MRFL) which can simultaneously provide the surgeon with a high-resolution view as well as a wide-angle overview was proposed and demonstrated to have great potential for improving the MIS. Although experiment results demonstrated the high-magnification probe has an adequate magnification for viewing surgical details, the dual-view MRFL is limited to two fixed levels of magnifications. A fine adjustment of the magnification is highly desired for obtaining high resolution images with desired field coverage. In this paper, a high magnification probe with continuous zooming capability without any mechanical moving parts is demonstrated. By taking the advantages of two electrically tunable lenses, one for optical zoom and the other for image focus compensation, the optical magnification of the high-magnification probe varies from 2 × to 3 × compared with that of the wide-angle probe, while the focused object position stays the same as the wide-angle probe. The optical design and the tunable lens analysis are presented, followed by prototype demonstration.

Characterization and in-vivo evaluation of a multi-resolution foveated laparoscope for minimally invasive surgery

The state-of-the-art laparoscope lacks the ability to capture high-magnification and wide-angle images simultaneously, which introduces challenges when both close- up views for details and wide-angle overviews for orientation are required in clinical practice. A multi-resolution foveated laparoscope (MRFL) which can provide the surgeon both high-magnification close-up and wide-angle images was proposed to address the limitations of the state-of-art surgical laparoscopes. In this paper, we present the overall system design from both clinical and optical system perspectives along with a set of experiments to characterize the optical performances of our prototype system and describe our preliminary in-vivo evaluation of the prototype with a pig model. The experimental results demonstrate that at the optimum working distance of 120mm, the high-magnification probe has a resolution of 6.35lp/mm and image a surgical area of 53 × 40mm2; the wide-angle probe provides a surgical area coverage of 160 × 120mm2 with a resolution of 2.83lp/mm. The in-vivo evaluation demonstrates that MRFL has great potential in clinical applications for improving the safety and efficiency of the laparoscopic surgery.

Multiresolution foveated laparoscope with high resolvability.

A key limitation of the state-of-the-art laparoscopes for minimally invasive surgery is the tradeoff between the field of view and spatial resolution in a single-view camera system. As such, surgical procedures are usually performed at a zoomed-in view, which limits the surgeon's ability to see much outside the immediate focus of interest and causes a situational awareness challenge. We proposed a multiresolution foveated laparoscope (MRFL) aiming to address this limitation. The MRFL is able to simultaneously capture wide-angle overview and high-resolution images in real time; it can scan and engage the high-resolution images to any subregion of the entire surgical field in analogy to the fovea of human eye. The MRFL is able to render equivalently 10 million pixel resolution with a low data bandwidth requirement. The system has a large working distance (WD) from 80 to 180mm. The spatial resolvability is about 45μm in the object space at an 80mm WD, while the resolvability of a conventional laparoscope is about 250μm at a typically 50mm surgical distance.

Multi-resolution foveated laparoscope for safer minimally invasive surgery

Multi-resolution foveated laparoscope for safer minimally invasive surgery