Real three-dimensional medical interactive display system based on integral imaging and gesture recognition

Abstract

In this paper, we developed a real three-dimensional medical interactive display system based on integral imaging and gesture recognition. The system includes a fast image processing module, a high-performance three-dimensional display module, and a gesture recognition module. The three-dimensional model is obtained through gray-scale interval overlay processing. The non-matching element image method and 4 K screen are used for three-dimensional integral imaging display with a large viewing range and high resolution. A pair of binocular cameras is used to capture and recognize gestures, and the maximum interaction frame rate is 22 FPS. Optical experiments were performed to validate the proposed medical interactive three-dimensional display system with existing patient skeletal data. The results showed that the proposed system can effectively solve the problem that surgeons cannot take off their gloves to operate in surgery and provide an intuitive and comfortable three-dimensional display. Our proposed system is expected to be widely used in scenarios such as preoperative simulation, surgical instruction, and medical teaching.