Real-time Scene Reconstruction Research Articles

A Forest Point Cloud Real-Time Reconstruction Method with Single-Line Lidar Based on Visual–IMU Fusion

In order to accurately obtain tree growth information from a forest at low cost, this paper proposes a forest point cloud real-time reconstruction method with a single-line lidar based on visual–IMU fusion. We build a collection device based on a monocular camera, inertial measurement unit (IMU), and single-line lidar. Firstly, pose information is obtained using the nonlinear optimization real-time location method. Then, lidar data are projected to the world coordinates and interpolated to form a dense spatial point cloud. Finally, an incremental iterative point cloud loopback detection algorithm based on visual key frames is utilized to optimize the global point cloud and further improve precision. Experiments are conducted in a real forest. Compared with a reconstruction based on the Kalman filter, the root mean square error of the point cloud map decreases by 4.65%, and the time of each frame is 903 μs; therefore, the proposed method can realize real-time scene reconstruction in large-scale forests.

Toward Assistive Technologies for Focus Adjustment in Teleoperated Robotic Non-Contact Laser Surgery

In the last decade, focused laser radiation has become an alternative instrumentation for soft tissue surgery due to precise ablation with simultaneous coagulation. However, adjustment of the focal position with respect to the tissue surface is crucial for optimal energy input and minimised trauma. Advantageous laser delivery without tissue contact poses challenges and demands for assistance. This contribution introduces three novel concepts for user assistance during focal adaptation in noncontact laser surgery with robotic teleoperation. Visual, haptic, and visuo-haptic feedback modalities are designed based on real-time scene reconstruction and laser-to-camera registration. The performance of proposed methods was assessed in a phantom study with positioning tasks under different assistance conditions using a teleoperated extensible continuum robot. Focal position error and task execution time were selected as performance metrics. In total, 15 subjects conducted trials under four different conditions ( $N=240$ ). A mean error reduction from 1.2 mm for nonassisted to 0.25 mm for visuo-haptic feedback was revealed and error outliers were eliminated efficiently. Mean execution times of below 13 s were achieved for applied conditions. Presented work decreased the error to one-fifth in comparison to absence of assistance in teleoperation. The results strongly motivate prospective integration to surgical devices for noncontact laser delivery.

Real-time scene reconstruction and triangle mesh generation using multiple RGB-D cameras

We present a novel 3D reconstruction system that can generate a stable triangle mesh using data from multiple RGB-D sensors in real time for dynamic scenes. The first part of the system uses moving least squares (MLS) point set surfaces to smooth and filter point clouds acquired from RGB-D sensors. The second part of the system generates triangle meshes from point clouds. The whole pipeline is executed on the GPU and is tailored to scale linearly with the size of the input data. Our contributions include changes to the MLS method for improving meshing, a fast triangle mesh generation method and GPU implementations of all parts of the pipeline.