Narrow Field Of View Research Articles

Patient‐Mounted Neuro Optical Coherence Tomography for Targeted Minimally Invasive Micro‐Resolution Volumetric Imaging in Brain In Vivo

Targeted neuroimaging plays a vital role in evaluating pathologies and guiding interventions in deep brain regions, such as biopsy, laser ablation, and deep brain stimulation. However, current neuroimaging techniques face several challenges when it comes to imaging the deep brain. These challenges include limited imaging depth, a narrow field of view, low resolution, and a lack of real‐time imaging and stereotactic deployment capabilities. To address these challenges, a patient‐mounted neuro optical coherence tomography (neuroOCT) system that combines a lightweight 5 degrees‐of‐freedom skull‐mounted robot (Skullbot) with a neuroendoscope measuring ≈0.6 mm in diameter is introduced. This innovative system enables targeted and minimally invasive neuroimaging with an axial resolution of ≈2.4 μm and a transverse resolution of around 4.5 μm. The Skullbot can be securely attached to the head and precisely deploys the neuroendoscope with an accuracy of ±1.5 mm in the transverse direction and ±0.25 mm in the longitudinal direction. This allows for motion‐insensitive stereotactic imaging within the brain. By utilizing the neuroOCT system, targeted imaging of a tumor in a brain phantom is successfully demonstrated. Furthermore, the system's capability for in vivo micro‐resolution volumetric neuroimaging of fine structures within a mouse brain is validated.

A calcification subtraction method for postmortem coronary computed tomography angiography.

Although coronary computed tomography (CT) angiography is a useful tool for evaluating coronary artery lesions both ante- and postmortem, accurate evaluation of the lumen is difficult when highly calcified lesions are present, owing to overestimation of stenosis caused by blooming and partial volume artifacts. In clinical practice, to overcome this diagnostic problem, a subtraction method has been devised to remove calcification by subtracting the precontrast image from the contrast image. In this report, we describe a calcification subtraction method using image analysis software for postmortem coronary CT angiography. This method was devised based on preliminary experimental results showing that the most accurate subtraction was achieved using images reconstructed with a narrower field of view and bone kernel, resulting in higher spatial resolution. This subtraction method allowed evaluation of lumen patency and the degree of stenosis on contrast-enhanced images in a verification using actual specimens where evaluation of the lumen had been difficult because of high calcification. The results were morphologically similar to the macroscopic findings. This method allows more rapid and reliable lesion retrieval and is expected to be useful for postmortem coronary angiography in forensic practice.

Real-time placental vessel segmentation in fetoscopic laser surgery for Twin-to-Twin Transfusion Syndrome

Twin-to-Twin Transfusion Syndrome (TTTS) is a rare condition that affects about 15% of monochorionic pregnancies, in which identical twins share a single placenta. Fetoscopic laser photocoagulation (FLP) is the standard treatment for TTTS, which significantly improves the survival of fetuses. The aim of FLP is to identify abnormal connections between blood vessels and to laser ablate them in order to equalize blood supply to both fetuses. However, performing fetoscopic surgery is challenging due to limited visibility, a narrow field of view, and significant variability among patients and domains. In order to enhance the visualization of placental vessels during surgery, we propose TTTSNet, a network architecture designed for real-time and accurate placental vessel segmentation. Our network architecture incorporates a novel channel attention module and multi-scale feature fusion module to precisely segment tiny placental vessels. To address the challenges posed by FLP-specific fiberscope and amniotic sac-based artifacts, we employed novel data augmentation techniques. These techniques simulate various artifacts, including laser pointer, amniotic sac particles, and structural and optical fiber artifacts. By incorporating these simulated artifacts during training, our network architecture demonstrated robust generalizability. We trained TTTSNet on a publicly available dataset of 2060 video frames from 18 independent fetoscopic procedures and evaluated it on a multi-center external dataset of 24 in-vivo procedures with a total of 2348 video frames. Our method achieved significant performance improvements compared to state-of-the-art methods, with a mean Intersection over Union of 78.26% for all placental vessels and 73.35% for a subset of tiny placental vessels. Moreover, our method achieved 172 and 152 frames per second on an A100 GPU, and Clara AGX, respectively. This potentially opens the door to real-time application during surgical procedures. The code is publicly available at https://github.com/SanoScience/TTTSNet.

Robot-based dynamic optics scanning infrared thermography for debonding defect detection of big-sized CFRP specimen

ABSTRACT The use of infrared thermography (IRT) for non-destructive testing (NDT) of large aerospace composites, such as carbon fibre reinforced polymer (CFRP), is often limited by the narrow field of view infrared cameras. This paper experimentally investigates the big-sized CFRP specimens with simulated debonding defects using a robot-based dynamic optics scanning infrared thermography (RDOS-IRT) method to meet these issues. Then, static pulse IRT and RDOS-IRT were performed, and the processing performance of different algorithms (pulsed phase thermography, PPT; total harmonic distortion, THD; matched filtering, MF; and supervised dimensionality reduction, SDR) on pulsed IRT was discussed. A combination of pseudo-static matrix reconstruction (FSMR) algorithm and static image sequence processing algorithm is innovatively adopted to improve the defect detection performance of RDOS-IRT results. The experiments show that the RDOS-IRT method with FSMR and static image sequence processing algorithm is effective; moreover, it has almost a similar detection ability compared to the traditional static pulse IRT method in practical applications and saves more time. Results show that the signal-to-noise ratio (SNR) of the repeated defects is not differentiated, and the detection effect is similar. Therefore, the RDOS-IRT method is a reliable tool for NDT tasks on big-sized CFRP specimens and can be practically applied in an industrial setting for high detection efficiency.

Off-axis reflective microscope objective with a centimeter scale field of view and micron resolution

Microscope objectives with wide field-of-view (FOV) and high resolution are urgently needed for the frontier research in life sciences. However, traditional transmission microscope objectives typically have a narrow FOV and severe chromatic aberration. A new off-axis reflective microscope objective with a centimeter scale FOV and micron resolution is proposed in this paper. This objective, with its simple structure, can operate over a wide wavelength range. A design method for a wide FOV optical system is presented, which can eliminate the obstruction of the rays and control the intermediate image plane. Using this method, we design a novel off-axis four-mirror microscope objective with a FOV of 10 mm × 1.5 mm and a numerical aperture of 0.33.

High-Magnification Object Tracking with Ultra-Fast View Adjustment and Continuous Autofocus Based on Dynamic-Range Focal Sweep.

Active vision systems (AVSs) have been widely used to obtain high-resolution images of objects of interest. However, tracking small objects in high-magnification scenes is challenging due to shallow depth of field (DoF) and narrow field of view (FoV). To address this, we introduce a novel high-speed AVS with a continuous autofocus (C-AF) approach based on dynamic-range focal sweep and a high-frame-rate (HFR) frame-by-frame tracking pipeline. Our AVS leverages an ultra-fast pan-tilt mechanism based on a Galvano mirror, enabling high-frequency view direction adjustment. Specifically, the proposed C-AF approach uses a 500 fps high-speed camera and a focus-tunable liquid lens operating at a sine wave, providing a 50 Hz focal sweep around the object's optimal focus. During each focal sweep, 10 images with varying focuses are captured, and the one with the highest focus value is selected, resulting in a stable output of well-focused images at 50 fps. Simultaneously, the object's depth is measured using the depth-from-focus (DFF) technique, allowing dynamic adjustment of the focal sweep range. Importantly, because the remaining images are only slightly less focused, all 500 fps images can be utilized for object tracking. The proposed tracking pipeline combines deep-learning-based object detection, K-means color clustering, and HFR tracking based on color filtering, achieving 500 fps frame-by-frame tracking. Experimental results demonstrate the effectiveness of the proposed C-AF approach and the advanced capabilities of the high-speed AVS for magnified object tracking.

Simplified channel model for indoor VLC systems

The channel model of indoor visible light communication (VLC) is primarily influenced by the receiver’s position and orientation. In this study, we decouple the effects of receiver position and orientation, treating them as independent random variables. This decoupling enables a clearer understanding of how each factor independently influences the channel model. Specifically, we focus on the impact of the azimuth angle, a critical component of orientation, on the VLC channel. We present a simplified channel model for indoor VLC systems based on partial orientation information. Theoretical analysis and simulation results collectively indicate that the accuracy of the simplified channel model is influenced by the field-of-view (FOV), with higher accuracy achieved at narrower FOVs. To evaluate practical performance, we compare our simplified model with the existing complete channel model in a two-user precoding system. Results show that our proposed simplified channel model maintains value, particularly in narrow-FOV scenarios. The research findings indicate that this simplified model is suitable for applications with high demands for channel accuracy in narrow-FOV indoor VLC systems, and can be extended to applications in wide-FOV systems where perfect channel information is not required.

Autoregressive Omni-Aware Outpainting for Open-Vocabulary 360-Degree Image Generation

A 360-degree (omni-directional) image provides an all-encompassing spherical view of a scene. Recently, there has been an increasing interest in synthesising 360-degree images from conventional narrow field of view (NFoV) images captured by digital cameras and smartphones, for providing immersive experiences in various scenarios such as virtual reality. Yet, existing methods typically fall short in synthesizing intricate visual details or ensure the generated images align consistently with user-provided prompts. In this study, autoregressive omni-aware generative network (AOG-Net) is proposed for 360-degree image generation by outpainting an incomplete 360-degree image progressively with NFoV and text guidances joinly or individually. This autoregressive scheme not only allows for deriving finer-grained and text-consistent patterns by dynamically generating and adjusting the process but also offers users greater flexibility to edit their conditions throughout the generation process. A global-local conditioning mechanism is devised to comprehensively formulate the outpainting guidance in each autoregressive step. Text guidances, omni-visual cues, NFoV inputs and omni-geometry are encoded and further formulated with cross-attention based transformers into a global stream and a local stream into a conditioned generative backbone model. As AOG-Net is compatible to leverage large-scale models for the conditional encoder and the generative prior, it enables the generation to use extensive open-vocabulary text guidances. Comprehensive experiments on two commonly used 360-degree image datasets for both indoor and outdoor settings demonstrate the state-of-the-art performance of our proposed method. Our code is available at https://github.com/zhuqiangLu/AOG-NET-360.

Feasibility of proton dosimetry overriding planning CT with daily CBCT elaborated through generative artificial intelligence tools

Radiotherapy commonly utilizes cone beam computed tomography (CBCT) for patient positioning and treatment monitoring. CBCT is deemed to be secure for patients, making it suitable for the delivery of fractional doses. However, limitations such as a narrow field of view, beam hardening, scattered radiation artifacts, and variability in pixel intensity hinder the direct use of raw CBCT for dose recalculation during treatment. To address this issue, reliable correction techniques are necessary to remove artifacts and remap pixel intensity into Hounsfield Units (HU) values. This study proposes a deep-learning framework for calibrating CBCT images acquired with narrow field of view (FOV) systems and demonstrates its potential use in proton treatment planning updates. Cycle-consistent generative adversarial networks (cGAN) processes raw CBCT to reduce scatter and remap HU. Monte Carlo simulation is used to generate CBCT scans, enabling the possibility to focus solely on the algorithm’s ability to reduce artifacts and cupping effects without considering intra-patient longitudinal variability and producing a fair comparison between planning CT (pCT) and calibrated CBCT dosimetry. To showcase the viability of the approach using real-world data, experiments were also conducted using real CBCT. Tests were performed on a publicly available dataset of 40 patients who received ablative radiation therapy for pancreatic cancer. The simulated CBCT calibration led to a difference in proton dosimetry of less than 2%, compared to the planning CT. The potential toxicity effect on the organs at risk decreased from about 50% (uncalibrated) up the 2% (calibrated). The gamma pass rate at 3%/2 mm produced an improvement of about 37% in replicating the prescribed dose before and after calibration (53.78% vs 90.26%). Real data also confirmed this with slightly inferior performances for the same criteria (65.36% vs 87.20%). These results may confirm that generative artificial intelligence brings the use of narrow FOV CBCT scans incrementally closer to clinical translation in proton therapy planning updates.

Thermal Characterization of Subcooled Flow Boiling in a Pin-Fin Coldplate With Non-Uniform Heating

Abstract Coldplates are a crucial component in various cooling applications, such as cooling data center servers and power electronics. The unprecedented growth in electronics power density, along with the resulting ultrahigh heat fluxes, demands a transition from single-phase forced convection to two-phase flow boiling heat transfer. The majority of studies in the literature have focused on flow boiling in fin-enhanced silicon microgaps and microchannels, with only a few addressing flow boiling in millimeter-scale heat sinks. In the present study, flow boiling of HFE-7200 dielectric fluid in a millimeter-scale pin-fin coldplate is experimentally investigated under nonuniform heating conditions. Four background heaters represent the low-dissipating-power devices. On the other hand, five hotspot heaters mimic the high-heat-flux devices and generate heat fluxes ranging from 50 W/cm2 to 1000 W/cm2, corresponding to hotspot heat inputs ranging from 62.5 W to 1.25 kW, respectively. The coldplate's thermohydraulic performance is investigated for various flow rates and inlet temperature ranging from 0.5 L/min to 1.5 L/min and from 25 °C to 60 °C, respectively. A high-speed camera is utilized for a narrow field of view (FOV) flow visualization at a frame rate of 2229 fps while a digital camera is used for a wider FOV at 60 fps. Flow visualization demonstrated the transition between bubbly, slug/churn, and stratified two-phase flow regimes.

3D Ultrasound Mosaic of the Whole Shoulder: A Feasibility Study

A protocol is proposed to acquire a tomographic ultrasound (US) scan of the musculoskeletal (MSK) anatomy in the rotator cuff region. Current clinical US imaging techniques are hindered by occlusions and a narrow field of view and require expert acquisition and interpretation. There is limited literature on 3D US image registration of the shoulder or volumetric reconstruction of the full shoulder complex. We believe that a clinically accurate US volume reconstruction of the entire shoulder can aid in pre-operative surgical planning and reduce the complexity of US interpretation. The protocol was used in generating data for deep learning model training to automatically register US mosaics in real-time. An in vivo 3D US tomographic reconstruction of the entire rotator cuff region was produced by registering 53 sequential 3D US volumes acquired by an MSK sonographer. Anatomical surface thicknesses and distances in the US mosaic were compared to their corresponding MRI measurements as the ground truth. The humeral head surface was marginally thicker in the reconstructed US mosaic than its original thickness observed in a single US volume by 0.65 mm. The humeral head diameter and acromiohumeral distance (ACHD) matched with their measured MRI distances with a reconstruction error of 0 mm and 1.2 mm, respectively. Furthermore, the demonstration of 20 relevant MSK structures was independently graded between 1 and 5 by two sonographers, with higher grades indicating poorer demonstration. The average demonstration grade for each anatomy was as follows: bones = 2, muscles = 3, tendons = 3, ligaments = 4–5 and labrum = 4–5. There was a substantial agreement between sonographers (Cohen’s Weighted kappa of 0.71) on the demonstration of the structures, and they both independently deemed the mosaic clinically acceptable for the visualisation of the bony anatomy. Ligaments and the labrum were poorly observed due to anatomy size, location and inaccessibility in a static scan, and artefact build-up from the registration and compounding approaches.

Absolute radiance calibration in the UV and visible spectral range using atmospheric observations during twilight

Abstract. We present an improved radiance calibration method for UV–Vis spectroscopic instruments with a narrow field of view (up to a few degrees) based on the calibration method by Wagner et al. (2015). The updated method uses only measurements during the twilight period instead of several hours as for the original method. The calibration is based on the comparison of measurements and simulations of the radiance of zenith-scattered sunlight. The main advantage of our method compared to radiance calibration methods in the laboratory is that the calibration can be directly applied in the field. This allows routine radiance calibrations whenever the sky is clear during twilight. The calibration can also be performed retrospectively and will thus be applicable for the large number of existing data sets. Also, potential changes in the instrument properties during transport from the laboratory to the field are avoided. The new version of the calibration method presented here has two main advantages. First, the required measurement period can be rather short (only a few minutes during twilight for cloud-free conditions). Second, even without knowledge of the aerosol optical depth (AOD), the errors in the calibration method are rather small, especially in the UV spectral range where they range from about 4 % at 340 nm to 8 % at 420 nm. If the AOD is known, the uncertainties are even smaller (about 3 % at 340 nm to 4 % at 420 nm). For visible wavelengths, good accuracy is only obtained if the AOD is approximately known with uncertainties from about 4 % at 420 nm to 10 % between about 550 and 700 nm (generally the AOD is nevertheless smaller in the visible than in the UV spectral range). One shortcoming of the method is that it is not possible to determine the AOD exactly at the time of the (twilight) measurements because AOD observations from sun photometer measurements or the MAX-DOAS (Multi-AXis Differential Optical Absorption Spectroscopy) measurements are usually not meaningful for such high solar zenith angle (SZA). But the related uncertainty can be minimised by repeating the radiance calibrations during the twilight periods of several days.

Dream360: Diverse and Immersive Outdoor Virtual Scene Creation via Transformer-Based 360° Image Outpainting.

360° images, with a field-of-view (FoV) of $180^{\circ}\times 360^{\circ}$, provide immersive and realistic environments for emerging virtual reality (VR) applications, such as virtual tourism, where users desire to create diverse panoramic scenes from a narrow FoV photo they take from a viewpoint via portable devices. It thus brings us to a technical challenge: 'How to allow the users to freely create diverse and immersive virtual scenes from a narrow FoV image with a specified viewport?' To this end, we propose a transformer-based 360° image outpainting framework called Dream360, which can generate diverse, high-fidelity, and high-resolution panoramas from user-selected viewports, considering the spherical properties of 360° images. Compared with existing methods, e.g., [3], which primarily focus on inputs with rectangular masks and central locations while overlooking the spherical property of 360° images, our Dream360 offers higher outpainting flexibility and fidelity based on the spherical representation. Dream360 comprises two key learning stages: (I) codebook-based panorama outpainting via Spherical-VQGAN (S-VQGAN), and (II) frequency-aware refinement with a novel frequency-aware consistency loss. Specifically, S-VQGAN learns a sphere-specific codebook from spherical harmonic (SH) values, providing a better representation of spherical data distribution for scene modeling. The frequency-aware refinement matches the resolution and further improves the semantic consistency and visual fidelity of the generated results. Our Dream360 achieves significantly lower Frechet Inception Distance (FID) scores and better visual fidelity than existing methods. We also conducted a user study involving 15 participants to interactively evaluate the quality of the generated results in VR, demonstrating the flexibility and superiority of our Dream360 framework.

Optimal Multi-Sensor Obstacle Detection System for Small Fixed-Wing UAVs

The safety enhancement of small fixed-wing UAVs regarding obstacle detection is addressed using optimization techniques to find the best sensor orientations of different multi-sensor configurations. Four types of sensors for obstacle detection are modeled, namely an ultrasonic sensor, laser rangefinder, LIDAR, and RADAR, using specifications from commercially available models. The simulation environment developed includes collision avoidance with the Potential Fields method. An optimization study is conducted using a genetic algorithm that identifies the best sensor sets and respective orientations relative to the UAV longitudinal axis for the highest obstacle avoidance success rate. The UAV performance is found to be critical for the solutions found, and its speed is considered in the range of 5–15 m/s with a turning rate limited to 45°/s. Forty collision scenarios with both stationary and moving obstacles are randomly generated. Among the combinations of the sensors studied, 12 sensor sets are presented. The ultrasonic sensors prove to be inadequate due to their very limited range, while the laser rangefinders benefit from extended range but have a narrow field of view. In contrast, LIDAR and RADAR emerge as promising options with significant ranges and wide field of views. The best configurations involve a front-facing LIDAR complemented with two laser rangefinders oriented at ±10° or two RADARs oriented at ±28°.

CNN-based automated approach to crack-feature detection in steam cycle components

Periodic manual inspection by trained specialists is an important element of asset management in the nuclear industry. Detection of cracks caused by stress corrosion is an important element of remote visual inspection (RVI) in power plant steam generator components such as boilers, superheaters and reheaters. Challenges exist in the interpretation of RVI footage, such as high degree of concentration for reviewing lengthy and disorienting footage due to narrow field of view offered by endoscope. Deep learning is considered useful to automate crack detection process for improved efficiency and accuracy, and has enjoyed success in related applications. This article utilises a new application of automated crack feature detection in steam cycle components to demonstrate a transferrable data-driven framework for a variety of anomaly inspections in such structures. Specifically, a case study of superheater (a type of reactor pressure vessel head) anomaly inspection is presented to automatically detect regions of crack-like features in inspection footage with a good accuracy of 92.97 % using convolutional neural network (CNN), even in challenging cases. Due to the black-box nature of the CNN classification, the explicability of the classification results is discussed to enhance the trustworthiness of the detection system.

Day and night continuous high-resolution shallow-water depth detection with single-photon underwater lidar.

Single-photon lidar has emerged as a strong technology for bathymetric measurements. However, its heightened sensitivity additionally makes it susceptible to solar radiation noise, particularly in the green light wavelength where solar radiation is strong, posing challenges for its daytime operation. To address this issue, a single-photon underwater lidar system is proposed and demonstrated. This scheme has these features. 1) Underwater applications not only mitigate the impact of the air-water interface on laser transmission but also significantly attenuate solar radiation reaching the lidar due to the absorption and scattering properties of water. 2) The telescope is designed with a small aperture and narrow field of view to significantly suppress solar radiation. 3) A combination of a narrowband laser and narrowband filter technique is effectively employed to minimize residual solar radiation, thus enabling continuous bathymetric observation capabilities during both day and night. 4) After acquiring the backscattered signal from the bottom, a water depth extraction algorithm utilizing bi-Gaussian fitting is proposed. To demonstrate the robustness of the lidar and the effectiveness of the algorithm, the underwater single-photon lidar system is deployed on a ship to conduct cruise surveys of two bays in the nearshore area, as well as a full-day stationary observation experiment. The lidar measurements are highly consistent with the synchronized sonar observations. The full-day stationary observation experiment showcased its capability to deliver continuous measurements throughout the day and night. These results demonstrate the potential of the system in various applications, including high-precision underwater terrain mapping, obstacle avoidance for underwater platforms, and underwater target imaging.

The Impact of LiDAR Configuration on Goal-Based Navigation within a Deep Reinforcement Learning Framework.

Over the years, deep reinforcement learning (DRL) has shown great potential in mapless autonomous robot navigation and path planning. These DRL methods rely on robots equipped with different light detection and range (LiDAR) sensors with a wide field of view (FOV) configuration to perceive their environment. These types of LiDAR sensors are expensive and are not suitable for small-scale applications. In this paper, we address the performance effect of the LiDAR sensor configuration in DRL models. Our focus is on avoiding static obstacles ahead. We propose a novel approach that determines an initial FOV by calculating an angle of view using the sensor's width and the minimum safe distance required between the robot and the obstacle. The beams returned within the FOV, the robot's velocities, the robot's orientation to the goal point, and the distance to the goal point are used as the input state to generate new velocity values as the output action of the DRL. The cost function of collision avoidance and path planning is defined as the reward of the DRL model. To verify the performance of the proposed method, we adjusted the proposed FOV by ±10° giving a narrower and wider FOV. These new FOVs are trained to obtain collision avoidance and path planning DRL models to validate the proposed method. Our experimental setup shows that the LiDAR configuration with the computed angle of view as its FOV performs best with a success rate of 98% and a lower time complexity of 0.25 m/s. Additionally, using a Husky Robot, we demonstrate the model's good performance and applicability in the real world.

FLARE: Fast Learning of Animatable and Relightable Mesh Avatars

Our goal is to efficiently learn personalized animatable 3D head avatars from videos that are geometrically accurate, realistic, relightable, and compatible with current rendering systems. While 3D meshes enable efficient processing and are highly portable, they lack realism in terms of shape and appearance. Neural representations, on the other hand, are realistic but lack compatibility and are slow to train and render. Our key insight is that it is possible to efficiently learn high-fidelity 3D mesh representations via differentiable rendering by exploiting highly-optimized methods from traditional computer graphics and approximating some of the components with neural networks. To that end, we introduce FLARE, a technique that enables the creation of animatable and relightable mesh avatars from a single monocular video. First, we learn a canonical geometry using a mesh representation, enabling efficient differentiable rasterization and straightforward animation via learned blendshapes and linear blend skinning weights. Second, we follow physically-based rendering and factor observed colors into intrinsic albedo, roughness, and a neural representation of the illumination, allowing the learned avatars to be relit in novel scenes. Since our input videos are captured on a single device with a narrow field of view, modeling the surrounding environment light is non-trivial. Based on the split-sum approximation for modeling specular reflections, we address this by approximating the prefiltered environment map with a multi-layer perceptron (MLP) modulated by the surface roughness, eliminating the need to explicitly model the light. We demonstrate that our mesh-based avatar formulation, combined with learned deformation, material, and lighting MLPs, produces avatars with high-quality geometry and appearance, while also being efficient to train and render compared to existing approaches.

Autonomous GPU-based UAS for inspection of confined spaces: Application to marine vessel classification

Inspection of confined spaces poses a series of health risks to human surveyors, and therefore a need for robotic solutions arises. In this paper, we design and demonstrate a real-time system for collecting 3D structural and visual data from a series of inspection points within a prior map of a confined space. The system consists of a GPU accelerated 3D point cloud registration and a visual inertial odometry estimate fused in an Unscented Kalman Filter. Using the state-of-the-art deep learning-based feature descriptors, FCGF –and the robust Teaser++ 3D registration algorithm– point clouds from a narrow field of view, time-of-flight, camera can be registered to a prior map of the environment, to provide accurate cm-level absolute pose estimates. The uncertainty of the system is furthermore estimated on the basis of the novel GPU-based Stein ICP algorithm. Visual defects, represented by augmented reality fiducial markers, are automatically detected during inspection, and their positions are estimated in the map frame of the confined space. The performance of the system has been evaluated in realtime onboard a small UAV, within a mock-up model of a water ballast tank from a marine vessel, where the UAV was able to navigate and inspect the ambiguous and featureless environment. All defects were estimated within ＋/−10 cm of their actual position.

An Automatic and Robust Visual SLAM Method for Intra-Abdominal Environment Reconstruction

Three-dimensional (3D) surface reconstruction is used to solve the problem of the narrow field of view in laparoscopy. It can provide surgeons or computer-assisted surgery systems with real-time complete internal abdominal anatomy. However, rapid changes in image depth, less texture, and specular reflection pose a challenge for the reconstruction. It is difficult to stably complete the reconstruction process using feature-based simultaneous localization and mapping (SLAM) method. This paper proposes a robust laparoscopic 3D surface reconstruction method using SLAM, which can automatically select appropriate parameters for stereo matching and robustly find matching point pairs for laparoscope motion estimation. The changing trend of disparity maps is used to predict stereo matching parameters to improve the quality of the disparity map. Feature patch extraction and tracking are selected to replace feature point extraction and matching in motion estimation, which reduces its failure and interruption in feature-based SLAM. The proposed feature patch matching method is suitable for parallel computing, which can improve its computing speed. Evaluation results on public in vivo and ex vivo porcine abdominal video data show the efficiency and robustness of our 3D surface reconstruction approach.