Robust near-infrared structured light scanning for 3D human model reconstruction

In this paper we present a novel sensing system, robust Near-infrared Structured Light Scanning (NIRSL) for three-dimensional human model scanning application. Human model scanning due to its nature of various hair and dress appearance and body motion has long been a challenging task. Previous structured light scanning methods typically emitted visible coded light patterns onto static and opaque objects to establish correspondence between a projector and a camera for triangulation. In the success of these methods rely on scanning objects with proper reflective surface for visible light, such as plaster, light colored cloth. Whereas for human model scanning application, conventional methods suffer from low signal to noise ratio caused by low contrast of visible light over the human body. The proposed robust NIRSL, as implemented with the near infrared light, is capable of recovering those dark surfaces, such as hair, dark jeans and black shoes under visible illumination. Moreover, successful structured light scan relies on the assumption that the subject is static during scanning. Due to the nature of body motion, it is very time sensitive to keep this assumption in the case of human model scan. The proposed sensing system, by utilizing the new near-infrared capable high speed LightCrafter DLP projector, is robust to motion, provides accurate and high resolution three-dimensional point cloud, making our system more efficient and robust for human model reconstruction. Experimental results demonstrate that our system is effective and efficient to scan real human models with various dark hair, jeans and shoes, robust to human body motion and produces accurate and high resolution 3D point cloud.

High resolution consumer cameras on Unmanned Aerial Vehicles (UAVs) allow for cheap acquisition of highly detailed images, e.g., of urban regions. Via image registration by means of Structure from Motion (SfM) and Multi View Stereo (MVS) the automatic generation of huge amounts of 3D points with a relative accuracy in the centimeter range is possible. Applications such as semantic classification have a need for accurate 3D point clouds, but do not benefit from an extremely high resolution/density. In this paper, we, therefore, propose a fast fusion of high resolution 3D point clouds based on occupancy grids. The result is used for semantic classification. In contrast to state-of-the-art classification methods, we accept a certain percentage of outliers, arguing that they can be considered in the classification process when a per point belief is determined in the fusion process. To this end, we employ an octree-based fusion which allows for the derivation of outlier probabilities. The probabilities give a belief for every 3D point, which is essential for the semantic classification to consider measurement noise. For an example point cloud with half a billion 3D points (cf. Figure 1), we show that our method can reduce runtime as well as improve classification accuracy and offers high scalability for large datasets.

Abstract. High resolution consumer cameras on Unmanned Aerial Vehicles (UAVs) allow for cheap acquisition of highly detailed images, e.g., of urban regions. Via image registration by means of Structure from Motion (SfM) and Multi View Stereo (MVS) the automatic generation of huge amounts of 3D points with a relative accuracy in the centimeter range is possible. Applications such as semantic classification have a need for accurate 3D point clouds, but do not benefit from an extremely high resolution/density. In this paper, we, therefore, propose a fast fusion of high resolution 3D point clouds based on occupancy grids. The result is used for semantic classification. In contrast to state-of-the-art classification methods, we accept a certain percentage of outliers, arguing that they can be considered in the classification process when a per point belief is determined in the fusion process. To this end, we employ an octree-based fusion which allows for the derivation of outlier probabilities. The probabilities give a belief for every 3D point, which is essential for the semantic classification to consider measurement noise. For an example point cloud with half a billion 3D points (cf. Figure 1), we show that our method can reduce runtime as well as improve classification accuracy and offers high scalability for large datasets.

Point cloud can assist unmanned equipment to locate and detect in electric power inspection. It needs equipment and surrounding environment to obtain point cloud directly by radar. The efficiency of obtaining 3D point cloud in patrol inspection can be improved by using deep learning network through single image generation. In order to generate high-precision reconstruction results, a two-stage training network for 3D point cloud reconstruction is proposed in this paper. Firstly, the network of image to point cloud is trained and used to generate rough point cloud. Secondly, the trained point cloud auto-encoder generates more accurate point cloud data. Finally, the two models are combined to obtain accurate point cloud reconstruction results from an image. This method can generate accurate and uniform point cloud 3D model. The validity and practicability of the model are proved by the test of synthetic data set and the quantitative and qualitative analysis. Compared with the other three famous networks, the proposed network reconstruction accuracy is improved.

An unsupervised automatic measurement of wheat spike dimensions in dense 3D point clouds for field application

We are studying the transmission of LED array-emitted near-infrared (NIR) light through human tissues. Herein, we simulated and measured transcranial NIR penetration in highly scattering human head tissues. Using finite element analysis, we simulated photon diffusion in a multilayered 3D human head model that consists of scalp, skull, cerebral spinal fluid, gray matter and white matter. The optical properties of each layer, namely scattering and absorption coefficient, correspond to the 850 nm NIR light. The geometry of the model is minimally modified from the IEEE standard and the multiple LED emitters in an array were evenly distributed on the scalp. Our results show that photon distribution produced by the array exhibits little variation at similar brain depth, suggesting that due to strong scattering effects of the tissues, discrete spatial arrangements of LED emitters in an array has the potential to create a quasi-radially symmetrical illumination field. Measurements on cadaveric human head tissues excised from occipital, parietal, frontal and temporal regions show that illumination with an 850 nm LED emitter rendered a photon flux that closely follows simulation results. In addition, prolonged illumination of LED emitted NIR showed minimal thermal effects on the brain.

Photobiomodulation (PBM) using near-infrared (NIR) light is a novel neuromodulation technique. However, despite the many in vivo studies, the stimulation protocols for PBM vary across studies, and the current understanding of the physiological effects of PBM, as well as the dose dependence, is limited. Specifically, although NIR light can be absorbed by melanin in the skin, the understanding of how skin tones compare and how their influence interacts with other dose parameters remains limited. This study investigates the effect of melanin, optical power density, and wavelength on light penetration and energy accumulation via forehead and intranasal PBM. We use Monte Carlo simulations of a single laser source for transcranial (tPBM, forehead position) and intranasal (iPBM, nostril position) irradiation on a healthy human brain model. We investigate wavelengths of 670, 810, and 1064 nm at various power densities in combination with light (“Caucasian”), medium (“Asian”), and dark (“African”) skin tone categories as defined in the literature. Our simulations show that a maximum of 15% of the incidental energy for tPBM and 1% for iPBM reaches the cortex from the light source. The rostral dorsal prefrontal cortex and the ventromedial prefrontal cortex accumulate the highest light energy in tPBM and iPBM, respectively, for both wavelengths. Notably, we show that nominally “Caucasian” skin allows the highest energy accumulation of all three skin tones. Moreover, the 810 nm wavelength for tPBM and the 1064 nm wavelength for iPBM produced the highest cortical energy accumulation, which was linearly correlated with optical power density, but these variations could be overridden by a difference in skin tone in the tPBM case.The simulations serve as a starting point for enabling hypothesis generation for in vivo PBM investigations. This study is the first to account for skin tone as a tPBM dosing consideration. For the future of PBM research, it is important to evaluate combinations of stimulation parameters (wavelength, optical power density, pulsation frequency, duration, light source) when working to determine an optimal dosage for PBM-based therapy.

The human body model scanned by a structured light scanner is based on the scan coordinate system. Since the structured light scanner is not fixed, when the scanner scanning human body in different position, we can get several models, the coordinates of the same point on these models are not the same. In order to solve this problem, we propose a method. We extract facial feature points with the use of mean curvature analysis. The feature points are used to determine the digital human head model coordinate system. We can convert the human head models from the scan coordinate system to the digital human head model coordinate system. After the conversion, the coordinates of a same point on different models are approximately the same, which can make the use of scanner more efficiency and user-friendliness.

In this paper, we present an approach for human model reconstruction from a set of pictures. We take a set of pictures around the medical human model using a digital camera. Through calibration, we obtain the intrinsic parameters and extrinsic parameters, and compute the projective matrix of each image. After pre-processing the images, we reconstruct the human model from images according to the shape from silhouettes, using marching cubes algorithm to get the mesh model. We the smoothed and simplified mesh model. Finally, we rig the skeleton to the mesh automatically and drive it using motion capture data.

Abstract. The digitisation of museum exhibits has played an essential role in geomatics research for generating digital replicas, as it offers the chance to address rather challenging issues. The use of different sensors, ranging from active to passive, and also structured light scanners or hybrid solutions, the various destinations and purposes of the final results combined with the extreme variety of possible objects have made it a field of investigation highly inquired in the literature.The present study aims to analyse and discuss a digitalisation workflow applied to four Sumerian civilisation masterpieces preserved in the British Museum. The dense and accurate 3D point clouds derived from a specimen of Articulated Arm Coordinate Measuring Machines in collaboration with Faro technologies have twofold roles: ground truth and geometric reference of the final digital replicas. Digital photogrammetry is employed to enrich the models with the relevant radiometric component. The significant contribution results, exploiting co-registration strategies, offer careful guidance of a photogrammetric protocol created in a highly controlled environment combined with skilful expedients and devices. The proposed approach enables the acquisition of high-quality and radiometrically balanced images and improves the possibility of automating the masking procedure before the photogrammetric processing.

Abstract. Unmanned Aerial System (UAS) technology is nowadays willingly used in small area topographic mapping due to low costs and good quality of derived products. Since cameras typically used with UAS have some limitations, e.g. cannot penetrate the vegetation, LiDAR sensors are increasingly getting attention in UAS mapping. Sensor developments reached the point when their costs and size suit the UAS platform, though, LiDAR UAS is still an emerging technology. One issue related to using LiDAR sensors on UAS is the limited performance of the navigation sensors used on UAS platforms. Therefore, various hardware and software solutions are investigated to increase the quality of UAS LiDAR point clouds. This work analyses several aspects of the UAS LiDAR point cloud generation performance based on UAS flights conducted with the Velodyne laser scanner and cameras. The attention was primarily paid to the trajectory reconstruction performance that is essential for accurate point cloud georeferencing. Since the navigation sensors, especially Inertial Measurement Units (IMUs), may not be of sufficient performance, the estimated camera poses could allow to increase the robustness of the estimated trajectory, and subsequently, the accuracy of the point cloud. The accuracy of the final UAS LiDAR point cloud was evaluated on the basis of the generated DSM, including comparison with point clouds obtained from dense image matching. The results showed the need for more investigation on MEMS IMU sensors used for UAS trajectory reconstruction. The accuracy of the UAS LiDAR point cloud, though lower than for point cloud obtained from images, may be still sufficient for certain mapping applications where the optical imagery is not useful.

Unmanned Aerial System (UAS) technology is nowadays willingly used in small area topographic mapping due to low costs and good quality of derived products. Since cameras typically used with UAS have some limitations, e.g. cannot penetrate the vegetation, LiDAR sensors are increasingly getting attention in UAS mapping. Sensor developments reached the point when their costs and size suit the UAS platform, though, LiDAR UAS is still an emerging technology. One issue related to using LiDAR sensors on UAS is the limited performance of the navigation sensors used on UAS platforms. Therefore, various hardware and software solutions are investigated to increase the quality of UAS LiDAR point clouds. This work analyses several aspects of the UAS LiDAR point cloud generation performance based on UAS flights conducted with the Velodyne laser scanner and cameras. The attention was primarily paid to the trajectory reconstruction performance that is essential for accurate point cloud georeferencing. Since the navigation sensors, especially Inertial Measurement Units (IMUs), may not be of sufficient performance, the estimated camera poses could allow to increase the robustness of the estimated trajectory, and subsequently, the accuracy of the point cloud. The accuracy of the final UAS LiDAR point cloud was evaluated on the basis of the generated DSM, including comparison with point clouds obtained from dense image matching. The results showed the need for more investigation on MEMS IMU sensors used for UAS trajectory reconstruction. The accuracy of the UAS LiDAR point cloud, though lower than for point cloud obtained from images, may be still sufficient for certain mapping applications where the optical imagery is not useful.

<p>Understanding the dynamics of coastal areas is crucial to mitigate the effects of global change though monitoring these places could be challenging, difficult and dangerous, especially in the presence of (unstable) cliffs. The recent development of Unmanned Aerial Systems (UAS) with accurate direct georeferencing systems facilitates this task. The objective of this work is to test the performance of different 3D data acquisition strategies in coastal cliffs, specifically RGB and LIDAR sensors on board UAS platforms equipped with direct georeferencing instruments based on Global Navigation Satellite Systems (GNSS: Real Time Kinematic-RTK and Post-Processing Kinematic-PPK approaches). Two UAS were used to capture data and produce point clouds of a coastal cliff in the Cantabrian Coast (Gerra beach, North Spain): a DJI Phantom 4 RTK (P4RTK) and a MD4-1000 LIDAR. The P4RTK may receive corrections to estimate accurate positions of the UAS during the acquisition of images (P4RTK processing approach), but also may record the trajectory of the UAS to carry out a PPK approach later to correct and estimate the location of the camera at every shot (P4RTK-PPK processing approach).  Two GNSS receivers (Leica 1200 working as base and rover) were used to survey 31 points distributed in the study area. The surveyed points were used in different number (from 0 to 10) as Ground Control Points (GCPs: to support the production of the point clouds) or Check Control Points (CCPs: to independently test the geometrical accuracy of the point clouds) in the photogrammetric processing (using two parallel pipelines with Agisoft Metashape and Pix4Dmapper Pro software packages). The MD4-1000 LIDAR is a quadcopter UAS equipped with the following instruments: a LIDAR sensor SICK LD-MRS4 (to capture the point cloud), a Ladybug RGB camera (to acquire images and colour the point cloud), and a GNSS antenna (Trimble APX-15v3) with an integrated Inertial Measurement Unit. The trajectory of the UAS recorded by the GNSS may be corrected using observations registered by a GNSS base station to obtain the accurate pose of the UAS using a PPK approach.</p><p>Additionally, a benchmark point cloud was acquired by a Terrestrial Laser Scanner (Leica ScanStation C10) placed at 5 locations. The resulting point cloud showed 23,4 million points with a registration error of 7 mm. Three parameters were used to test the quality of the resulting point clouds: point cloud density and coverage, distance to the benchmark point cloud and RMSE of CCPs. The results showed that any of the strategies produced very accurate point clouds with a geometrical accuracy <10 cm. The P4RTK (RTK, PPK or using GCPs) produced more accurate and denser point clouds than the MD4-1000 LIDAR system (only PPK approach). The use of GCPs did not improved substantially the point clouds produced by photogrammetry (and RTK or PPK approaches) if an oblique pass is included in the flight plan to improve the camera focal estimation and corrections are available.</p>

Monitoring the dynamics of coastal cliffs is fundamental for the safety of communities, buildings, utilities, and infrastructures located near the coastline. Structure-from-Motion and Multi View Stereo (SfM-MVS) photogrammetry based on Unmanned Aerial Systems (UAS) is a flexible and cost-effective surveying technique for generating a dense 3D point cloud of the whole cliff face (from bottom to top), with high spatial and temporal resolution. In this paper, in order to generate a reproducible, reliable, precise, accurate, and dense point cloud of the cliff face, a comprehensive analysis of the SfM-MVS processing parameters, image redundancy and acquisition geometry was performed. Using two different UAS, a fixed-wing and a multi-rotor, two flight missions were executed with the aim of reconstructing the geometry of an almost vertical cliff located at the central Portuguese coast. The results indicated that optimizing the processing parameters of Agisoft Metashape can improve the 3D accuracy of the point cloud up to 2 cm. Regarding the image acquisition geometry, the high off-nadir (90°) dataset taken by the multi-rotor generated a denser and more accurate point cloud, with lesser data gaps, than that generated by the low off-nadir dataset (3°) taken by the fixed wing. Yet, it was found that reducing properly the high overlap of the image dataset acquired by the multi-rotor drone permits to get an optimal image dataset, allowing to speed up the processing time without compromising the accuracy and density of the generated point cloud. The analysis and results presented in this paper improve the knowledge required for the 3D reconstruction of coastal cliffs by UAS, providing new insights into the technical aspects needed for optimizing the monitoring surveys.

ContextRecently, Unoccupied Aerial Systems (UAS) with photographic or Light Detection and Ranging (LIDAR) sensors have incorporated on-board survey-grade Global Navigation Satellite Systems that allow the direct georeferencing of the resulting datasets without Ground Control Points either in Real-Time (RTK) or Post-Processing Kinematic (PPK) modes. These approaches can be useful in hard-to-reach or hazardous areas. However, the resulting 3D models have not been widely tested, as previous studies tend to evaluate only a few points and conclude that systematic errors can be found.ObjectivesWe test the absolute positional accuracy of point clouds produced using UAS with direct-georeferencing systems.MethodsWe test the accuracy and characteristics of point clouds produced using a UAS-LIDAR (with PPK) and a UAS-RGB (Structure-from-Motion or SfM photogrammetry with RTK and PPK) in a challenging environment: a coastline with a composite beach and cliff. The resulting models of each processing were tested using as a benchmark a point cloud surveyed simultaneously by a Terrestrial Laser Scanner.ResultsThe UAS-LIDAR produced the most accurate point cloud, with homogeneous cover and no noise. The systematic bias previously observed in the UAS-RGB RTK approaches are minimized using oblique images. The accuracy observed across the different surveyed landforms varied significantly.ConclusionsThe UAS-LIDAR and UAS-RGB with PPK produced unbiased point clouds, being the latter the most cost-effective method. For the other direct georeferencing systems/approaches, the acquisition of GCP or the co-registration of the resulting point cloud is still necessary.