DLP technology application: 3D head tracking and motion correction in medical brain imaging

In this paper we present a novel sensing system, robust Near-infrared Structured Light Scanning (NIRSL) for three-dimensional human model scanning application. Human model scanning due to its nature of various hair and dress appearance and body motion has long been a challenging task. Previous structured light scanning methods typically emitted visible coded light patterns onto static and opaque objects to establish correspondence between a projector and a camera for triangulation. In the success of these methods rely on scanning objects with proper reflective surface for visible light, such as plaster, light colored cloth. Whereas for human model scanning application, conventional methods suffer from low signal to noise ratio caused by low contrast of visible light over the human body. The proposed robust NIRSL, as implemented with the near infrared light, is capable of recovering those dark surfaces, such as hair, dark jeans and black shoes under visible illumination. Moreover, successful structured light scan relies on the assumption that the subject is static during scanning. Due to the nature of body motion, it is very time sensitive to keep this assumption in the case of human model scan. The proposed sensing system, by utilizing the new near-infrared capable high speed LightCrafter DLP projector, is robust to motion, provides accurate and high resolution three-dimensional point cloud, making our system more efficient and robust for human model reconstruction. Experimental results demonstrate that our system is effective and efficient to scan real human models with various dark hair, jeans and shoes, robust to human body motion and produces accurate and high resolution 3D point cloud.

High resolution consumer cameras on Unmanned Aerial Vehicles (UAVs) allow for cheap acquisition of highly detailed images, e.g., of urban regions. Via image registration by means of Structure from Motion (SfM) and Multi View Stereo (MVS) the automatic generation of huge amounts of 3D points with a relative accuracy in the centimeter range is possible. Applications such as semantic classification have a need for accurate 3D point clouds, but do not benefit from an extremely high resolution/density. In this paper, we, therefore, propose a fast fusion of high resolution 3D point clouds based on occupancy grids. The result is used for semantic classification. In contrast to state-of-the-art classification methods, we accept a certain percentage of outliers, arguing that they can be considered in the classification process when a per point belief is determined in the fusion process. To this end, we employ an octree-based fusion which allows for the derivation of outlier probabilities. The probabilities give a belief for every 3D point, which is essential for the semantic classification to consider measurement noise. For an example point cloud with half a billion 3D points (cf. Figure 1), we show that our method can reduce runtime as well as improve classification accuracy and offers high scalability for large datasets.

Abstract. High resolution consumer cameras on Unmanned Aerial Vehicles (UAVs) allow for cheap acquisition of highly detailed images, e.g., of urban regions. Via image registration by means of Structure from Motion (SfM) and Multi View Stereo (MVS) the automatic generation of huge amounts of 3D points with a relative accuracy in the centimeter range is possible. Applications such as semantic classification have a need for accurate 3D point clouds, but do not benefit from an extremely high resolution/density. In this paper, we, therefore, propose a fast fusion of high resolution 3D point clouds based on occupancy grids. The result is used for semantic classification. In contrast to state-of-the-art classification methods, we accept a certain percentage of outliers, arguing that they can be considered in the classification process when a per point belief is determined in the fusion process. To this end, we employ an octree-based fusion which allows for the derivation of outlier probabilities. The probabilities give a belief for every 3D point, which is essential for the semantic classification to consider measurement noise. For an example point cloud with half a billion 3D points (cf. Figure 1), we show that our method can reduce runtime as well as improve classification accuracy and offers high scalability for large datasets.

Point cloud can assist unmanned equipment to locate and detect in electric power inspection. It needs equipment and surrounding environment to obtain point cloud directly by radar. The efficiency of obtaining 3D point cloud in patrol inspection can be improved by using deep learning network through single image generation. In order to generate high-precision reconstruction results, a two-stage training network for 3D point cloud reconstruction is proposed in this paper. Firstly, the network of image to point cloud is trained and used to generate rough point cloud. Secondly, the trained point cloud auto-encoder generates more accurate point cloud data. Finally, the two models are combined to obtain accurate point cloud reconstruction results from an image. This method can generate accurate and uniform point cloud 3D model. The validity and practicability of the model are proved by the test of synthetic data set and the quantitative and qualitative analysis. Compared with the other three famous networks, the proposed network reconstruction accuracy is improved.

Abstract. The digitisation of museum exhibits has played an essential role in geomatics research for generating digital replicas, as it offers the chance to address rather challenging issues. The use of different sensors, ranging from active to passive, and also structured light scanners or hybrid solutions, the various destinations and purposes of the final results combined with the extreme variety of possible objects have made it a field of investigation highly inquired in the literature.The present study aims to analyse and discuss a digitalisation workflow applied to four Sumerian civilisation masterpieces preserved in the British Museum. The dense and accurate 3D point clouds derived from a specimen of Articulated Arm Coordinate Measuring Machines in collaboration with Faro technologies have twofold roles: ground truth and geometric reference of the final digital replicas. Digital photogrammetry is employed to enrich the models with the relevant radiometric component. The significant contribution results, exploiting co-registration strategies, offer careful guidance of a photogrammetric protocol created in a highly controlled environment combined with skilful expedients and devices. The proposed approach enables the acquisition of high-quality and radiometrically balanced images and improves the possibility of automating the masking procedure before the photogrammetric processing.

An unsupervised automatic measurement of wheat spike dimensions in dense 3D point clouds for field application

The human body model scanned by a structured light scanner is based on the scan coordinate system. Since the structured light scanner is not fixed, when the scanner scanning human body in different position, we can get several models, the coordinates of the same point on these models are not the same. In order to solve this problem, we propose a method. We extract facial feature points with the use of mean curvature analysis. The feature points are used to determine the digital human head model coordinate system. We can convert the human head models from the scan coordinate system to the digital human head model coordinate system. After the conversion, the coordinates of a same point on different models are approximately the same, which can make the use of scanner more efficiency and user-friendliness.

Abstract. Unmanned Aerial System (UAS) technology is nowadays willingly used in small area topographic mapping due to low costs and good quality of derived products. Since cameras typically used with UAS have some limitations, e.g. cannot penetrate the vegetation, LiDAR sensors are increasingly getting attention in UAS mapping. Sensor developments reached the point when their costs and size suit the UAS platform, though, LiDAR UAS is still an emerging technology. One issue related to using LiDAR sensors on UAS is the limited performance of the navigation sensors used on UAS platforms. Therefore, various hardware and software solutions are investigated to increase the quality of UAS LiDAR point clouds. This work analyses several aspects of the UAS LiDAR point cloud generation performance based on UAS flights conducted with the Velodyne laser scanner and cameras. The attention was primarily paid to the trajectory reconstruction performance that is essential for accurate point cloud georeferencing. Since the navigation sensors, especially Inertial Measurement Units (IMUs), may not be of sufficient performance, the estimated camera poses could allow to increase the robustness of the estimated trajectory, and subsequently, the accuracy of the point cloud. The accuracy of the final UAS LiDAR point cloud was evaluated on the basis of the generated DSM, including comparison with point clouds obtained from dense image matching. The results showed the need for more investigation on MEMS IMU sensors used for UAS trajectory reconstruction. The accuracy of the UAS LiDAR point cloud, though lower than for point cloud obtained from images, may be still sufficient for certain mapping applications where the optical imagery is not useful.

Unmanned Aerial System (UAS) technology is nowadays willingly used in small area topographic mapping due to low costs and good quality of derived products. Since cameras typically used with UAS have some limitations, e.g. cannot penetrate the vegetation, LiDAR sensors are increasingly getting attention in UAS mapping. Sensor developments reached the point when their costs and size suit the UAS platform, though, LiDAR UAS is still an emerging technology. One issue related to using LiDAR sensors on UAS is the limited performance of the navigation sensors used on UAS platforms. Therefore, various hardware and software solutions are investigated to increase the quality of UAS LiDAR point clouds. This work analyses several aspects of the UAS LiDAR point cloud generation performance based on UAS flights conducted with the Velodyne laser scanner and cameras. The attention was primarily paid to the trajectory reconstruction performance that is essential for accurate point cloud georeferencing. Since the navigation sensors, especially Inertial Measurement Units (IMUs), may not be of sufficient performance, the estimated camera poses could allow to increase the robustness of the estimated trajectory, and subsequently, the accuracy of the point cloud. The accuracy of the final UAS LiDAR point cloud was evaluated on the basis of the generated DSM, including comparison with point clouds obtained from dense image matching. The results showed the need for more investigation on MEMS IMU sensors used for UAS trajectory reconstruction. The accuracy of the UAS LiDAR point cloud, though lower than for point cloud obtained from images, may be still sufficient for certain mapping applications where the optical imagery is not useful.

<p>Understanding the dynamics of coastal areas is crucial to mitigate the effects of global change though monitoring these places could be challenging, difficult and dangerous, especially in the presence of (unstable) cliffs. The recent development of Unmanned Aerial Systems (UAS) with accurate direct georeferencing systems facilitates this task. The objective of this work is to test the performance of different 3D data acquisition strategies in coastal cliffs, specifically RGB and LIDAR sensors on board UAS platforms equipped with direct georeferencing instruments based on Global Navigation Satellite Systems (GNSS: Real Time Kinematic-RTK and Post-Processing Kinematic-PPK approaches). Two UAS were used to capture data and produce point clouds of a coastal cliff in the Cantabrian Coast (Gerra beach, North Spain): a DJI Phantom 4 RTK (P4RTK) and a MD4-1000 LIDAR. The P4RTK may receive corrections to estimate accurate positions of the UAS during the acquisition of images (P4RTK processing approach), but also may record the trajectory of the UAS to carry out a PPK approach later to correct and estimate the location of the camera at every shot (P4RTK-PPK processing approach).  Two GNSS receivers (Leica 1200 working as base and rover) were used to survey 31 points distributed in the study area. The surveyed points were used in different number (from 0 to 10) as Ground Control Points (GCPs: to support the production of the point clouds) or Check Control Points (CCPs: to independently test the geometrical accuracy of the point clouds) in the photogrammetric processing (using two parallel pipelines with Agisoft Metashape and Pix4Dmapper Pro software packages). The MD4-1000 LIDAR is a quadcopter UAS equipped with the following instruments: a LIDAR sensor SICK LD-MRS4 (to capture the point cloud), a Ladybug RGB camera (to acquire images and colour the point cloud), and a GNSS antenna (Trimble APX-15v3) with an integrated Inertial Measurement Unit. The trajectory of the UAS recorded by the GNSS may be corrected using observations registered by a GNSS base station to obtain the accurate pose of the UAS using a PPK approach.</p><p>Additionally, a benchmark point cloud was acquired by a Terrestrial Laser Scanner (Leica ScanStation C10) placed at 5 locations. The resulting point cloud showed 23,4 million points with a registration error of 7 mm. Three parameters were used to test the quality of the resulting point clouds: point cloud density and coverage, distance to the benchmark point cloud and RMSE of CCPs. The results showed that any of the strategies produced very accurate point clouds with a geometrical accuracy <10 cm. The P4RTK (RTK, PPK or using GCPs) produced more accurate and denser point clouds than the MD4-1000 LIDAR system (only PPK approach). The use of GCPs did not improved substantially the point clouds produced by photogrammetry (and RTK or PPK approaches) if an oblique pass is included in the flight plan to improve the camera focal estimation and corrections are available.</p>

Monitoring the dynamics of coastal cliffs is fundamental for the safety of communities, buildings, utilities, and infrastructures located near the coastline. Structure-from-Motion and Multi View Stereo (SfM-MVS) photogrammetry based on Unmanned Aerial Systems (UAS) is a flexible and cost-effective surveying technique for generating a dense 3D point cloud of the whole cliff face (from bottom to top), with high spatial and temporal resolution. In this paper, in order to generate a reproducible, reliable, precise, accurate, and dense point cloud of the cliff face, a comprehensive analysis of the SfM-MVS processing parameters, image redundancy and acquisition geometry was performed. Using two different UAS, a fixed-wing and a multi-rotor, two flight missions were executed with the aim of reconstructing the geometry of an almost vertical cliff located at the central Portuguese coast. The results indicated that optimizing the processing parameters of Agisoft Metashape can improve the 3D accuracy of the point cloud up to 2 cm. Regarding the image acquisition geometry, the high off-nadir (90°) dataset taken by the multi-rotor generated a denser and more accurate point cloud, with lesser data gaps, than that generated by the low off-nadir dataset (3°) taken by the fixed wing. Yet, it was found that reducing properly the high overlap of the image dataset acquired by the multi-rotor drone permits to get an optimal image dataset, allowing to speed up the processing time without compromising the accuracy and density of the generated point cloud. The analysis and results presented in this paper improve the knowledge required for the 3D reconstruction of coastal cliffs by UAS, providing new insights into the technical aspects needed for optimizing the monitoring surveys.

ContextRecently, Unoccupied Aerial Systems (UAS) with photographic or Light Detection and Ranging (LIDAR) sensors have incorporated on-board survey-grade Global Navigation Satellite Systems that allow the direct georeferencing of the resulting datasets without Ground Control Points either in Real-Time (RTK) or Post-Processing Kinematic (PPK) modes. These approaches can be useful in hard-to-reach or hazardous areas. However, the resulting 3D models have not been widely tested, as previous studies tend to evaluate only a few points and conclude that systematic errors can be found.ObjectivesWe test the absolute positional accuracy of point clouds produced using UAS with direct-georeferencing systems.MethodsWe test the accuracy and characteristics of point clouds produced using a UAS-LIDAR (with PPK) and a UAS-RGB (Structure-from-Motion or SfM photogrammetry with RTK and PPK) in a challenging environment: a coastline with a composite beach and cliff. The resulting models of each processing were tested using as a benchmark a point cloud surveyed simultaneously by a Terrestrial Laser Scanner.ResultsThe UAS-LIDAR produced the most accurate point cloud, with homogeneous cover and no noise. The systematic bias previously observed in the UAS-RGB RTK approaches are minimized using oblique images. The accuracy observed across the different surveyed landforms varied significantly.ConclusionsThe UAS-LIDAR and UAS-RGB with PPK produced unbiased point clouds, being the latter the most cost-effective method. For the other direct georeferencing systems/approaches, the acquisition of GCP or the co-registration of the resulting point cloud is still necessary.

For most dense multi-view stereo methods, the process of finding correspondences is the basis and is independent of acquiring 3D information, and this often brings about erroneous correspondences followed by erroneous 3D information. To tackle this problem, by expanding matched points and by expanding 3D patches, this paper proposes an effective approach to acquire dense and accurate point clouds from multi-view uncalibrated images. In the approach, two novel algorithms are newly designed and are placed before and after the Bundler: 1) the match expansion algorithm, which generates evenly distributed correspondences with geometric consistency; after using Bundler to produce geometry estimation and quasi-dense point clouds which are not dense and accurate, 2) the point-cloud expansion algorithm, which is proposed to improve the density and accuracy of point clouds by optimizing the geometry of each 3D patch and expanding each good patch to its neighborhood. A large number of experimental results demonstrate the proposed approach get more accurate and denser point clouds than the state-of-the-art methods. A quantitative evaluation shows the accuracy of the proposed method favorable to PMVS.KeywordsMultiview stereoMatch expansionPoint-cloudexpansion

High resolution 3d point cloud data obtained by 3d laser scanning system has become a research hotspot and difficulty in recent years due to its large data volume, irregular data and high scene complexity. Target detection is the basis of scene analysis and understanding, which provides the underlying object and analysis basis for high-level scene understanding. Based on high resolution three-dimensional point cloud data of target recognition and tracking problem both in theory and application is facing great challenge, is a new research topic in this paper, according to the laser point cloud data processing as the research object, analyses the characteristics of lidar point cloud data and data processing of train of thought, analysis of lidar point cloud data storage and retrieval strategy, on the basis of the target recognition based on the laser point cloud data. The lidar data are distributed discretely in form. The discretization here refers to the irregular distribution of the positions and intervals of exponential data points in the three-dimensional space, namely the irregular distribution of data. In recent years, with the rise of deep learning and the large-scale application of deep learning in image detection, speech recognition, text processing and other related fields, it has become one of the current important research topics to use the method of deep learning for target recognition of three-dimensional point cloud data. Its main idea is to learn hierarchical feature expression through supervised way and describe the object from the bottom to the top. This method can effectively improve the ability of object feature representation and the performance of object recognition. Deep learning is also widely used in object recognition, object detection, scene segmentation and other image processing. Therefore, this paper adopts the method of deep learning to classify and identify 3d objects.

The psychological and social impact of the lipodystrophy syndrome on HIV-infected individuals may be quite considerable and adversely affect their quality of life. Currently no validated assessment tool for facial lipoatrophy is available. The main objective of this paper is to evaluate the reliability of interactive anthropometric landmark localization based on digitized 3D facial images. By comparing both computed tomography (CT) and structured light scanning we try to demonstrate that surface scanning shows a higher sensitivity in measuring facial reference points. Besides, we evaluate the reproducibility of facial 3D white-light scans. Three HIVpositive men attending our plastic surgery outpatient clinic for treatment of facial lipodystrophy were enrolled in the study. Localization of anthropometric landmarks measurements was performed on the patients. All patients underwent a facial CT and a facial white-light scanning on the same day. The inter-landmark distances measured on facial models developed from CT aided with VirSSPA 3D software and structured light scanning were compared to the real human models. We found that facial distances measured in the CT 3D reconstruction showed a mean error margin of 0.357 cm from the real distances measured on patients. On the contrary, mean error margin with the white-light scanning was of 0.096 cm. In both cases, measurements were found to be statistically significant (p < 0.05). When compared to CT reconstructions, white-light surface scanning offers a more accurate landmark localization as well as reliable reconstructions of up to less than the tenth of a millimetre as average when compared to real measurements on facial human models.