Proposed 3D Reconstruction Method Research Articles

Three-dimensional nanoscale reduced-angle ptycho-tomographic imaging with deep learning (RAPID)

X-ray ptychographic tomography is a nondestructive method for three dimensional (3D) imaging with nanometer-sized resolvable features. The size of the volume that can be imaged is almost arbitrary, limited only by the penetration depth and the available scanning time. Here we present a method that rapidly accelerates the imaging operation over a given volume through acquiring a limited set of data via large angular reduction and compensating for the resulting ill-posedness through deeply learned priors. The proposed 3D reconstruction method “RAPID” relies initially on a subset of the object measured with the nominal number of required illumination angles and treats the reconstructions from the conventional two-step approach as ground truth. It is then trained to reproduce equal fidelity from much fewer angles. After training, it performs with similar fidelity on the hitherto unexamined portions of the object, previously not shown during training, with a limited set of acquisitions. In our experimental demonstration, the nominal number of angles was 349 and the reduced number of angles was 21, resulting in a times 140 aggregate speedup over a volume of 4.48times 93.18times 3.92, upmu text {m}^3 and with (14,text {nm})^3 feature size, i.e. sim 10^8 voxels. RAPID’s key distinguishing feature over earlier attempts is the incorporation of atrous spatial pyramid pooling modules into the deep neural network framework in an anisotropic way. We found that adjusting the atrous rate improves reconstruction fidelity because it expands the convolutional kernels’ range to match the physics of multi-slice ptychography without significantly increasing the number of parameters.

EDIT Software: A tool for the semi-automatic 3D reconstruction of bladder cancer and urinary bladder of animal models

Background and objectiveThis study presents the EDIT software, a tool for the visualization of the urinary bladder anatomy in the 3D space and for its semi-automatic 3D reconstruction. Methods: The inner bladder wall was computed by applying a Region of Interest (ROI) feedback-based active contour algorithm on the ultrasound images while the outer bladder wall was calculated by expanding the inner borders to approach the vascularization area on the photoacoustic images. The validation strategy of the proposed software was divided into two processes. Initially, the 3D automated reconstruction was performed on 6 phantom objects of different volume in order to compare the software computed volumes of the models with the true volumes of phantoms. Secondly, the in-vivo 3D reconstruction of the urinary bladder for 10 animals with orthotopic bladder cancer, which range in different stages of tumor progression was performed. Results: The results showed that the minimum volume similarity of the proposed 3D reconstruction method applied on phantoms is 95.59%. It is noteworthy to mention that the EDIT software enables the user to reconstruct the 3D bladder wall with high precision, even if the bladder silhouette has been significantly deformed by the tumor. Indeed, by taking into account the dataset of the 2251 in-vivo ultrasound and photoacoustic images, the presented software performs segmentation with dice similarity 96.96% and 90.91% for the inner and the outer borders of the bladder wall, respectively. Conclusions: This study delivers the EDIT software, a novel software tool that uses ultrasound and photoacoustic images to extract different 3D components of the bladder.

FUSED 3D TRANSPARENT VISUALIZATION FOR LARGE-SCALE CULTURAL HERITAGE USING DEEP LEARNING-BASED MONOCULAR RECONSTRUCTION

Abstract. This paper proposes a fused 3D transparent visualization method with the aim of achieving see-through imaging of large-scale cultural heritage by combining photogrammetry point cloud data and 3D reconstructed models. 3D reconstructed models are efficiently reconstructed from a single monocular photo using deep learning. It is demonstrated that the proposed method can be widely applied, particularly to instances of incomplete cultural heritages. In this study, the proposed method is applied to a typical example, the Borobudur temple in Indonesia. The Borobudur temple possesses the most complete collection of Buddhist reliefs. However, some parts of the Borobudur reliefs have been hidden by stone walls and became not visible following the reinforcements during the Dutch rule. Today, only gray-scale monocular photos of those hidden parts are displayed in the Borobudur Museum. In this paper, the visible parts of the temple are first digitized into point cloud data by photogrammetry scanning. For the hidden parts, a 3D reconstruction method based on deep learning is proposed to reconstruct the invisible parts into point cloud data directly from single monocular photos from the museum. The proposed 3D reconstruction method achieves 95% accuracy of the reconstructed point cloud on average. With the point cloud data of both the visible parts and the hidden parts, the proposed transparent visualization method called the stochastic point-based rendering is applied to achieve a fused 3D transparent visualization of the valuable temple.

3D Reconstruction of Slug Flow in Mini-Channels with a Simple and Low-Cost Optical Sensor.

The present work provides a new approach for 3D image reconstruction of gas-liquid two-phase flow (GLF) in mini-channels based on a new optical sensor. The sensor consists of a vertical and a horizontal photodiode array. Firstly, with the optical signals obtained by the vertical array, a measurement model developed by Support Vector Regression (SVR) was used to determine the cross-sectional information. The determined information was further used to reconstruct cross-sectional 2D images. Then, the gas velocity was calculated according to the signals obtained by the horizontal array, and the spatial interval of the 2D images was determined. Finally, 3D images were reconstructed by piling up the 2D images. In this work, the cross-sectional gas-liquid interface was considered as circular, and high-speed visualization was utilized to provide the reference values. The image deformation caused by channel wall was also considered. Experiments of slug flow in a channel with an inner diameter of 4.0 mm were carried out. The results verify the feasibility of the proposed 3D reconstruction method. The proposed method has the advantages of simple construct, low cost, and easily multipliable. The reconstructed 3D images can provide detailed and undistorted information of flow structure, which could further improve the measurement accuracy of other important parameters of gas-liquid two-phase flow, such as void fraction, pressure drop, and flow pattern.

Volumetric estimation using 3D reconstruction method for grading of fruits

Grading of the fruits is one of the important post harvest tasks that the fruit processing agro-industries do. Although the internal quality of the fruit is important, the external quality of the fruit influences the consumers and the market price significantly. External quality of the fruit is based on the features such as color, maturity, shape, texture and size of the fruit. Apart from being expensive and time consuming, the manual grading process may face challenges such as subjectivity in grading, inconsistency and non-availability of the experts during peak seasons. On the other hand, computer vision based fruit grading systems using 2D techniques do not consider self occluding surface of the fruit and fail to determine the percentage of the matured region accurately. The grading systems which approximate the shape of the fruit to a known geometrical shape fail to compute the volume of the fruits with arbitrary shapes accurately. This paper presents a nondestructive and accurate fruit grading system based on the volume and maturity feature implemented using Fuzzy Rule Based Classifier (FRBC). The system estimates the volume of the fruit using volumetric 3D reconstruction method in multiple-camera environment and computes the percentage of the matured region of the fruit with high accuracy. The experimental results show that the accuracy of the proposed grading system in volume estimation and fruit grading is 98.5%. The ability of the proposed 3D reconstruction method to reconstruct the fruits with arbitrary shapes makes the grading system more robust and dynamic.

Volumetric 3D reconstruction of real objects using voxel mapping approach in a multiple-camera environment

Extracting 3D information from 2D images is an inverse estimation problem and a challenging task in itself. The aim of 2D to 3D reconstruction is to generate either a volume or a surface representing the object from multiple views. This paper presents a simple and accurate multiple-view volumetric 3D reconstruction method using an integrated approach based on homography estimation and voxel mapping. The homography-based approaches give accurate estimates but do not provide system dynamics. The voxel-based volumetric reconstruction methods provide system dynamics that are essential for system modeling. However, they face challenges while modeling the concavities. This paper presents a proposed 3D reconstruction method that combines homography estimation and the voxel mapping approach for improving the accuracy of 3D reconstruction. Experimental results show that the method efficiently reconstructs objects of known and unknown shape, fragile objects, and complex scenes with multiple objects. The use of homography along with voxel mapping in a multiple-camera environment brings out more details of the object for improving the quality of reconstruction.

A Fast 3-Dimensional Magnetic Resonance Imaging Reconstruction for Surgical Planning of Uterine Myomectomy.

BackgroundUterine myoma is the most common benign gynecologic tumor in reproductive-aged women. During myomectomy for women who want to preserve fertility, it is advisable to detect and remove all myomas to decrease the risk of additional surgery. However, finding myomas during surgery is often challenging, especially for deep-seated myomas. Therefore, three-dimensional (3D) preoperative localization of myomas can be helpful for the surgical planning for myomectomy. However, the previously reported manual 3D segmenting method takes too much time and effort for clinical use. The objective of this study was to propose a new method of rapid 3D visualization of uterine myoma using a uterine template.MethodsMagnetic resonance images were listed according to the slide spacing on each plane of the multiplanar reconstruction, and images that were determined to be myomas were selected by simply scrolling the mouse down. By using the selected images, a 3D grid with a slide spacing interval was constructed and filled on its plane and finally registered to a uterine template.ResultsThe location of multiple myomas in the uterus was visualized in 3D and this proposed method is over 95% faster than the existing manual-segmentation method. Not only the size and location of the myomas, but also the shortest distance between the uterine surface and the myomas, can be calculated. This technique also enables the surgeon to know the number of total, removed, and remaining myomas on the 3D image.ConclusionThis proposed 3D reconstruction method with a uterine template enables faster 3D visualization of myomas.

Adaptive Geometric Tessellation for 3D Reconstruction of Anisotropically Developing Cells in Multilayer Tissues from Sparse Volumetric Microscopy Images

The need for quantification of cell growth patterns in a multilayer, multi-cellular tissue necessitates the development of a 3D reconstruction technique that can estimate 3D shapes and sizes of individual cells from Confocal Microscopy (CLSM) image slices. However, the current methods of 3D reconstruction using CLSM imaging require large number of image slices per cell. But, in case of Live Cell Imaging of an actively developing tissue, large depth resolution is not feasible in order to avoid damage to cells from prolonged exposure to laser radiation. In the present work, we have proposed an anisotropic Voronoi tessellation based 3D reconstruction framework for a tightly packed multilayer tissue with extreme z-sparsity (2–4 slices/cell) and wide range of cell shapes and sizes. The proposed method, named as the ‘Adaptive Quadratic Voronoi Tessellation’ (AQVT), is capable of handling both the sparsity problem and the non-uniformity in cell shapes by estimating the tessellation parameters for each cell from the sparse data-points on its boundaries. We have tested the proposed 3D reconstruction method on time-lapse CLSM image stacks of the Arabidopsis Shoot Apical Meristem (SAM) and have shown that the AQVT based reconstruction method can correctly estimate the 3D shapes of a large number of SAM cells.

Three-dimensional reconstruction based on multiple virtual planes by using fusion-based camera network

A novel approach for three-dimensional (3D) volumetric reconstruction of an object inside a scene is proposed. A camera network is used to observe the scene. Each camera within the network is rigidly coupled with an Inertial Sensor (IS). A virtual camera is defined for each IS-camera couple using the concept of infinite homography, by fusion of inertial and visual information. Using the inertial data and without planar ground assumption, a set of virtual horizontal planes are defined. The intersections of these inertial-based virtual planes with the object are registered using the concept of planar homography. Moreover a method to estimate the translation vectors among virtual cameras is proposed, which just needs the relative heights of two 3D points in the scene with respect to one of the cameras and their correspondences on the image planes. Different experimental results for the proposed 3D reconstruction method are provided on two different types of scenarios. In the first type, a single IS-camera couple is used and placed in different locations around the object. In the second type, the 3D reconstruction of a walking person (dynamic case) is performed where a set of installed cameras in a smart-room is used for the data acquisition. Moreover, a set of experiments are simulated to analyse the accuracy of the translation estimation method. The experimental results show the feasibility and effectiveness of the proposed framework for the purpose of multi-layer data registration and volumetric reconstruction.

중심신경망을 이용한 3차원 선소의 군집화에 의한 위성영상의 3차원 건물모델 재구성

본 논문에서는 중심신경망을 이용하여 위성영상으로부터 직사각형 형태의 3차원 건물의 지붕모델을 재구성하는 방법을 연구하였다. 제안된 3차원 지붕모델 재구성 기법의 핵심은 3차원 선소의 군집화에 있다. 이를 위해 한 쌍의 스테레오 영상으로부터 구해진 DEM (Digital Elevation Map) 데이터와 2차원 선소에 의해서 3자원 선소를 발생하였다. 제안된 군집화 과정은 중심신경망을 이용한 방법에 의해 수행되며, 2단계로 구성된다. 첫 번째 단계에서는 선소 추출과정에서 끊어지거나, 중복된 3차원 선소를 건물을 이루는 주된 선소로 군집화하고, 두 번째 단계에서는 건물을 구성하는 주된 선소를 구하기 위해 서로 평행인 선소들의 군으로 군집화를 수행한다. 이 군집화 결과를 최종 클러스터링 과정을 통해 직사각형 형태의 지붕모델로 재구성하게 된다. 제안된 방법이 대전지역의 고해상도 IKONOS 위성영상에 의해 실험되었다. 재구성된 건물모델이 원래 건물의 위치와 형태를 대체로 정확히 반영하여, 본 논문에서 제안된 기법을 고해상도 위성영상에 적용하여 도시지역의 건물모델을 구축하는데 효과적으로 사용될 수 있음이 입증되었다. This paper highlights the reconstruction of the rectilinear type of 3D rooftop model from satellite image data using centroid neural network. The main idea of the proposed 3D reconstruction method is based on the grouping of 3D line segments. 3D lines are extracted by 2D lines and DEM (Digital Elevation Map) data evaluated from a pair of stereo images. Our grouping process consists of two steps. We carry out the first grouping process to group fragmented or duplicated 3D lines into the principal 3D lines, which can be used to construct the rooftop model, and construct the groups of lines that are parallel each other in the second step. From the grouping result, 3D rooftop models are reconstructed by the final clustering process. High-resolution IKONOS images are utilized for the experiments. The experimental result's indicate that the reconstructed building models almost reflect the actual position and shape of buildings in a precise manner, and that the proposed approach can be efficiently applied to building reconstruction problem from high-resolution satellite images of an urban area.

Self-calibration of a biplane X-ray imaging system for an optimal three dimensional reconstruction

The purpose of this paper is to elaborate a three dimensional (3D) reconstruction method, using biplane X-ray angiograms acquired daily by the clinician, without any special calibration procedure during the X-ray examination. The absolute geometry of the X-ray imaging system is determined by an iterative procedure based on the minimization of the mean square distance between observed and analytical projections of a set of reference points identified by the clinician on the simultaneous pair of images. Once the geometry of the imaging system is found the 3D structure of interest is retrieved from classical methods of binocular stereovision. This 3D information is a prerequisite for an accurate evaluation of the degree of severity of a vascular structure or motion anomaly and therefore, for establishing an appropriate diagnosis. The proposed 3D reconstruction method is validated on synthetic and real data and is shown to perform robustly and accurately in the presence of noise. The method should be particularly useful in clinical applications as it needs very little intervention from the clinician.