Method For Object Reconstruction Research Articles

RL-NBV: A deep reinforcement learning based next-best-view method for unknown object reconstruction

The Next-Best-View (NBV) algorithm is a key component in autonomous unknown object reconstruction. It iteratively determines the optimal sensor pose to capture the maximum information about the object under reconstruction. However, prevailing deep reinforcement learning (DRL) based NBV algorithms tend to transform point cloud, the raw sensor data, into different representations, thereby obscuring natural invariances of the data. In this work, we propose an innovative DRL-based method, denoted as RL-NBV, to learn NBV policy directly from the raw point cloud data. Specifically, we interpret the observation space as the current state of the reconstructed object represented by point clouds and current view selection states. Experimental results indicate that our method outperforms existing methods in terms of reconstruction performance. Moreover, our method significantly improves efficiency over ray-casting-based algorithms as time-consuming ray casting and data transformation are unnecessary.

3D colored object reconstruction from a single view image through diffusion

In this paper we propose a novel 3D colored object reconstruction method from a single view image. Given a reference image, a conditional diffusion probabilistic model is built to reconstruct both a 3D point cloud shape and the corresponding color features at each point, and then images from arbitrary views can be synthesized using a volume rendering technique. The approach involves several sequential steps. First, the reference RGB image is encoded into separate shape and color latent variables. Then, a shape prediction module predicts reverse geometric noise based on the shape latent variable within the diffusion model. Next, a color prediction module predicts color features for each 3D point using information from the color latent variable. Finally, a volume rendering module transforms the generated colored point cloud into 2D image space, facilitating training based solely on a reference image. Experimental results demonstrate that the proposed method achieves competitive performance on colored 3D shape reconstruction and novel view image synthesis.

Image Quality Enhancement for Digital Breast Tomosynthesis: High-Density Object Artifact Reduction.

Breast cancer has a high incidence and mortality rate among women, early diagnosis is essential as it gives insight regarding the most appropriate therapeutic strategy for each case. Among all imaging diagnostic methods, digital breast tomosynthesis (DBT) is effective for early breast cancer detection. In DBT images, high-density object artifacts are generated when imaging objects with high X-ray absorptivity, which include metal artifacts, ripple artifacts, and deformation artifacts. In this study, we analyze the causes of these artifacts and propose a set of high-density object reconstruction methods based on iterative algorithms. Our method includes a reprojection-based high-density object projection data segmentation algorithm and an iterative reconstruction algorithm based on volume expansion. The experiments on simulation data and the human breast data with artificial surgical needles prove the effectiveness of our method. By using our algorithm, the problem of distorting the shape, size, and position of high-density objects during DBT reconstruction is raised, the influence of these artifacts is reduced, and the quality of the DBT reconstructed image is improved. We hope that our algorithm might contribute to promoting the usage of DBT.

Research on 3D Object Reconstruction Method based on Deep Learning

3D reconstruction is a classic task in the field of computer graphics. More and more researchers try to replicate the success of deep learning in 2D image processing tasks to 3D reconstruction tasks, so 3D reconstruction related research based on deep learning has gradually become a research hotspot. Compared with the traditional 3D reconstruction methods that require precision acquisition equipment and strict calibration of image information, the 3D reconstruction method based on deep learning completes the matching of 2D images to 3D models through deep neural networks, and can reconstruct 3D models of various categories of objects from RGB images obtained by ordinary acquisition equipment in a large number and quickly. This paper introduces the state of the art of 3D voxel reconstruction, 3D points cloud reconstruction and 3D mesh reconstruction, respectively. According to the different representation methods of 3D objects, the 3D object reconstruction methods based on deep learning are classified and reviewed, the characteristics and shortcomings of existing methods are analyzed, and three important research trends are summarized.

Component-Aware High-Resolution 3D Object Reconstruction

<p indent="0mm">The computational cost of the voxel-based 3D object reconstruction grows cubically with the increase of the resolution. To address this problem, the proposed component-aware 3D object reconstruction method decomposes a 3D object into several components to reconstruct the 3D object by predicting and assembling a series of components, which transforms the high-resolution 3D object reconstruction into a series of low-resolution component reconstruction. The proposed method predicts the positions of all components using a component position prediction module. Then the geometric and appearance feature of a component are fused into a joint feature with a component feature extraction module. The joint feature is utilized by a component shape reconstruction module to predict the geometry of components. Finally, all components are assembled into a high-resolution 3D object with the guidance of their positions. Experiments are performed on ShapeNet dataset using an NVIDIA 1080 Maxwell GPU with 12 GB of memory. The comparison methods include an octree-based high-resolution reconstruction method, a LSTM-based low-resolution reconstruction method and a baseline method using encoder-decoder architecture. The results of high-resolution reconstruction experiment demonstrate that the component-aware 3D reconstruction method achieves a satisfactory 3D reconstruction accuracy with a low computational cost. In the low-resolution reconstruction experiment, the proposed method also performs better and the average accuracy in 13 categories reaches 0.618.

3D-RVP: A method for 3D object reconstruction from a single depth view using voxel and point

Three-dimensional object reconstruction technology has a wide range of applications such as augment reality, virtual reality, industrial manufacturing and intelligent robotics. Although deep learning-based 3D object reconstruction technology has developed rapidly in recent years, there remain important problems to be solved. One of them is that the resolution of reconstructed 3D models is hard to improve because of the limitation of memory and computational efficiency when deployed on resource-limited devices. In this paper, we propose 3D-RVP to reconstruct a complete and accurate 3D geometry from a single depth view, where R, V and P represent Reconstruction, Voxel and Point, respectively. It is a novel two-stage method that combines a 3D encoder-decoder network with a point prediction network. In the first stage, we propose a 3D encoder-decoder network with residual learning to output coarse prediction results. In the second stage, we propose an iterative subdivision algorithm to predict the labels of adaptively selected points. The proposed method can output high-resolution 3D models by increasing a small number of parameters. Experiments are conducted on widely used benchmarks of a ShapeNet dataset in which four categories of models are selected to test the performance of neural networks. Experimental results show that our proposed method outperforms the state-of-the-arts, and achieves about 2.7% improvement in terms of the intersection-over-union metric.

Sharp Feature Detection as a Useful Tool in Smart Manufacturing

Industry 4.0 comprises a wide spectrum of developmental processes within the management of manufacturing and chain production. Presently, there is a huge effort to automate manufacturing and have automatic control of the production. This intention leads to the increased need for high-quality methods for digitization and object reconstruction, especially in the area of reverse engineering. Commonly used scanning software based on well-known algorithms can correctly process smooth objects. Nevertheless, they are usually not applicable for complex-shaped models with sharp features. The number of the points on the edges is extremely limited due to the principle of laser scanning and sometimes also low scanning resolution. Therefore, a correct edge reconstruction problem occurs. The same problem appears in many other laser scanning applications, i.e., in the representation of the buildings from airborne laser scans for 3D city models. We focus on a method for preservation and reconstruction of sharp features. We provide a detailed description of all three key steps: point cloud segmentation, edge detection, and correct B-spline edge representation. The feature detection algorithm is based on the conventional region-growing method and we derive the optimal input value of curvature threshold using logarithmic least square regression. Subsequent edge representation stands on the iterative algorithm of B-spline approximation where we compute the weighted asymmetric error using the golden ratio. The series of examples indicates that our method gives better or comparable results to other methods.

3D Object Reconstruction from Imperfect Depth Data Using Extended YOLOv3 Network.

State-of-the-art intelligent versatile applications provoke the usage of full 3D, depth-based streams, especially in the scenarios of intelligent remote control and communications, where virtual and augmented reality will soon become outdated and are forecasted to be replaced by point cloud streams providing explorable 3D environments of communication and industrial data. One of the most novel approaches employed in modern object reconstruction methods is to use a priori knowledge of the objects that are being reconstructed. Our approach is different as we strive to reconstruct a 3D object within much more difficult scenarios of limited data availability. Data stream is often limited by insufficient depth camera coverage and, as a result, the objects are occluded and data is lost. Our proposed hybrid artificial neural network modifications have improved the reconstruction results by 8.53% which allows us for much more precise filling of occluded object sides and reduction of noise during the process. Furthermore, the addition of object segmentation masks and the individual object instance classification is a leap forward towards a general-purpose scene reconstruction as opposed to a single object reconstruction task due to the ability to mask out overlapping object instances and using only masked object area in the reconstruction process.

High-Quality Object Reconstruction From One-Dimensional Compressed Encrypted Signal Based on Multi-Network Mixed Learning

Conventional optical image encryption methods based on phase-shifting interferometry need at least two interferograms, and the storage or transmission of interferograms needs to occupy a lot of resources. At the same time, the low quality of reconstructed complex natural images has always been a main limiting factor in the application of optical image security. In this paper, a high-quality object reconstruction method from one-dimensional compressed encrypted signal based on multi-network mixed learning is proposed. First, an encrypted interferogram can be obtained using the double random phase encoding (DRPE) method. Then, we can obtain the one-dimensional compressed sampling signal of the encrypted hologram on the photodiode using single-pixel compressive holographic imaging method. Finally, the mapping of 1D signal to 2D object image can be learned utilizing multiple neural network models. Numerical simulation results show that the complex natural images can be reconstructed using the proposed method with high quality at lower sampling ratio.

Fall detection for multiple pedestrians using a PCA approach to 3-D inclination

Aging populations in developing countries have driven increased research interest in issues related to “eldercare,” with a particular focus on early fall detection. This article describes an approach to processing images captured using a depth camera to predict the probable inclination of an imminent fall by a pedestrian. First, the 3-D object reconstruction method is explained and realized. Then, the 3-D object inclination analysis is given. Experimental results show prediction rates achieved accuracy up to 98.11%, making this approach promising for use in applications in hospital and home environments in the near future.

Investigation of Morphological Features of Autophagy During Plant Programmed Cell Death.

The investigation of autophagy particularly when observed during programmed cell death (PCD) is strongly based on the morphological features recorded with transmission electron microscope (TEM). Here we describe methods to induce and to inhibit autophagy in plants. Also some tips for obtaining better preservation of biological membranes, crucial for the investigation of autophagy, are provided together with information about plant autophagic mutants, use of antibodies and methods for 3D reconstruction of large membrane-bound objects that are commonly formed during autophagic processes leading to PCD in plants.

An accurate method for 3D object reconstruction from unordered sparse views

3D object reconstruction from depth scans has been long investigated problem, but complete solutions are still missing. Recent advances of consumer depth sensors resulted in a number of excellent methods for 3D object reconstruction, with the most impressive results obtained by KinectFusion and Visual Odometry. Both methods reconstruct an object from a dense set of RGB-D video frames, with the important restrictions that the object has to be static and camera transitions has to be smooth during the reconstruction. As an alternative, we propose a method which is more accurate, while being able to reconstruct the object from a sparse set of unordered depth scans. Our method starts by pairwise view matching which relies on robust point pair features. It is followed by the efficient graph-based view consistency check to produce an initial 3D model reconstruction. To improve the accuracy, we perform a simultaneous optimization of the camera positions and the object model, using depth and color information. A quantitative evaluation demonstrates that we can reconstruct objects with equal or better accuracy than the KinectFusion and Visual Odometry, but using significantly lower number of unordered input scans.

关联成像目标重构的伪逆迭代降噪方法

关联成像目标重构的伪逆迭代降噪方法

Comparison of holographic and iterative methods for amplitude object reconstruction

Comparison of holographic and iterative methods for amplitude object reconstruction

Parallelization of a method for dense 3D object reconstruction in structured light scanning

Optical three-dimensional shape measurement based on structured light has been widely used for 3D measurement in many different applications. Although many different methods for 3D object reconstruction have been proposed last years, surprisingly none of the proposals includes a parallelization study of the tasks executed on computers, preventing these methods from reaching their maximum performance. In this paper, we propose first the computational evaluation of a previously proposed 3D object reconstruction method. Based on that evaluation, we also propose the parallelization of the reconstruction method using the OpenMP API specification for shared-memory parallel programming. The results show that most of the execution time is consumed by the tasks depending on the I/O hardware (camera, projector, hard disk, etc.), in such a way that the tasks performed by the computer should be overlapped as much as possible with the tasks performed by the I/O hardware, for those scenarios where the acquisition of the images should be performed on-line. For those scenarios where all the images are already available, then the inherent parallelism of the application increases, allowing a reduction of the execution time that can reach up to a 82 %. These results validate the proposed parallelization as a valuable implementation for data centers that provide web services for 3D object reconstruction purposes.

Object reconstitution using pseudo-inverse for ghost imaging.

We propose a novel method for object reconstruction of ghost imaging based on Pseudo-Inverse, where the original objects are reconstructed by computing the pseudo-inverse of the matrix constituted by the row vectors of each speckle field. We conduct reconstructions for binary images and gray-scale images. With equal number of measurements, our method presents a satisfying performance on enhancing Peak Signal to Noise Ratio (PSNR) and reducing computing time. Being compared with the other existing methods, its PSNR distinctly exceeds that of the traditional Ghost Imaging (GI) and Differential Ghost Imaging (DGI). In comparison with the Compressive-sensing Ghost Imaging (CGI), the computing time is substantially shortened, and in regard to PSNR our method exceeds CGI on grayscale images and performs as well as CGI visually on binary images. The influence of both the detection noise and the accuracy of measurement matrix on PSNR are also presented.

유비쿼터스 환경에서 구조광 기반 사용자 친화적 3차원 객체 재구성 기법

3차원 공간에서 물체를 재구성하기 위하여 제안된 기존 기법들은 여러 개의 카메라를 사용하거나 여러장의 사진을 촬영하여 물체를 재구성하기 때문에 특정한 하드웨어를 필요로 하거나 촬영 장소에 민감하고 시간이 오래 걸린다는 단점이 있다. 본 논문에서는 유비쿼터스 환경에서 구조광 기법을 이용하여 한 장의 사진의 정보로부터 3차원 물체를 재구성하는 방법을 제안한다. 구조광 기법 적용을 위한 칼라 패턴은 기존연구에서 제안된 것을 사용한다. 본 논문에서는 실제로 객체 재구성 성능에 매우 중요한 줄무늬 패턴 추출과 줄무늬 패턴 매칭에 다양한 영상 처리 기법들로 구성된 파이프라인을 새롭게 제안하고, 패턴 매칭을 위한 최적의 비용 함수를 제안한다. 제안 기법을 통하여 갤럭시 빔과 같은 초소형 프로젝터를 탑재한 스마트폰과 같은 유비쿼터스 혹은 모바일 환경에서 적은 계산 복잡도와 간단한 환경 설정으로 3차원 물체를 재구성하는 것이 가능하다. Since conventional methods for the reconstruction of 3D objects used a number of cameras or pictures, they required specific hardwares or they were sensitive to the photography environment with a lot of processing time. In this paper, we propose a 3D object reconstruction method using one photograph based on structured light in ubiquitous environments. We use color pattern of the conventional method for structured light. In this paper, we propose a novel pipeline consisting of various image processing techniques for line pattern extraction and matching, which are very important for the performance of the object reconstruction. And we propose the optimal cost function for the pattern matching. Using our method, it is possible to reconstruct a 3D object with efficient computation and easy setting in ubiquitous or mobile environments, for example, a smartphone with a subminiature projector like Galaxy Beam.

Performance evaluation of a multi-image 3D reconstruction software on a low-feature artefact

Nowadays, multi-image 3D reconstruction is an active research field and a number of commercial and free software tools have been already made available to the public. These provide methods for the 3D reconstruction of real world objects by matching feature points and retrieving depth information from a set of unordered digital images. This is achieved by exploiting computer vision algorithms such as Structure-From-Motion (SFM) and Dense Multi-View 3D Reconstruction (DMVR). In this work, we evaluate the performance of a low-cost commercial SFM–DMVR software by digitising a Cycladic woman figurine. Although the surface properties of the specific artefact are considered 3D laser scanner friendly, its almost featureless white-grey surface composes a challenging digitisation candidate for image based methodologies as no strong feature points are available. We quantify the quality of the 3D data produced by the SFM–DMVR software in relation to the data produced by a high accuracy 3D laser scanner in terms of surface deviation and topological errors. We question the applicability and efficiency of two digitisation pipelines (SFM–DMVR and laser scanner) in relation to hardware requirements, background knowledge and man-hours. This is achieved by producing a complete 3D digital replica of the Cycladic artefact by following both pipelines.

A search-classify approach for cluttered indoor scene understanding

We present an algorithm for recognition and reconstruction of scanned 3D indoor scenes. 3D indoor reconstruction is particularly challenging due to object interferences, occlusions and overlapping which yield incomplete yet very complex scene arrangements. Since it is hard to assemble scanned segments into complete models, traditional methods for object recognition and reconstruction would be inefficient. We present a search-classify approach which interleaves segmentation and classification in an iterative manner. Using a robust classifier we traverse the scene and gradually propagate classification information. We reinforce classification by a template fitting step which yields a scene reconstruction. We deform-to-fit templates to classified objects to resolve classification ambiguities. The resulting reconstruction is an approximation which captures the general scene arrangement. Our results demonstrate successful classification and reconstruction of cluttered indoor scenes, captured in just few minutes.

IMAGO Visualization System: An Interactive Web-Based 3D Visualization System for Cultural Heritage Applications

Due to the evolution of technologies and methods for realistic 3D reconstruction of objects, in many projects it can be found efficient ways to make research results in digital preservation available on the Internet. 3D visualization of cultural heritage is highlighted in this scenario, helping to expand research activities in this field by providing proper tools to allow for example, remote access to historical artifacts. Thus, visualization systems must be able to handle important aspects in the context of digital preservation, such as user profiles, security and ease access to 3D models. This paper presents the development of an effective web-based 3D visualization system whose architecture offers an easy and fast interactivity with 3D models even when limited computer resources are available. The system has been successfully adopted in developing of 3D Virtual Museums in the Universidade Federal do Parana (UFPR) in Brazil, providing an important tool to promote research, educational, social and cultural activities.