Three-dimensional Shape Recovery Research Articles

Spatial coding strategy for dual-frequency phase-shifting profilometry

Dual-frequency fringe projection techniques have been broadly utilized for three-dimensional (3D) shape recovery. However, a pervasive challenge across these methodologies lies in the precise implementation of high-speed measurements. This paper explores a spatial coding strategy designed for the dual-frequency phase unwrapping method to tackle this. The strategy mandates the utilization of only four patterns, encapsulating the phase information of both high and low frequencies. The codewords are strategically embedded within the low-frequency phase, facilitating the acquisition of an accurate absolute phase map. A robust decoding algorithm engineered to ensure dependable phase unwrapping, particularly in scenarios characterized by a substantial number of high-frequency cycles. The proposed method exhibits superior noise immunity and enhances accuracy when compared with the previous dual-frequency phase-shifting methodology. Simulations and experiments demonstrate the robustness and efficiency of the proposed strategy.

Jitter noise modeling and its removal using recursive least squares in shape from focus systems

Three-dimensional shape recovery from the set of 2D images has many applications in computer vision and related fields. Passive techniques of 3D shape recovery utilize a single view point and one of these techniques is Shape from Focus or SFF. In SFF systems, a stack of images is taken with a single camera by manipulating its focus settings. During the image acquisition, the inter-frame distance or the sampling step size is predetermined and assumed constant. However, in a practical situation, this step size cannot remain constant due to mechanical vibrations of the translational stage, causing jitter. This jitter produces Jitter noise in the resulting focus curves. Jitter noise is invisible in every image, because all images in the stack are exposed to the same error in focus; thus, limiting the use of traditional noise removal techniques. This manuscript formulates a model of Jitter noise based on Quadratic function and the Taylor series. The proposed method, then, solves the jittering problem for SFF systems through recursive least squares (RLS) filtering. Different noise levels were considered during experiments performed on both real as well as simulated objects. A new metric measure is also proposed, referred to as depth distortion (DD), which calculates the number of pixels contributing to the RMSE in percentage. The proposed measure is used along with the RMSE and correlation, to compute and test the reconstructed shape quality. The results confirm the effectiveness of the proposed scheme.

A simplified approach using deep neural network for fast and accurate shape from focus.

Three-dimensional shape recovery is an important issue in the field of computer vision. Shape from Focus (SFF) is one of the passive techniques that uses focus information to estimate the three-dimensional shape of an object in the scene. Images are taken at multiple positions along the optical axis of the imaging device and are stored in a stack. In order to reconstruct the three dimensional shape of the object, the best-focused positions are acquired by maximizing the focus curves obtained via application of a focus measure operator. In this article, Deep Neural Network (DNN) is employed to extract the more accurate depth of each object point in the image stack. The size of each image in the stack is first reduced and then provided to the proposed DNN network to aggregate the shape. The initial shape is refined by applying a median filter, and later the reconstructed shape is sized back to original by utilizing bi-linear interpolation. The results are compared with commonly used focus measure operators by employing root mean squared error (RMSE), correlation, and image quality index (Q). Compared to other methods, the proposed SFF method using DNN shows higher precision and low computational time consumption.

Reflectance and Shape Estimation with a Light Field Camera Under Natural Illumination

Reflectance and shape are two important components in visually perceiving the real world. Inferring the reflectance and shape of an object through cameras is a fundamental research topic in the field of computer vision. While three-dimensional shape recovery is pervasive with varieties of approaches and practical applications, reflectance recovery has only emerged recently. Reflectance recovery is a challenging task that is usually conducted in controlled environments, such as a laboratory environment with a special apparatus. However, it is desirable that the reflectance be recovered in the field with a handy camera so that reflectance can be jointly recovered with the shape. To that end, we present a solution that simultaneously recovers the reflectance and shape (i.e., dense depth and normal maps) of an object under natural illumination with commercially available handy cameras. We employ a light field camera to capture one light field image of the object, and a 360-degree camera to capture the illumination. The proposed method provides positive results in both simulation and real-world experiments.

3차원 형상 복원을 위한 수학적 모폴로지 기반의 초점 측도 기법

Shape from focus (SFF) 기법은 카메라 렌즈를 다양한 초점 거리로 놓고 촬영한 영상을 이용해 물체의 3차원 정보를 추출하는 방법이다. 이 논문에서는 미소 객체(microscopic object)의 3차원 깊이 정보를 추출하기 위해 수학적 모폴로지의 기울기 연산자를 이용하는 새로운 SFF방법을 제안한다. 전통적으로 SFF 기법에서는 초점의 품질을 측정하기 위해 하나의 초점 측도(focus measure)를 사용한다. 그러나 미소 객체의 복잡한 형태와 텍스쳐 특성에 따라 하나의 초점 측도만을 사용하는 것은 충분하지가 않은데, 본 논문에서는 향상된 초점 측도를 위해 다수의 형태소(multi-structuring elements)를 사용하는 모폴로지 연산자를 사용하는 방법을 제안한다. 최종적으로 모든 초점 측도 결과를 통합하여 최적의 깊이 맵을 계산하게 된다. 실험을 통해 제안된 알고리즘이 기존의 방법들에 비해 3차원 형상 복원 측면에서 더 정밀한 깊이 맵을 제공하는 것을 알 수 있었다. Shape from focus (SFF) is a technique used to reconstruct 3D shape of objects from a sequence of images obtained at different focus settings of the lens. In this paper, a new shape from focus method for 3D reconstruction of microscopic objects is described, which is based on gradient operator in Mathematical Morphology. Conventionally, in SFF methods, a single focus measure is used for measuring the focus quality. Due to the complex shape and texture of microscopic objects, single measure based operators are not sufficient, so we propose morphological operators with multi-structuring elements for computing the focus values. Finally, an optimal focus measure is obtained by combining the response of all focus measures. The experimental results showed that the proposed algorithm has provided more accurate depth maps than the existing methods in terms of three-dimensional shape recovery.

Robust focus measure operator using adaptive log-polar mapping for three-dimensional shape recovery.

Shape from focus (SFF) is a passive optical technique that reconstructs object shape from a sequence of image taken at different focus levels. In SFF techniques, computing focus measurement for each pixel in the image sequence, through a focus measure operator, is the fundamental step. Commonly used focus measure operators compute focus quality in Cartesian space and suffer from erroneous focus quality and lack in robustness. Thus, they provide erroneous depth maps. In this paper, we introduce a new focus measure operator that computes focus quality in log-polar transform (LPT) Properties of LPT, such as biological inspiration, data selection, and edge invariance, enable computation of better focus quality in the presence of noise. Moreover, instead of using a fixed patch of the image, we suggest the use of an adaptive window. The focus quality is assessed by computing variation in LPT. The effectiveness of the proposed technique is evaluated by conducting experiments using image sequences of different simulated and real objects. The comparative analysis shows that the proposed method is robust and effective in the presence of various types of noise.

A Fuzzy-Neural approach for estimation of depth map using focus

Depth map is used for recovery of three-dimensional structure of the object which is required in many high level vision applications. In this paper, we present a new algorithm for the estimation of depth map for three-dimensional shape recovery. This algorithm is based on Fuzzy-Neural approach using shape from focus (SFF). A Fuzzy Inference System (FIS) is designed for the calculation of the depth map and an initial set of membership functions and fuzzy rules are proposed. Then Neural Network is used to train the FIS. The training is done using back propagation algorithm in combination with the least squares method. Hence, a new set of input membership functions are generated while discarding the initial ones. Lastly, the trained FIS is used to obtain final depth map. The results are compared with five other methods including the traditional SFF method and the Focused Image Surface SFF method (FISM). Six different types of objects are used for testing the proposed algorithm.

Focus measure based on the energy of high-frequency components in the S transform

Focus measure plays a fundamental role in the shape from focus technique. In this Letter, we suggest a focus measure in the S-transform domain that is based on the energy of high-frequency components. A localized spectrum by using variable window size provides a more accurate method of measuring image sharpness as compared to other focus measures proposed in spectral domains. An optimal focus measure is obtained by selecting an appropriate frequency-dependent window width. The performance of the proposed focus measure is compared with those of existing focus measures in terms of three-dimensional shape recovery. Experimental results demonstrate the effectiveness of the proposed focus measure.

Optimization technique for three-dimensional shape recovery from image focus

The problem of 3-D shape recovery from image focus can be described as the problem of determining the shape of the focused image surface (FIS)—the surface formed by the best focused points. The shape from focus (SFF) methods in the literature are fast but inaccurate because of the piecewise constant approximation of FIS. The SFF method based on FIS has shown better results by exhaustive search of FIS shape using a planar surface approximation at the cost of a considerably higher number of computations. We present a method to search FIS shape as an optimization problem, i.e., maximization of focus measure in the 3-D image volume. Each image frame in the image volume (sequence) is divided into subimage frames, and the whole image volume is divided into a number of subimage volumes. A rough depth map at only the central pixel of each subimage frame is determined using one of the traditional SFF methods. A few image frames around the image frame, whose image number in the image volume is obtained from the rough depth at the central pixel of subimage frame, are selected for the subimage volumes. The search of FIS shape is now performed in the subimage volumes using a dynamic programming optimization technique. The final depth map is obtained by collecting the depth map of the subimage volumes. The new algorithm considerably decreases the computational complexity by searching FIS shape in subimage volumes and shows better results.

A novel algorithm for estimation of depth map using image focus for 3D shape recovery in the presence of noise

Three-dimensional shape recovery from one or multiple observations is a challenging problem of computer vision. In this paper, we present a new Focus Measure for the estimation of a depth map using image focus. This depth map can subsequently be used in techniques and algorithms leading to the recovery of a three-dimensional structure of the object, a requirement of a number of high level vision applications. The proposed Focus Measure has shown robustness in the presence of noise as compared to the earlier Focus Measures. This new Focus Measure is based on an optical transfer function implemented in the Fourier domain. The results of the proposed Focus Measure have shown drastic improvements in estimation of a depth map, with respect to the earlier Focus Measures, in the presence of various types of noise including Gaussian, Shot, and Speckle noises. The results of a range of Focus Measures are compared using root mean square error and correlation metric measures.

Surface Deformation Models for Nonrigid 3D Shape Recovery

Three-dimensional detection and shape recovery of a nonrigid surface from video sequences require deformation models to effectively take advantage of potentially noisy image data. Here, we introduce an approach to creating such models for deformable 3D surfaces. We exploit the fact that the shape of an inextensible triangulated mesh can be parameterized in terms of a small subset of the angles between its facets. We use this set of angles to create a representative set of potential shapes, which we feed to a simple dimensionality reduction technique to produce low-dimensional 3D deformation models. We show that these models can be used to accurately model a wide range of deforming 3D surfaces from video sequences acquired under realistic conditions.

Research and development of fringe projection-based methods in 3D shape reconstruction

This paper discusses current research and development of fringe projection-based techniques. A system based on Fourier transform profilometry (FTP) is proposed for three-dimensional (3D) shape recovery. The system improves the method of phase unwrapping to gain accurate 3D shapes of objects. The method uses a region-growing algorithm for the path prediction guided by the quality map to increase the recovering accuracy and provides a fast and simple tool for 3D shape recovery. The shape measurement and data recovery are integrated to offer a new method of 3D modelling. Examples are presented to verify the feasibility of the proposed method.

A heuristic approach for finding best focused shape

The most popular shape from focus (SFF) methods in the literature are based on the concept of focused image surface (FIS)-the surface formed by the best focus points. According to paraxial-geometric optics, there is one-to-one correspondence between the shape of an object and the shape of its FIS. Therefore, the problem of three-dimensional (3-D) shape recovery from image focus can be described as the problem of determining the shape of the FIS. The conventional SFF method is inaccurate because of piecewise constant approximation of the FIS. The SFF method based on the FIS has shown better results by exhaustive search of the FIS shape using planar surface approximation at the cost of considerably higher computations. In this paper, search of the FIS shape is presented as an optimization problem, i.e., maximization of the focus measure in the 3-D image volume. The proposed method searches the optimal focus measure in the whole image volume, instead of the small volume as adopted in previous methods. The dynamic programming, instead of the approximation techniques, is used to search the optimal FIS shape. A direct application of dynamic programming on a 3-D data is impractical, because of higher computational complexity. Therefore a fast heuristic model based on dynamic programming is proposed for the search of FIS shape. The shape recovery results of the new method are better than previous methods. The proposed algorithm is significantly faster than the FIS algorithm, but a little slower than the conventional algorithm.

3D shape-contingent processing of luminance gratings

Studies on binocular contrast sensitivity have predominantly focused on flat, two-dimensional (2D) gratings. The underlying hypothesis of such studies is that contrast sensitivity is determined at the early stages of visual processing and is not influenced by the process of three-dimensional (3D) shape recovery. However, it can be argued that contrast detection involves identifying changes in albedo of a 3D surface rather than strictly determining the presence of 2D luminance changes. In support of this hypothesis, in three experiments we found that the relative salience of oriented luminance modulations was affected by the disparity content of the stimuli, when the same luminance distribution was assigned to a flat surface or to a surface modulated in depth. In the first experiment, in particular, we found that the relative salience of an oriented luminance grating decreased when it could be interpreted as the shading produced by a Lambertian illumination, rather than a change in reflectance. In the other two experiments, moreover, we found that this effect was reduced when the frequency of the luminance modulation did not match the frequency of the stereo corrugation. These results are consistent with the idea that the appearance of a luminance distribution is affected by the perceived 3D properties of the surface on which the luminance distribution is located.

Three-dimensional shape recovery across two views using approximate geometric constraints

We propose a new approach to object matching and shape reconstruction across two views. The approach relies on the topological relationship among the feature points and a planarity approximation for the object surfaces in carrying the object from the model view to the scene view. The correspondences are then refined by fitting an active contour model to the transferred feature points on the scene view, which automatically assumes the shape of the object on correct matching. For the purpose of 3-D reconstruction, it could be shown that while the establishment of such correct correspondences alone would result in ambiguities under the assumption of weak perspective projection, the adoption of a regularization criterion, in the form of a depth-continuity constraint, and a set of geometric constraints associated with selected subset of surfaces and contours of the object, would overcome this problem and lead to the recovery of the approximate 3-D structure of the object.

Three-dimensional shape recovery from the focused-image surface

Three-dimensional shape recovery from the focused-image surface

Recovery of three-dimensional shapes by using defocused structured light

We present a new concept for three-dimensional shape recovery using defocused structured light (DSL) images. The DSL technique externally extracts the depth information from the scene by using projections of cylindrical wavefronts on the object. These projections show different degrees of defocus as a function of the depth. Efficient algorithms for shape recovery are developed taking advantage of the spatial regularity of the projected light and the profile of each fringe. In this way a depth map of the scene is obtained using only one image. Experimental results are shown and we discuss the possibility of a real time implementation.

Shape recovery of specular objects from multiple lights

This paper describes the method of shape recovery of specular objects illuminated by multiple light sources by pattern recognition. In this method, a three-dimensional shape is recovered from an epipolar plane image (EPI) of a rotating image. The highlight reflected from an object has a traveling characteristic corresponding to the surface profile of the object. The highlight appears in EPI as image speed. Therefore, the three-dimensional shape can be quantitatively recovered by tracing the locus of the highlight on EPI if the relationship among the shape of the object, light source, direction of view, and highlight is equated. In this study, the relationship among the object shape under illumination from multiple light sources, direction of view, and highlight is equated in order to quantitatively recover the cross-sectional view of a columnar object. In addition, three-dimensional shape recovery of a general object is accomplished by introducing the light source model of a cone-shaped illumination plane. Finally, in order to show the validity of this method, the experimental result on the recovery of specular objects and its evaluation will be described.

Accurate recovery of three-dimensional shape from image focus

A new shape-from-focus method is described which is based on a new concept, named focused image surface (FIS). FIS of an object is defined as the surface formed by the set of points at which the object points are focused by a camera lens. According to paraxial-geometric optics, there is a one-to-one correspondence between the shape of an object and the shape of its FIS. Therefore, the problem of shape recovery can be posed as the problem of determining the shape of the FIS. From the shape of FIS the shape of the object is easily obtained. In this paper the shape of the FIS is determined by searching for a shape which maximizes a focus measure. In contrast to previous literature where the focus measure is computed over the planar image detector of the camera, here the focus measure is computed over the FIS. This results in more accurate shape recovery than the traditional methods. Also, using FIS, a more accurate focused image can be reconstructed from a sequence of images than is possible with traditional methods. The new method has been implemented on an actual camera system, and the results of shape recovery and focused image reconstruction are presented.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

SHAPE RECOVERY AND ERROR CORRECTION BASED ON HYPOTHETICAL CONSTRAINTS BY PARALLEL NETWORK FOR ENERGY MINIMIZATION

Three-dimensional (3D) shape recovery from a monocular image is the inverse process of optical projection and necessitates the use of additional constraints on 3D features of the objects. We have proposed the method of shape recovery based on some hypothetical constraints and its realization by energy minimization using a parallel network. In this paper, we will discuss an extension of our proposal with the aim of correcting noise and errors in an input image through the process of shape recovery. It is hardly possible to extract edges and vertexes free from noise and errors from a real image. For the robustness of the system, we must provide our system with the ability to correct such noise and errors. In this extension, we can also modulate the inclination of the process toward model-driven or data-driven.