Shape From Focus Research Articles

Automated analysis of the void structure in hardened concrete based on shape from focus

The quality of the air void system is essential for the frost resistance of concrete. Therefore, it is crucial to assess the quality of the air void system in hardened concrete in an appropriate way. In this study, the shape from focus (SFF) method is applied to identify the focused image of each point on the polished surface of hardened concrete and thereby acquire the depth information of the entire surface. The performance of various focus measure operators and window sizes is evaluated. Thereafter, the selected focus measure operator and window size are applied for the SFF analysis of all samples. Based on the obtained depth map, the voids are identified, and the void parameters of the hardened concrete are determined by an automated procedure. The main advantage of the SFF method is that it is possible to carry out an automated air void analysis without contrast enhancement.

Focus Estimation Methods for Use in Industrial SFF Imaging Systems

A Shape-From-Focus (SFF) is a three-dimensional imaging method based on focus information. It is not yet widely used for in-line industrial inspection or measurement tasks. The main reasons are the time it takes to capture a 3D image of the inspected product and the presence of interference affecting image quality. This paper compares operators for estimating focus in source images in the scope of their use in constructing an industrial 3D scanner. Interferences were introduced by using additional illuminators and changing the acquisition parameters. The use of industrial-grade cameras, industrial-grade illuminators, and electrically controlled optics are discussed. A novel approach in the research is using an electrically tunable lens to move the position of the image plane during 3D image acquisition. The research was conducted on various surfaces found on typical industrial products. The research showed which focus estimation operators can be applied to SFF imaging within the range of interference considered. It was also confirmed that using the centre of gravity method for scene reconstruction allows for an increase in resolution compared to the maximum method.

Enhancing Focus Volume through Perceptual Focus Factor in Shape-from-Focus

Shape From Focus (SFF) reconstructs a scene’s shape using a series of images with varied focus settings. However, the effectiveness of SFF largely depends on the Focus Measure (FM) used, which is prone to noise-induced inaccuracies in focus values. To address these issues, we introduce a perception-influenced factor to refine the traditional Focus Volume (FV) derived from a traditional FM. Owing to the strong relationship between the Difference of Gaussians (DoG) and how the visual system perceives edges in a scene, we apply it to local areas of the image sequence by segmenting the image sequence into non-overlapping blocks. This process yields a new metric, the Perceptual Focus Factor (PFF), which we combine with the traditional FV to obtain an enhanced FV and, ultimately, an enhanced depth map. Intensive experiments are conducted by using fourteen synthetic and six real-world data sets. The performance of the proposed method is evaluated using quantitative measures, such as Root Mean Square Error (RMSE) and correlation. For fourteen synthetic data sets, the average RMSE measure of 6.88 and correction measure of 0.65 are obtained, which are improved through PFF from an RMSE of 7.44 and correlation of 0.56, respectively. Experimental results and comparative analysis demonstrate that the proposed approach outperforms the traditional state-of-the-art FMs in extracting depth maps.

Improved shape-from-focus reconstruction for high dynamic range freedom surface

The 3D reconstruction of freedom surface has been a major challenge for computer vision as well as metrology. Complex surface structures such as large undulations and deep grooves, along with surface reflection properties like high dynamic range (HDR) pose many difficulties for 3D reconstruction. Shape from focus (SFF) is widely studied for its low cost, ease of deployment, and effectively avoiding occlusion. However, its inherent disadvantages, such as inaccuracy and sensitivity to noise, cannot be ignored. This article demonstrates an improved depth map optimization method to solve these problems. Firstly, a more comprehensive and reliable depth recovery evaluation method is proposed. Then an efficient optimization algorithm is applied to the filtered depth maps. Comparable experiments confirm our method's effective suppression of noise and the reliable compensation of missing signals when measuring freedom surfaces.

Shape-from-focus reconstruction using block processing followed by local heat-diffusion-based refinement

Shape from focus (SFF) is a method of recovering the three-dimensional shape of objects from the image sequence through various focusing measurement operators, and is widely used in the fields of medical diagnosis, PCB inspection, industrial manufacturing, and robotic operations. In SFF, the wrong focus measurement causes an inaccurate reconstruction of the object. Generally, to improve the accuracy of focusing measurements, many focus measure (FM) operators are proposed, which is a method that improves the focus volume (FV) performance. However, due to the influence of blur diffusion, which refers to the overlapping of blur circles between different regions, the calculation results of the FM operator in the local window often do not meet the stability requirements of complex scenarios. It is difficult to provide high-precision reconstruction results in practical applications. To overcome this limitation, we proposed an SFF framework based on block processing and heat-diffusion refinement to obtain a refined depth map (RDM). The acquisition of an RDM is mainly divided into two steps. First, the image is divided into blocks to reduce the effect of blur diffusion, and the FM operator is used to calculate the focus measurement value of each block to form the FV. The initial depth map (IDM) is obtained by searching for the best focus position in the FV. The second step is to refine the IDM based on the heat-diffusion principle to obtain the best focus position for all pixels in each block and derive the RDM. The results of experiments conducted on synthetic and real objects demonstrate the superiority of our method in terms of accuracy and its high stability in different application scenarios.

Directional Ring Difference Filter for Robust Shape-from-Focus

In the shape-from-focus (SFF) method, the quality of the 3D shape generated relies heavily on the focus measure operator (FM) used. Unfortunately, most FMs are sensitive to noise and provide inaccurate depth maps. Among recent FMs, the ring difference filter (RDF) has demonstrated excellent robustness against noise and reasonable performance in computing accurate depth maps. However, it also suffers from the response cancellation problem (RCP) encountered in multidimensional kernel-based FMs. To address this issue, we propose an effective and robust FM called the directional ring difference filter (DRDF). In DRDF, the focus quality is computed by aggregating responses of RDF from multiple kernels in different directions. We conducted experiments using synthetic and real image datasets and found that the proposed DRDF method outperforms traditional FMs in terms of noise handling and producing a higher quality 3D shape estimate of the object.

An Improved Shape from Focus Method for Measurement of Three-Dimensional Features of Fuel Nozzles

The precise three-dimensional measurement of fuel nozzles is of great significance to assess the manufacturing accuracy and improve the spray and atomization performance. This paper proposes an improved fast shape from focus (SFF) method for three-dimensional measurement of key features of fuel nozzles. In order to ensure the measurement accuracy and efficiency of the SFF, the dispersion of the measured points from a standard flat plane was used to select the optimal combination of the focus measure operator, window size and sampling step size. In addition, an approximate method for the focus measure interval is proposed to improve the measurement efficiency, which uses the peak region of the central pixel to replace the peak region of other pixels. The results show that the proposed method decreased the average computation time of the focus measure by 79.19% for the cone section and by 38.30% for the swirl slot. Compared with a reference laser scanning microscope, the measurement error in length is within 10 μm and the error in angle is within a maximum 0.15°.

Incorporating structural prior for depth regularization in shape from focus

Depth map reconstructed through passive or active methods usually has noise and exhibits poor shape quality. In the case of shape from focus (SFF), improvement techniques can be divided mainly into two categories: one, which enhances the image focus volume, and second, which tries to refine the depth map. In both of these categories, no additional information about the shape of the object is taken into consideration, and hence, these techniques usually provide little improvement in the depth map. In this paper, we propose to incorporate a structural prior that helps to maintain the structural details in the recovered depth map. For this, we devise variations in the all-in-focus (AIF) image as the structural prior. By exploiting guided filtering, we improve the initial depth map through weighted least squares (WLS) based regularization for which our prior provides efficient weights. Experiments have been conducted on a variety of synthetic and real image sequences, and the results demonstrate that the proposed structural prior improves the accuracy of depth reconstruction.

Thin structures retrieval using anisotropic neighborhoods of superpixels: application to shape-from-focus

Shape-from-focus (SFF) refers to the challenging inverse problem of recovering the scene depth from a given set of focused images using a static camera. Standard approaches model the interactions between neighboring pixels to get a regularized solution. Nevertheless, isotropic regularization is known to introduce undesired artifacts and to remove early thin structures. These structures have a small size in at least one dimension and are more numerous when considering superpixel preprocessing. This paper addresses the improvement of SFF regularization through the estimation of the presence of such structures and the construction of anisotropic neighborhoods sticking along image edges and proposes a flexible formulation over pixels or superpixels. A thoroughly study comparing different strategies for constructing these neighborhoods in terms of accuracy and running time for the targeted application is provided. Notably, experiments performed on a reference dataset show the overall superiority of the approach, e.g. a decrease of the RMSE value by about 20%, and its robustness against generated superpixels.

Shape from focus using gradient of focus measure curve

Finding the best focused position is the key to shape from focus (SFF). Focus measure (FM) operators, sampling step, especially various noises usually cause unstable focused position searching and reduce the accuracy of SFF. Traditional methods usually use specific functions like Gaussian function and polynomial function to fit the focus measure curve and take the position of the peak of the fitting curve as the best focused position. However, due to the complexity of the imaging process and the noise, a fixed function is difficult to describe the focus measure curve. In this paper, a robust SFF method is proposed. In the process of finding the best focused position, the signal-to-noise ratio (SNR) of the values near the peak of the focus measure curve is relatively low, which usually leads to incorrect result. However, the rising and falling parts of the focus measure curve are more robust than the low SNR region, so we calculate the gradient of the focus measure curve with an adaptive derivative step. With the derivative step, we can reduce the influence of the SNR region. Then the zero point of the gradient curve and the derivative step are used to find the best focused position. Without any additional pre-work or constraints, our method can achieve high accuracy and strong robustness simultaneously. The effectiveness of the proposed method is evaluated by the simulated objects and real objects with different shapes. The 3D shape recovery results show that the proposed method is a superior alternative to enhance the focus volume in SFF. The RMSE of simulated objects is reduced by more than 25% when the proposed method is applied. For real objects, the RMSE of our method is 36%, 53% lower, compared with the Gaussian interpolation method and the polynomial fitting method, respectively.

Continuous optical zoom microscope with extended depth of field and 3D reconstruction

Microscope such as fluorescence microscope, confocal microscope and two-photon microscope plays an important role in life science, laser processing and other fields. However, most microscopes only have discrete zoom rates. In this paper, a continuous optical zoom microscope with extended depth of field and 3D reconstruction is demonstrated for the first time. It consists of a zoom objective lens, a microscope holder, an adjustable three-dimensional object stage, an Abbe condenser and an LED light source. The zoom objective lens is composed of several liquid lenses and solid lenses. By adjusting the applied voltage to the liquid lens, the proposed microscope can achieve a large continuous magnification from 10? to 60?. Moreover, an improved shape from focus (SFF) algorithm and image fusion algorithm are designed for 3D reproduction. Based on the liquid lenses, the axial focusing position can be adjusted to obtain images with different depths, and then the extended depth of field and 3D reconstruction can be realized. Our experimental results demonstrate the feasibility of the proposed microscope. The proposed microscope is expected to be applied in the fields of pathological diagnosis, biological detection, etc.

Adaptive weighted guided image filtering for depth enhancement in shape-from-focus

Existing shape from focus (SFF) techniques cannot preserve depth edges and fine structural details from a sequence of multi-focus images. Moreover, noise in the sequence affects the accuracy of the depth map. In this paper, a novel depth enhancement algorithm for the SFF based on an adaptive weighted guided image filtering (AWGIF) is proposed to address the above issues. The AWGIF is applied to decompose an initial depth map estimated by the traditional SFF into base and detail layers. In order to preserve the edges accurately in the refined depth map, the guidance image is constructed from the sequence, and the coefficient of the AWGIF is utilized to suppress the noise while enhancing the fine depth details. Experiments on real and synthetic objects demonstrate the superiority of our algorithm in terms of anti-noise, and the ability to preserve depth edges and fine structural details w.r.t. existing methods.

Energy minimization for image focus volume in shape from focus

In shape from focus (SFF) methods, the quality of depth map is mainly dependent on the accuracy level of image focus volume. Most of the SFF techniques optimize focus volume without incorporating any prior or additional structural information about the scene and thus resultant depth maps are deteriorated. We mitigate this deficiency by proposing to optimize focus volume through energy minimization. The proposed energy function contains smoothness and structural similarity along with data term. Smoothness constraint enforces spatial coherence while structural similarity constraint tries to preserve structures which are consistent with image sequence. This results in an optimized focus volume that imitates the underlying scene accurately. For the implementation of our 3D objective function, we employ an efficient technique that decomposes the problem into a sequence of 1D simple sub-problems. Experiments conducted on synthetic and real image sequences from a variety of datasets demonstrate that the proposed method optimizes the focus volume effectively and thus provides improved depth maps.

Multidirectional focus measure for accurate three‐dimensional shape recovery of microscopic objects

Shape from focus (SFF) is a technique to recover the shape of an object from multiple images taken at various focus settings. Most of conventional SFF techniques compute focus value of a pixel by applying one of focus measure operators on neighboring pixels on the same image frame. However, in the optics with limited depth of field, neighboring pixels of an image have different degree of focus for curved objects, thus the computed focus value does not reflect the accurate focus level of the pixel. Ideally, an accurate focus value of a pixel needs to be measured from the neighboring pixels lying on tangential plane of the pixel in image space. In this article, a tangential plane on each pixel location (i, j) in image sensor is searched by selecting one of five candidate planes based on the assumption that the maximum variance of focus values along the optical axis is achieved from the neighborhood lying on tangential plane of the pixel (i, j). Then, a focus measure operator is applied on neighboring pixels lying on the searched plane. The experimental results on both the synthetic and real microscopic objects show the proposed method produces more accurate three-dimensional shape in comparison to conventional SFF method that applies focus measures on original image planes.

Mathematical Principles of Object 3D Reconstruction by Shape-from-Focus Methods

Shape-from-Focus (SFF) methods have been developed for about twenty years. They able to obtain the shape of 3D objects from a series of partially focused images. The plane to which the microscope or camera is focused intersects the 3D object in a contour line. Due to wave properties of light and due to finite resolution of the output device, the image can be considered as sharp not only on this contour line, but also in a certain interval of height—the zone of sharpness. SSFs are able to identify these focused parts to compose a fully focused 2D image and to reconstruct a 3D profile of the surface to be observed.

Investigation of Shape-from-Focus Precision by Texture Frequency Analysis

This paper reports the precision of shape-from-focus (SFF) imaging according to the texture frequencies and window sizes of a focus measure. SFF is one of various depth measurement techniques for optical imaging, such as microscopy and endoscopy. SFF measures the depth of an object according to focus measure, which is generally computed with a fixed window. The window size affects the performance of SFF and should be adjusted for the texture of an object. In this study, we investigated the precision difference of SFF in texture frequencies and by window size. Two experiments were performed: precision validation in texture frequencies with a fixed window size, and precision validation in various window sizes related to pixel-cycle lengths. The first experimental results showed that a smaller window size could not provide a correct focus measure, and the second results showed that a window size that is approximately equal to a pixel-cycle length of the texture could provide better precision. These findings could potentially contribute to determining the appropriate window size of focus measure operation in shape-from-focus reconstruction.

Roughness measurement of leaf surface based on shape from focus

BackgroundSurface roughness has a significant effect on leaf wettability. Consequently, it influences the efficiency and effectiveness of pesticide application. Therefore, roughness measurement of leaf surface offers support to the relevant research efforts. To characterize surface roughness, the prevailing methods have drawn support from large equipment that often come with high costs and poor portability, which is not suitable for field measurement. Additionally, such equipment may even suffer from inherent drawbacks like the absence of relationship between pixel intensity and corresponding height for scanning electron microscope (SEM).ResultsAn imaging system with variable object distance was created to capture images of plant leaves, and a method based on shape from focus (SFF) was proposed. The given space-variantly blurred images were processed with the proposed algorithm to obtain the surface roughness of plant leaves. The algorithm improves the current SFF method through image alignment, focus distortion correction, and the introduction of NaN values that allows it to be applied for precise 3d-reconstruction and small-scale surface roughness measurement.ConclusionCompared with methods that rely on optical three-dimensional interference microscope, the method proposed in this paper preserves the overall topography of leaf surface, and achieves superior cost performance at the same time. It is clear from experiments on standard gauge blocks that the RMSE of step was approximately 4.44 µm. Furthermore, according to the Friedman/Nemenyi test, the focus measure operator SML was expected to demonstrate the best performance.

Depth Hypotheses Fusion through 3D Weighted Least Squares in Shape from Focus.

In shape-from-focus (SFF) methods, a single focus measure is used to compute the focus volume. However, it seems that a single focus measure operator is not capable of computing accurate focus values for the images of diverse types of object shapes. Furthermore, most of the SFF methods try to improve the depth map without considering any additional structural or prior information. Consequently, the extracted shape of the object might lack important details. In this work, we address these problems and suggest a method in which depth hypotheses are combined for a more accurate 3D shape through 3D weighted least squares. First, depth hypotheses are obtained by applying a number of focus operators. Then, structural prior or guidance volume is extracted from the focus measure volumes. Finally, a 3D weighted least squares optimization technique is applied to the depth hypothesis volume, where weights are computed from the guidance volume. Thus, by inducing structural prior, an improved resultant depth map is obtained. The proposed method was tested using various image sequences of synthetic and microscopic real objects. Experimental results and comparative analysis demonstrated the effectiveness of the proposed method.

Robust Focus Volume Regularization in Shape From Focus.

Shape from focus (SFF) reconstructs 3D shape of the scene from a sequence of multi-focus images, and the quality of reconstructed shape mainly depends on the accuracy of image focus volume (FV). Traditional SFF techniques exhibit poor performance in preserving structural edges and fine details while removing noisy artifacts, and mostly they do not incorporate any additional shape prior. Therefore, in this paper, we propose to refine FV by formulating an energy minimization framework that employs a nonconvex regularizer and incorporates two types of shape priors. The proposed regularizer is robust against noisy focus values. The first proposed shape prior is input image sequence and it is a single and static shape prior. While, the second shape prior corresponds to a series of shape priors. These shape priors are FVs which are iteratively obtained on-the-fly. Both of these shape priors constrain the solution space for output FV. We optimize nonconvex energy function through majorize-minimization algorithm which iteratively guarantees a local minimum and converges quickly. Experiments have been conducted to evaluate accuracy and convergence properties of the proposed method. Experimental results of synthetic and real image sequences demonstrate that our method achieves superior results in terms of ability to reconstruct accurate 3D shapes as compared to existing approaches.

Focus Measurement in Color Space for Shape From Focus Systems

Shape from Focus (SFF) has been studied extensively in computer vision for 3D shape and depth recovery. The first stage in SFF methods is to compute the focus value of every pixel by converting the colored images into gray scale and then apply the focus measure operator. Converting colored values in the images into gray scale values may lead to imprecise mapping of pixels with different colored values onto the same gray scale value, this affects the overall accuracy of the system. In a colored image, the focused pixels maintain a considerable color difference from their neighboring pixels as compared to the defocused ones, which are blended into their neighborhood. This article presents an alternative method to measure the degree of focus by directly processing colored images. The color differences of the neighbor pixels with respect to the central pixel are obtained and summed together, this is followed by calculating their spread. The sum and the spread are combined to measure the degree of focus of the pixel in consideration. The proposed focus measure is then used for shape recovery of various simulated and real objects and is compared with previous techniques. The comparison results show the proposed method has the highest correlation and smallest RMSE values confirming the effectiveness of using color images for shape recovery.