Pixel Projection Research Articles

Simultaneous Projection and Positioning of Laser Projector Pixels.

This paper presents a novel projected pixel localization principle for online geometric registration in dynamic projection mapping applications. We propose applying a time measurement of a laser projector raster-scanning beam using a photosensor to estimate its position while the projector displays meaningful visual information to human observers. Based on this principle, we develop two types of position estimation techniques. One estimates the position of a projected beam when it directly illuminates a photosensor. The other localizes a beam by measuring the reflection from a retro-reflective marker with the photosensor placed in the optical path of the projector. We conduct system evaluations using prototypes to validate this method as well as to confirm the applicability of our principle. In addition, we discuss the technical limitations of the prototypes based on the evaluation results. Finally, we build several dynamic projection mapping applications to demonstrate the feasibility of our principle.

Sub-pixel projector calibration method for fringe projection profilometry.

Digital projectors are used as standard parts at present in fringe projection profilometry systems to project structured-light patterns onto the object surface to be measured, and the distortion of the projector lens must be calibrated and compensated accurately to satisfy the accuracy requirement of industrial applications. A novel method is proposed to determine the projector pixel coordinates of the marker points of a calibration target accurately in terms of projective transform. With the method, the projector can be calibrated with accuracy of sub-pixel level. The method is applicable for the calibration target with a chessboard pattern or a circle pattern, and the calibration result is independent on the results of camera calibration. Experimental results are shown to demonstrate the effectiveness and validity of the proposed method.

Efficient Computerized-Tomography Reconstruction Using Low-Cost FPGA-DSP Chip

In this paper, filtered back-projection algorithm is optimally implemented using low-cost Spartan 3A-DSP 3400 chip. The optimization enables parallel implementation. The combination of the pixel parallelism and projection parallelism is presented to significantly reduce the total reconstruction time to produce the image. The applied data is presented in fixed point format to achieve efficient implementation with maximum speed. The selection of data bus-width is optimized with very little error and good visual quality required for medical images. Before implementation, the computer tomography ( CT ) reconstruction simulator is developed to provide a testing reference for the hardware implementation. Using the combination of the pixel parallelism and projection parallelism, the presented hardware design achieves image reconstruction of a 512-by-512 pixel image from 1024 projections in 134. 8ms using 50 MHz clock cycles. It achieves the reduction of the required number of clock cycles to form an image from projections by 60 % comparing to the state of the art of the reconstruction time using field programmable gate array ( FPGA ) design.

The automatic interpretation of structural plane parameters in borehole camera images from drilling engineering

Digital panoramic borehole camera technology has been widely employed in actual projects, and a large number of high-accuracy borehole camera images have been obtained. Borehole camera images accurately record the geological information, especially the feature parameters of structure plane. However, since the acquisition of these features is usually done by hand, the workload is large and the results can be affected by human error. To solve this problem, this paper presents an automatic interpretation method of structure plane parameters in borehole camera image. In this method, image pixel gray, gradient values and projection method are employed to distinguish the occurrence region of structural planes. Then, standard sine function matching method is employed to search the structure plane in the region. Lastly, the optimal sine curve is screened out and adopted as the feature curve of structure plane. Related parameters of sine curve are analyzed and converted into the parameters of structure plane, such as the central position, orientation, dip angle and fracture width of structural planes. This method can automatically identify structure planes in the image continuously and quickly, and obtain the corresponding structural parameters. The method is stable and reliable, and greatly improves the working efficiency. It can perform the automatic interpretation of structure planes and extraction of geometric parameters, and provide an effective and consistent solution for drilling information acquisition and borehole camera image signal processing.

An optimisation design of adaptive illumination for a multi-reflective 3D scene

An illumination optimisation technique applied to multi-reflective 3-D machine vision based on a projector-camera system is introduced, in which the projector plays a key role to compensate for surface reflectance at each pixel to be inversely proportional to the brightness of the pixel under ambient light. The adaptive illumination technology was achieved by iterations emphasising different illumination intensities according to different surface orientations and requiring an accurate correspondence between the projector pixels and the camera pixels. In order to establish the most effective correspondence to prepare for subsequent adaptive illumination, 4 kinds of grating patterns, including sinusoidal, rectangular, triangular, and dual-frequency sinusoidal grating patterns, were projected and compared. The iterations were halted when an optimally lit scene was obtained; the further experiments under weak and strong light searched for the best method of illumination optimisation and confirmed the reliability of the adaptive illumination. The proposed optimisation design could run in real time and became a viable solution for industry.

Easy-to-operate calibration method for structured light systems.

Traditional calibration methods of structured light measurement systems are expensive and difficult to operate. In this paper, an accurate, low-cost, and easy-to-operate calibration method is proposed, where each projector pixel is considered as a line-of-sight in space, and an off-the-shelf LCD monitor is used to display the calibration pattern. Experimental results verify the accuracy and validity of the proposed method.

Single-Shot Dense 3D Reconstruction Using Self-Equalizing De Bruijn Sequence.

Single-shot dense 3D reconstruction using colored structured light is a difficult problem due to the undesired effects of ambient lighting, object albedo, non-equal channel gains, and channel cross-talk. We propose a novel single-shot dense 3D reconstruction using colored structured light. Our method combines the self-equalizing De Bruijn sequence, scale-space analysis, and bandpass complex Hilbert filters to achieve insensitivity to ambient lighting, object albedo, and non-equal channel gains. The proposed method reconstructs about 85% of points compared to time-multiplexing structured light strategies and the decoding error in the recovered projector coordinate is less than one projector pixel for about 90% of reconstructed points.

Simulation-Based Building Change Detection From Multiangle SAR Images and Digital Surface Models

This paper presents two change-detection strategies based on the fusion of scene knowledge and two high-resolution synthetic aperture radar (SAR) images (pre-event, postevent) with focus on individual buildings and facades. Avoiding the dependence of the signal incidence angle, the methods increase the flexibility with respect to near-real-time SAR image analysis after unexpected events. Knowledge of the scene geometry is provided by digital surface models (DSMs), which are integrated into an automated simulation processing chain. Using strategy 1 (based on building fill ratio, BFR), building changes are detected based on change ratios considering layover and shadow areas. Strategy 2 (based on wall fill position, WFP) enables one to analyze individual facades of buildings without clear decision from strategy 1, which is based on a geometric projection of facade layover pixels. In a case study (Munich city center), the sensitivity of the change-detection methods is exemplified with respect to destroyed buildings and partly changed buildings. The results confirm the significance of integrating prior knowledge from DSMs into the analysis of high-resolution SAR images.

Real-time pixel luminance optimization for dynamic multi-projection mapping

Using projection mapping enables us to bring virtual worlds into shared physical spaces. In this paper, we present a novel, adaptable and real-time projection mapping system, which supports multiple projectors and high quality rendering of dynamic content on surfaces of complex geometrical shape. Our system allows for smooth blending across multiple projectors using a new optimization framework that simulates the diffuse direct light transport of the physical world to continuously adapt the color output of each projector pixel. We present a real-time solution to this optimization problem using off-the-shelf graphics hardware, depth cameras and projectors. Our approach enables us to move projectors, depth camera or objects while maintaining the correct illumination, in realtime, without the need for markers on the object. It also allows for projectors to be removed or dynamically added, and provides compelling results with only commodity hardware.

Statistical 3D face shape estimation from occluding contours

This paper addresses the problem of 3D face shape approximation from occluding contours, i.e., the boundaries between the facial region and the background. To this end, a linear regression process that models the relationship between a set of 2D occluding contours and a set of 3D vertices is applied onto the corresponding training sets using Partial Least Squares. The result of this step is a regression matrix which is capable of estimating new 3D face point clouds from the out-of-training 2D Cartesian pixel positions of the selected contours. Our approach benefits from the highly correlated spaces spanned by the 3D vertices around the occluding boundaries of a face and their corresponding 2D pixel projections. As a result, the proposed method resembles dense surface shape recovery from missing data. Our technique is evaluated over four scenarios designed to investigate both the influence of the contours included in the training set and the considered number of contours. Qualitative and quantitative experiments demonstrate that using contours outperform the state of the art on the database used in this article. Even using a limited number of contours provides a useful approximation to the 3D face surface.

Новый способ определения углов наклона космических снимков по известным размерам изображенных на них объектов местности

Abstract: To obtain geoinformation materials using remote sensing it is necessary to know rotation parameters of exterior orientation of the images processed. The author describes in the article a new method of satellite images angles determination, or, to be more exact, of camera focal plane inclination. Its essence is as follows: size of image pixel projection ground size is determined with photo camera focal plane inclination angle. So, if image pixel projection ground size is known, it is possible to find an inclination angle value of photo camera focal plane relatively to direction of nadir. The author’s formulas are given for determination of inclination angles (pitch and roll) of the camera focal plane according to the known values of pixel projection ground size.

Accurate projector calibration method by using an optical coaxial camera.

Digital light processing (DLP) projectors have been widely utilized to project digital structured-light patterns in 3D imaging systems. In order to obtain accurate 3D shape data, it is important to calibrate DLP projectors to obtain the internal parameters. The existing projector calibration methods have complicated procedures or low accuracy of the obtained parameters. This paper presents a novel method to accurately calibrate a DLP projector by using an optical coaxial camera. The optical coaxial geometry is realized by a plate beam splitter, so the DLP projector can be treated as a true inverse camera. A plate having discrete markers on the surface is used to calibrate the projector. The corresponding projector pixel coordinate of each marker on the plate is determined by projecting vertical and horizontal sinusoidal fringe patterns on the plate surface and calculating the absolute phase. The internal parameters of the DLP projector are obtained by the corresponding point pair between the projector pixel coordinate and the world coordinate of discrete markers. Experimental results show that the proposed method can accurately calibrate the internal parameters of a DLP projector.

적응 휴리스틱 분할 알고리즘을 이용한 실시간 차량 번호판 인식 시스템

차량 번호판 인식 시스템은 복잡한 교통환경의 효율적 관리를 위해 발전되어 현재 많은 곳에 사용되고 있다. 그러나 조명, 잡음, 배경변화, 번호판 훼손 등 환경변화에 큰 영향을 받기 때문에 제한된 환경에서만 동작하며, 실시간으로 사용하기 어렵다. 본 논문에서는 조명변화와 잡음에 강건하며 빠른 번호판 인식을 위한 휴리스틱 분할 알고리즘 및 이를 이용한 실시간 번호판 인식 시스템을 제안한다. 첫 번째 단계는 Haar-like 특징과 Adaboost를 이용하여 번호판을 검출한다. 이 방법은 적분영상을 이용하며 케스케이드 구조로 구성되어 있어 빠른 검출이 가능하다. 두 번째 단계에서 적응 히스토그램 평활화 방법과 노이즈를 경감시키는 바이레터럴 필터를 이용하여 번호판의 종류를 결정한 후, 번호판 종류에 따라 적분영상을 이용한 적응 이진화, 픽셀 프로젝션, 사전지식 등을 기반으로 빠르고 정확한 문자 분할을 한다. 세번째 단계에서는 HOG와 신경망 알고리즘을 이용하여 숫자를 인식하고, SVM을 이용해 한글을 인식한다. 실험결과는 번호판검출에 94.29%의 검출률, 2.94%의 오경보율을 보이며, 문자분할에서는 검출률 97.23%, 2.94%의 오경보율을 보였다. 문자인식에서 평균 인식률은 98.38%이다. 평균 운용시간은 140ms으로 빠르고 강인한 실시간 시스템을 만들 수 있다. The LPR(License plate recognition) system has been developed to efficient control for complex traffic environment and currently be used in many places. However, because of light, noise, background changes, environmental changes, damaged plate, it only works limited environment, so it is difficult to use in real-time. This paper presents a heuristic segmentation algorithm for robust to noise and illumination changes and introduce a real-time license plate recognition system using it. In first step, We detect the plate utilized Haar-like feature and Adaboost. This method is possible to rapid detection used integral image and cascade structure. Second step, we determine the type of license plate with adaptive histogram equalization, bilateral filtering for denoise and segment accurate character based on adaptive threshold, pixel projection and associated with the prior knowledge. The last step is character recognition that used histogram of oriented gradients (HOG) and multi-layer perceptron(MLP) for number recognition and support vector machine(SVM) for number and Korean character classifier respectively. The experimental results show license plate detection rate of 94.29%, license plate false alarm rate of 2.94%. In character segmentation method, character hit rate is 97.23% and character false alarm rate is 1.37%. And in character recognition, the average character recognition rate is 98.38%. Total average running time in our proposed method is 140ms. It is possible to be real-time system with efficiency and robustness.

Artifact Reduction in Radiometric Compensation of Projector-Camera Systems for Steep Reflectance Variations

In this paper, we propose a novel radiometric compensation method that applies a high-spatial-resolution camera to a projector-camera system to reduce the artifacts around the regions where the reflectance of the projection surface changes steeply. The proposed method measures the reflection in the region of a single projector pixel on a projection surface with multiple camera pixels. From the measurement, it computes multiple color-mixing matrices, each of which represents a color space conversion between each camera and the projector pixels. Using these matrices, we calculate the optimal projection color by applying the linear least squares method, so that the displayed color in the projector pixel region is as close as possible to the target appearance. Through projection experiments, we confirm that our proposed method reduces the artifacts around the regions where the reflectance changes steeply, when compared with other conventional compensation methods.

Development of a DC-DC conversion powering scheme for the CMS Phase-1 pixel upgrade

A novel powering scheme based on the DC-DC conversion technique will be exploited to power the CMS Phase-1 pixel detector. DC-DC buck converters for the CMS pixel project have been developed, based on the AMIS5 ASIC designed by CERN. The powering system of the Phase-1 pixel detector is described and the performance of the converter prototypes is detailed, including power efficiency, stability of the output voltage, shielding, and thermal management. Results from a test of the magnetic field tolerance of the DC-DC converters are reported. System tests with pixel modules using many components of the future pixel barrel system are summarized. Finally first impressions from a pre-series of 200 DC-DC converters are presented.

Consistent Stereo-Assisted Absolute Phase Unwrapping Methods for Structured Light Systems

Phase shifted sinusoidal patterns have proven to be effective in structured light systems, which typically consist of a camera and projector. They offer low decoding complexity, require as few as three projection frames per reconstruction, and are well suited for capturing dynamic scenes. In these systems, depth is reconstructed by determining the phase projected onto each pixel in the camera and establishing correspondences between camera and projector pixels. Typically, multiple periods are projected within the set of sinusoidal patterns, thus requiring phase unwrapping on the phase image before correspondences can be established. A second camera can be added to the structured light system to help with phase unwrapping. In this work, we present two consistent phase unwrapping methods for two-camera stereo structured light systems. The first method enforces viewpoint consistency by phase unwrapping in the projector domain. Loopy belief propagation is run over the graph of projector pixels to select pixel correspondences between the left and right camera that align in 3-D space and are spatially smooth in each 2-D image. The second method enforces temporal consistency by unwrapping across space and time. We combine a quality guided phase unwrapping approach with absolute phase estimates from the stereo cameras to solve for the absolute phase of connected regions. We present results for both methods to show their effectiveness on real world scenes.

Anchoring Graph Cuts Towards Accurate Depth Estimation in Integral Images

Integral imaging is a three-dimensional (3D) imaging technique that allows the displaying of full color images with continuous parallax. Its commercial potential has been increased, due to its ability of presenting to the viewers smooth 3D images, with full parallax, in a wide viewing zone. Being able to extract the inherent 3D information from the planar integral images and produce 3D reconstructions, offers advantages in various applications of immersive entertainment and communications. On this scope, this paper addresses the problem of accurate depth estimation in integral images. The proposed method, relying on the assumption that a pixel is the projection of a 3D imaging ray, aims to specify the first intersection of each pixel's projection ray with the 3D scene in order to assign to it the corresponding depth value. This task is formulated as an energy optimization problem and the graph cuts approach is utilized to solve it. The energy term is twofold; its first part aims to restrict the desired solution to be close to the observed data, i.e., the integral image, while the second one enforces regional smoothness in the depth estimation. This combination offers an accurate and spatially smooth scene structure. The novelty of the paper lies on the framework's formulation as one single optimization procedure and on the way that this optimization is constrained by a set of reliably estimated 3D surface points, called the “anchor points”. Anchoring the optimization results in enhanced depth estimation accuracy, while decreasing the optimization processing burden. The proposed algorithm is evaluated in both synthetic and real integral images consisting of complicated object scenes. A comparison against other state-of-the-art algorithms proves the superiority of the proposed method in terms of depth estimation accuracy.

Improved number plate character segmentation algorithm and its efficient FPGA implementation

Character segmentation is an important stage in Automatic Number Plate Recognition systems as good character separation leads to a high recognition rate. This paper presents an improved character segmentation algorithm based on pixel projection and morphological operations. An efficient architecture based on the proposed algorithm is also presented. The architecture has been successfully implemented and verified using the Mentor Graphics RC240 FPGA (Field Programmable Gate Arrays) development board equipped with a 4M-Gate Xilinx Virtex-4 LX40. A database of 1,000 UK binary NPs with varying resolution has been used for testing the performance of the proposed architecture. Results achieved have shown that the proposed architecture can process a number plate image in 0.2---1.4 ms with 97.7 % successful segmentation rate and consumes only 11 % of the available area in the used FPGA.

Fast high-resolution appearance editing using superimposed projections

We present a system that superimposes multiple projections onto an object of arbitrary shape and color to produce high-resolution appearance changes. Our system produces appearances at an improved resolution compared to prior works and can change appearances at near interactive rates. Three main components are central to our system. First, the problem of computing compensation images is formulated as a constrained optimization which yields high-resolution appearances. Second, decomposition of the target appearance into base and scale images enables fast swapping of appearances on the object by requiring the constrained optimization to be computed only once per object. Finally, to make high-quality appearance edits practical, an elliptical Gaussian is used to model projector pixels and their interaction between projectors. To the best of our knowledge, we build the first system that achieves high-resolution and high-quality appearance edits using multiple superimposed projectors on complex nonplanar colored objects. We demonstrate several appearance edits including specular lighting, subsurface scattering, inter-reflections, and color, texture, and geometry changes on objects with different shapes and colors.

A Structured-Light Approach for the Reconstruction of Complex Objects

Recently, one of the central issues in the fields of Photogrammetry, Computer Vision, Computer Graphics and Image Processing is the development of tools for the automatic reconstruction of complex 3D objects. Among various approaches, one of the most promising is Structured Light 3D scanning (SL) which combines automation and high accuracy with low cost, given the steady decrease in price of cameras and projectors. SL relies on the projection of different light patterns, by means of a video projector, on 3D object sur faces, which are recorded by one or more digital cameras. Automatic pattern identification on images allows reconstructing the shape of recorded 3D objects via triangulation of the optical rays corresponding to projector and camera pixels. Models draped with realistic phototexture may be thus also generated, reproducing both geometry and appearance of the 3D world. In this context, subject of our research is a synthesis of state-of-the-art as well as the development of novel algorithms, in order to implement a 3D scanning system consisting, at this stage, of one consumer digital camera (DSLR) and a video projector. In the following, the main principles of structured light scanning and the algorithms implemented in our system are presented, and results are given to demonstrate the potential of such a system. Since this work is part of an ongoing research project, future tasks are also discussed.