Image Plane Of Camera Research Articles

Vision-based tracking of a dynamic target with application to multi-axis position control

Two sub-pixel resolution approaches to measure in-plane displacements and in-plane rotation of a known target, through image processing, are described in this research. A dynamic known target is displayed on a pixel grid, which is attached to one end of the kinematic chain of an XY? Z stage; the latter represents the experimental testbed. At the other end of the kinematic chain, a digital monochrome camera is fixed to the bottom of the stage and provides 3D position information used as the feedback signal to the vision-based control system in charge of the tool's motion. The illuminated pixels on the display are captured in real time by the digital camera, and the stage motion control system attempts to keep the displayed image in the proper location with respect to the camera image plane. The result is a direct sensing multi-DOF position feedback system. The proposed camera-pixel grid sensing setup eliminates the reliance on the kinematic model and also avoids the need for traditional error compensation techniques, along with their associated cost and complexity. Positioning resolutions on the order of 1/100th of the pixel size on the display are achieved.

듀얼 모드(고정형+PTZ 카메라) 감시 카메라를 이용한 효과적인 화상 감시 시스템에 관한 연구

An effective dual-mode camera system(a passive wide-angle camera and a pan-tilt-zoom camera) is proposed in order to improve the performance of visual surveillance. The fixed wide-angle camera is used to monitor large open areas, but the moving objects on the images are too small to view in detail. And, the PTZ camera is capable of increasing the monitoring area and enhancing the image quality by tracking and zooming in on a specific moving target. However, its FOV (Field of View) is limited when zooming in on a specific target. Therefore, the cooperation of wide-angle and PTZ cameras is complementary. In this paper, we propose an automatic initial set-up algorithm and coordinate transform method from the wide-angle camera coordinate to the PTZ one, which are necessary to achieve the cooperation. The automatic initial set-up algorithm is able to synchronize the views of two cameras. When a moving object appears on the image plane of a wide-angle camera after the initial set-up positioning, the obtained values of the wide-angle camera should be transformed to the PTZ values based on the coordinate transform method. We also develope the PTZ control method. Various in-door and out-door experiments show that the proposed dual-camera system is feasible for the effective visual surveillance.

Multi-sensor 3D volumetric reconstruction using CUDA

This paper presents a full-body volumetric reconstruction of a person in a scene using a sensor network, where some of them can be mobile. The sensor network is comprised of couples of camera and inertial sensor (IS). Taking advantage of IS, the 3D reconstruction is performed using no planar ground assumption. Moreover, IS in each couple is used to define a virtual camera whose image plane is horizontal and aligned with the earth cardinal directions. The IS is furthermore used to define a set of inertial planes in the scene. The image plane of each virtual camera is projected onto this set of parallel-horizontal inertial-planes, using some adapted homography functions. A parallel processing architecture is proposed in order to perform human real-time volumetric reconstruction. The real-time characteristic is obtained by implementing the reconstruction algorithm on a graphics processing unit (GPU) using Compute Unified Device Architecture (CUDA). In order to show the effectiveness of the proposed algorithm, a variety of the gestures of a person acting in the scene is reconstructed and demonstrated. Some analyses have been carried out to measure the performance of the algorithm in terms of processing time. The proposed framework has potential to be used by different applications such as smart-room, human behavior analysis and 3D teleconference.

Grid point extraction and coding for structured light system

A structured light system simplifies three-dimensional reconstruction by illuminating a specially designed pattern to the target object, thereby generating a distinct texture on it for imaging and further processing. Success of the system hinges upon what features are to be coded in the projected pattern, extracted in the captured image, and matched between the projector's display panel and the camera's image plane. The codes have to be such that they are largely preserved in the image data upon illumination from the projector, reflection from the target object, and projective distortion in the imaging process. The features also need to be reliably extracted in the image domain. In this article, a two-dimensional pseudorandom pattern consisting of rhombic color elements is proposed, and the grid points between the pattern elements are chosen as the feature points. We describe how a type classification of the grid points plus the pseudorandomness of the projected pattern can equip each grid point with a unique label that is preserved in the captured image. We also present a grid point detector that extracts the grid points without the need of segmenting the pattern elements, and that localizes the grid points in subpixel accuracy. Extensive experiments are presented to illustrate that, with the proposed pattern feature definition and feature detector, more features points in higher accuracy can be reconstructed in comparison with the existing pseudorandomly encoded structured light systems.

Stereo rectification of calibrated image pairs based on geometric transformation

The objective of stereo rectification is to make the corresponding epipolar lines of image pairs be parallel to the horizontal direction, so that the efficiency of stereo matching is improved as the corresponding points stay in the same horizontal lines of both images. In this paper,a simple and convenient rectification method of calibrated image pairs based on geometric transformation is proposed, which can avoid the complicated calculation of many previous algorithms such as based on epipolar lines, based on fundamental matrix or directly depend on corresponding points. This method is divided into two steps including coordinate system transformation and re-projection of image points. Firstly, we establish two virtual cameras with parallel optical axis by coordinate system transformation based on the pose relationship of the two cameras from calibration result. Secondly, we re-project the points of the original image onto new image planes of the virtual cameras through geometrical method, and then realized the stereo rectification. Experiments of real stereo image pairs show that the proposed method is able to realize the rectification of stereo image pairs accurately and efficiently.

AN AUTOMATIC CALIBRATION METHOD BASED ON FEATURE POINT MATCHING FOR THE COOPERATION OF WIDE-ANGLE AND PAN-TILT-ZOOM CAMERAS

Wide-angle cameras are used to monitor large open areas, but moving objects on the image are often too small to view in detail. However, pan-tilt-zoom (PTZ) cameras have zooming capabilities that can be used to capture close-up images of a moving object. Therefore, the cooperation of these two types of cameras has been studied extensively in recent years. A calibration method is necessary to achieve the above cooperation. For any coordinate on the image plane of a wide-angle camera, this calibration method is needed to obtain the corresponding pan, tilt, and zoom values, known as the point correspondence of the PTZ camera. It is inconvenient to manually establish point correspondences, as shown in previous studies. Therefore, an automatic calibration method is proposed in this paper. The method is mainly based on the matching of feature points on the images of wide-angle and PTZ cameras. A feature point (FP) is extracted based on several edge operators derived from a 2D Gaussian smoothing function. A two-phase FP match is then designed to obtain the point correspondences. In the study of a prototype for capturing close-up images of moving objects, the calibration process took approximately 7 minutes for 44 FPs. The average successful rate of the capturing tasks was 82 percent. It shows that the automatic calibration method is feasible for the cooperation of wide-angle and PTZ cameras.http://dx.doi.org/10.5755/j01.itc.40.1.191

Vision-based detection and tracking of vehicles to the rear with perspective correction in low-light conditions

In this study, the authors present a video-processing system that utilises a camera to detect and track vehicles to the rear at night. Vehicle detection is a pivotal component of camera-based advanced driver assistance systems (ADAS) such as collision warning, blind-spot monitoring and overtaking vehicle detection. When driving in dark conditions, vehicles to the rear are primarily visible by their headlamps. This system implements a low, static camera exposure to ensure headlamps appear distinct and not as enlarged bloomed regions. We further describe a method to identify vehicle headlamp pairs using region-growing thresholding and cross-correlation bilateral symmetry analysis. Images of vehicles at different yaw angles to the camera image plane, such as those turning, or engaging road bends, suffer from perspective distortion, which interferes with the symmetry between lamps. We perform a perspective transformation to correct for this, ensuring consistent detection performance throughout all road manoeuvres. False positives resulting from multiple vehicle situations are considered and addressed. Detected target vehicles are tracked using a Kalman filter which is updated by inter-frame cross-correlation.

SU‐FF‐J‐28: A Guidance System for Optical Patient Alignment During Breast Radiotherapy

Purpose: Breast radiotherapy, particularly IMRT, involves large dose gradients and difficult patient positioning problems. A critical requirement for successful treatment is accurate reproduction of the patient's position assumed during CT simulation and planning. We have developed an optical image‐guided technique, which assists in accurately and reproducibly positioning the patient, by displaying her real‐time optical image superimposed on a perspective projection image of her 3D CT data. Methods and Materials: The Single Projection Technique (SPT) accurately determines the 3‐D position and orientation of a camera from a single image acquired of a known model. A calibration jig, composed of ten identifiable reflecting spheres, was constructed and CT imaged to provide this model. To implement our method, a digital photograph of the jig was acquired and the 2D coordinates of the spheres were found. Using this information, 3D CT patient data is projected onto the camera's imaging plane, and is displayed on a monitor, superimposed on the real‐time patient image. This enables the therapist to view both the patient's current and desired positions, and guides proper patient positioning. Results: The SPT can determine the position and orientation of the camera to an accuracy of 0.2 cm and 0.3°, respectively. Investigations are ongoing to determine the accuracy and reproducibility of our method in terms of the dose distribution of IMRT plans. Film measurements performed on a breast phantom will allow us to determine the spatial accuracy of isodose curves measured in several planes. Conclusions: We have developed a method to calibrate an optical camera system and superimpose a perspective projection of a CT image on a patient's real‐time optical image. Displaying this visual information will assist in accurate setup during breast radiotherapy. Future work will enable us to quantify the setup and dose delivery accuracy of this technique.

A New Calibration Method and its Application for the Cooperation of Wide-Angle and Pan-Tilt-Zoom Cameras

In this study, we present a new calibration method for the cooperation of wide-angle and pan-tilt-zoom (PTZ) cameras. A wide-angle camera enables the surveillance of a large area. However, it may be difficult to perceive the detail of an object. On the other hand, a PTZ camera enables the surveillance of the object detail. However, its view is limited when the camera zooms on a specific object. Therefore, the cooperation can complement the disadvantage of two types of cameras. For an object appears in the image of wide-angle camera, the desired pan and tilt angles of a PTZ camera for focusing on the object can be computed from its image coordinate. Previous methods can be classified into three categories: 3D world coordinate system, scene-dependent, or special technique. They are not simple and instinctive enough. The proposed method is simply based on the 2D image plane of wide-angle camera. A set of calibration points are selected directly on the image. Every point is a pair of pan/tilt angles (p, t) and 2D coordinate (x, y). Then, the calibration parameters are calculated from the set of points. An experiment was designed to evaluate the accuracy of the proposed method at fixed points. The results show that the proposed method is accurate enough for the cooperation of dual cameras. In advance, a prototype system for capturing close-up images of moving objects in an open area was implemented to demonstrate the feasibility of the proposed method.

SU-GG-J-29: An Optical Guidance System for Patient Setup in Breast Radiotherapy

Purpose: Breast radiotherapy, particularly IMRT, involves large dose gradients and difficult patient positioning problems. A critical requirement for successful treatment is accurate reproduction of the patient's position assumed during CT simulation and planning. We have developed an image-guided technique, which assists in accurately and reproducibly positioning the patient, by displaying her real-time optical image superimposed on a perspective projection image of her 3D CT data. Method. The Single Projection Technique (SPT) accurately determines the 3-D position and orientation of a camera from a single image acquired of a known model. A calibration jig, composed of ten identifiable points, was constructed and CT imaged to provide this model. In a preliminary study, a regular digital camera was installed in the treatment suite and used to obtain an image of the jig. The SPT then provided the coordinates and orientation of the camera. Using this information, 3D CT patient data could then be projected onto the camera's imaging plane. The CT image data would then be superimposed onto the real-time patient image using standard graphical software and displayed on a monitor. This would enable the therapist to view both the patient's current and desired positions, and guide the patient into assuming the correct position. Results: The SPT can determine the position and orientation of the camera to an accuracy of 0.2 cm and 0.3°, respectively. This allows an estimated accuracy of 5 mm in the fidelity of the patient's external anatomy to the surface CT data. This includes anatomical points not easily positioned properly, such as the raised arm and breast skin surface. Conclusion: We have developed a method to superimpose a perspective projection of a CT image on a patient's real-time optical image. Displaying this visual information will assist in accurate setup during breast radiotherapy.

Camera and visual veiling glare in HDR images

Abstract— High‐dynamic‐range (HDR) images are superior to conventional images. The experiments in this paper measure camera and human responses to calibrated HDR test targets. We calibrated a 4.3‐log‐unit test target, with minimal and maximal glare from a changeable surround. Glare is an uncontrolled spread of an image‐dependent fraction of scene luminance in cameras and in the eye. We use this standard test target to measure the range of luminances that can be captured on a camera's image plane. Further, we measure the appearance of these test luminance patches. We discuss why HDR is better than conventional imaging, despite the fact the reproduction of luminance is inaccurate.

Minimizing errors during in vitro testing of multisegmental spine specimens: Considerations for component selection and kinematic measurement

Apparatus-induced artifacts may invalidate standard spine testing protocols. Kinematic measurements may be compromised by the configuration of motion capture equipment. This study has determined: (1) the influence of machine design (component friction) on in vitro spinal kinetics; (2) the sensitivity of kinematic measurements to variations in the placement of motion capture markers. A spinal loading simulator has been developed to dynamically apply pure bending moments (three axes) with or without a simultaneous compressive preload. Two linear slider types with different friction coefficients, one with caged ball bearings and one with high-precision roller bearings on rails, were mounted and specimen response compared in sequential tests. Three different optoelectronic marker cluster configurations were mounted on the specimen and motion data was captured simultaneously from all clusters during testing. A polymer tube with a uniform bending stiffness approximately equivalent to a polysegmental lumbar spine specimen was selected to allow reproducible behavior over multiple tests. The selection of sliders for linear degrees of freedom had a marked influence on parasitic shear forces. Higher shear forces were recorded with the caged-bearing design than with the high-precision rollers and consequently a higher moment was required to achieve a given rotation. Kinematic accuracy varied with each marker configuration, but in general higher accuracy was achieved with larger marker spacings and situations where markers moved predominantly parallel to the camera's imaging plane. Relatively common alternatives in the mechanical components used in an apparatus for in vitro spine testing can have a significant influence on the measured kinematic and kinetics. Low-magnitude parasitic shear forces due to friction in sliders induces a linearly increasing moment along the length of the specimen, precluding the ideal of pure moment application. This effect is compounded in polysegmental specimens. Kinematic measurements are highly sensitive to marker design and placement, despite equivalent absolute precision of individual marker measurements, however marker configurations can be designed to minimize errors related to spatial distribution and system bias.

Accurate phase–height mapping algorithm for PMP

In phase measurement profilometry (PMP), the projector can be regarded as another camera according to the reversibility of the light path principle. The relationship of projecting spatial points to image plane of camera and projector is studied, and the phase–height mapping equation without projector distortion is obtained. The equation is then expanded to a polynomial for the convenience of calibration. Furthermore, the relation between the distortion value and the phase is investigated. Finally the phase–height mapping algorithm considering projector distortion and its polynomial expression are acquired. The accuracy of approximation is studied and compared with another two existing algorithms by computer simulation. It is revealed that the absolute error of the new algorithm expressed with quartic polynomial reaches 5.380× 10−3 mm and its standard deviation reaches 3.354× 10−4 mm under general lens distortion. The accuracy of the new algorithm is the highest among the three algorithms. In experiment, the standard deviation of the measurement reaches 0.04 mm even though the result is affected by measurement error.

Direct visual tracking control of remote cellular robots

This paper presents the design of a stable non-linear control system for the remote visual tracking of cellular robots. The robots are controlled through visual feedback based on the processing of the image captured by a fixed video camera observing the workspace. The control algorithm is based only on measurements on the image plane of the visual camera–direct visual control–thus avoiding the problems related to camera calibration. In addition, the camera plane may have any (unknown) orientation with respect to the robot workspace. The controller uses an on-line estimation of the image Jacobians. Considering the Jacobians’ estimation errors, the control system is capable of tracking a reference point moving on the image plane–defining the reference trajectory–with an ultimately bounded error. An obstacle avoidance strategy is also developed in the same context, based on the visual impedance concept. Experimental results show the performance of the overall control system.

Design and development of automated 3D visual tracking system†

Camera‐based systems are frequently used to track moving objects which are in the field of their view. This paper describes design and development of a camerabased visual system that can constantly track a moving object without the necessity of calibrating the camera in real world coordinates. This reduces the complexity of the system and processing time due to unnecessary conversions and calibrations. The system consists of a two‐motor pan‐tilt camera driving mechanism, PCI image acquisition board, and PWM‐based DC‐motor driver board. It uses image processing techniques to identify and locate the object in the 3D scene and motion control algorithms to direct the camera towards the object. Thus the objective of the project is to develop a vision system and control algorithms which can be locked on a moving object within the field of its view. The developed software and related interface hardware monitor and control the motors in such a way that the moving object should always be located right at the center of the camera image plane. The system, in general, simulates the 3D motion of the human eye which always tends to focus on a moving object within the range of its view. It actually imitates the tracking ability of human eye.

Calibration of a 3D vision system using pattern projection

In this paper, we describe a calibration method for 3D vision system using pattern projection. The calibration consists of two phases: off-line calibration of the parameters of the pattern projector and on-line calibration of the varying intrinsic and extrinsic parameters of the camera. During the on-line recalibration, we do not need to calibrate the homographies of all the light stripe planes relative to the camera, but just need to calibrate those of the two or more arbitrary light planes. Then all of the other light stripe planes’ homographies relative to the camera image plane can be recovered based on the relation derived in this paper between the homographies of the light stripe planes and their transformations with respect to the world coordinate frame. By this method, we can easily implement recalibration of the 3D vision system with its pattern projection when the intrinsic and extrinsic parameters of the camera are changed.

Identification of Riccati dynamics under perspective and orthographic observations

The problem of identifying and shape parameters of a planar object undergoing a Riccati motion, from the associated optical flow generated on the image plane of a single CCD camera, is studied. The optical flow is generated by projecting feature points on the object onto the image plane via perspective and orthographic projections. An important result we show is that, under perspective projection, the parameters of a specific Riccati dynamics that extend the well-known motion can be identified up to choice of a sign. The paper also discusses other Riccati equations obtained from quadratic extension of a rigid and affine motion. For each of the various models considered and for each of the two projection models, we show that the extent to which and shape parameters can be recovered from optical flow can in fact be recovered from the linear approximation of the optical flow. We also extend our analysis to a pair of cameras.

高精度位置計測のための視野分離撮像法

Visual sensing and measurement are utilized widely in the various production processes. In this visual sensing, high accurate micro sensing in the macro processing area is required. Wide field of vision is required for the visual sensing in a macro area. On the other hands, short pixel length in a pictured image is required for high accurate visual sensing. These two requirements are opposite for visual sensing by TV camera. In general, visual sensing by the plural TV cameras or movement of a TV camera is engaged for the visual sensing of the separated regions. But, these methods have significant problems in a calibration process of the optical parameters and their calibration accuracy. If the two separated region of the object works for measurement can be pictured on a same image plane of a TV camera, high accurate micro sensing in a macro view area can be realized and the calibration process will be done simply. In this research, we propose an imaging method with separated fields of vision by using a prism, and make clear the relation. between the optical parameters and the fields of vision. The distance between the pictured fields of vision is fixed by a vertical angle and a refractive index of a prism and the distance between a pictured object and a lens of a TV camera.

ＶＲを支える映像情報メディア技術 パン・チルト・ズームカメラの簡易モデルによるキャリブレーション

We propose simple model for calibrating of our pan/tilt/zoom camera, which consists of two rotation stages, an electrically-controlled zoom lens, and a CCD camera. The simple model assumes that zoom changes do not cause any change in the optical axis, the image center, the position of the image plane of camera in world coordinates system, or the rotation axes of pan/tilt. We show that when our model is used, calibration can be achieved independently of the zoom setting and the number of parameters that must be estimated in calibration decreases.Because calibration of the pan/tilt/zoom camera was achieved, we can obtain the images by which the viewing angle and the magnification are set in an arbitrary value. And we show that the images are easily integrated using the position between the images derived from the estimated parameters.

Measurement of slopes and profile of an optical lens by shearography

In this paper, an optical technique based on speckle-shearing interferometry and a double refractive-index method is employed for the measurement of surface slopes and profile of an optical lens. The lens to be studied is mounted on one side of a glass tank filled with water and illuminated with a collimated coherent light source. The lens image is captured on a photographic film placed at the image plane of an image-shearing camera. A second exposure on the photographic film is made after the container is refilled with a liquid of different refractive index. The resulting photograph yields a fringe pattern which represents lines of constant derivatives of the surface profile. Test results conducted on a plano-concave lens compared favorably with theoretical predictions.