Image Plane Coordinates Research Articles

Infrastructure camera calibration with GNSS for vehicle localisation

Intelligent transportation and smart city applications are currently on the rise. In many applications, diverse and accurate sensor perception of vehicles is crucial. Relevant information could be conveniently acquired with traffic cameras, as there is an abundance of cameras in cities. However, cameras have to be calibrated in order to acquire position data of vehicles. This paper proposes a novel automated calibration approach for partially connected vehicle environments. The approach utilises Global Navigation Satellite System positioning information shared by connected vehicles. Corresponding vehicle Global Navigation Satellite System locations and image coordinates are utilised to fit a direct transformation between image and ground plane coordinates. The proposed approach was validated with a research vehicle equipped with a Real-Time Kinematic-corrected Global Navigation Satellite System receiver driving past three different cameras. On average, the camera estimates contained errors ranging from 1.5 to 2.0 m, when compared to the Global Navigation Satellite System positions of the vehicle. Considering the vast lengths of the overlooked road sections, up to 140 m, the accuracy of the camera-based localisation should be adequate for a number of intelligent transportation applications. In future, the calibration approach should be evaluated with fusion of stand-alone Global Navigation Satellite System positioning and inertial measurements, to validate the calibration methodology with more common vehicle sensor equipment.

Image Principal Distance Calibration Method by Binocular Camera Based on Improved Differential Evolution Algorithm

The traditional camera calibration method usually relies on the calibration plate, mainly uses the exhaustive method, and converts the spatial coordinates into image plane coordinates by solving the resection of the equation. The accuracy and calculation rate of traditional algorithms can no longer fully meet the needs of engineering applications. In view of this situation, this paper proposes an improved differential evolution algorithm (IDE) method for binocular camera calibration. The differential evolution algorithm (DE) is an algorithm that simulates the biological evolution process and uses the survival of the fittest to gradually screen individuals to find the optimal solution. The standard DE algorithm needs to perform population initialization, mutation operation and crossover operation, and finally screen individuals through greedy algorithm. In the mutation operation of the DE algorithm, the coefficient of variation F plays a crucial role in the result but it is difficult to obtain a value. Therefore, it is necessary to optimize the operator of the coefficient of variation F, so that the coefficient of variation F also reaches the optimal solution, which is the IDE algorithm. In this paper, the traditional calculation method, the standard DE algorithm and the IDE algorithm are used to calibrate the principal distance of the two images, and the accuracy and iteration rate are compared. The experimental results show that the IDE algorithm has obvious advantages in terms of accuracy, especially the calculation speed.

A photogrammetry-based variational optimization method for river surface velocity measurement

The ease of access to media resources and computational power has recently generated interest in using vision-based approaches for hydraulic monitoring. A key challenge for non-intrusive, image-based hydrology measurement methods is incorporating different hydraulic variables as prior knowledge with image information. We propose a photogrammetry-based method called L1-Diffusion derived from the convection–diffusion equation commonly used in hydrodynamics with an additional regularization term to estimate the fluid motion field in the image plane, from which the free surface velocity can be further obtained using the photogrammetric projection relationship between the image plane and world coordinates. The inverse problem is used to discuss the relationship between the widely used space–time image velocimetry (STIV) and the proposed L1-Diffusion. To validate the proposed method, unmanned aerial vehicle (UAV) images as well as in-situ acoustic Doppler current profiler (ADCP) experiments were carried out. Based on comparison results with the ADCP measurement and vision-based flow field estimation, the newly proposed L1-Diffusion algorithm can accurately and efficiently estimate the free surface velocity of a river from the image sequences in a variety of scenarios.

Robust geometric, phase and colour structured light projection system calibration

This paper introduces a new comprehensive procedure for both geometric and colour calibration of structured light system. In order to perform both geometric and colour calibration procedure, a new calibration artifact is proposed. The intrinsic and extrinsic parameters of projector and camera are estimated by using an extended pinhole camera model with a tangential and radial distortion. Camera image plane coordinates are obtained by extracting features from images of a calibration artifact. Projector image plane coordinates are calculated on the basis of continuous phase maps obtained from a fringe pattern phase reconstruction procedure. In order to stereo calibrate camera-projector system, pairs of corresponding image plane points are calculated with subpixel accuracy. In addition, one of three pattern views is used in colour calibration. RGB values of a colour field pattern detected by camera and their reference values are compared. This comparison leads to derivation of a colour transformation matrix. The performance of the proposed method is tested by measuring plane, sphere and distance reference. Also 360 degrees complex object 3D model from a set of measurements is obtained. Residual mean errors for all tests performed are calculated.

RGB-D SLAM Based on Extended Bundle Adjustment with 2D and 3D Information.

In the study of SLAM problem using an RGB-D camera, depth information and visual information as two types of primary measurement data are rarely tightly coupled during refinement of camera pose estimation. In this paper, a new method of RGB-D camera SLAM is proposed based on extended bundle adjustment with integrated 2D and 3D information on the basis of a new projection model. First, the geometric relationship between the image plane coordinates and the depth values is constructed through RGB-D camera calibration. Then, 2D and 3D feature points are automatically extracted and matched between consecutive frames to build a continuous image network. Finally, extended bundle adjustment based on the new projection model, which takes both image and depth measurements into consideration, is applied to the image network for high-precision pose estimation. Field experiments show that the proposed method has a notably better performance than the traditional method, and the experimental results demonstrate the effectiveness of the proposed method in improving localization accuracy.

2A1-Q11 McKibben型空気圧アクチュエータを用いた超音波プローブ走査機構の撮像断面の位置制御への応用(医療ロボティクス・メカトロニクス(2))

Recently, robots for ultrasound probe-control are studied to assist ultrasound diagnosis. However, the relative position between the image plane coordinates and ultrasound probe has not been considered for robot control. Thus, we propose robot control based on echogram position by using a probe scan mechanism with McKibben pneumatic actuators. The method considers image coordinates as the end effector of the mechanism. In concrete, the relative position between the image plane coordinates and an ultrasound probe was calibrated, and the image plane coordinates were controlled as the end effector of the mechanism that has six degrees of freedom by six McKibben pneumatic actuators. From phantom experiment results, this system could position the echogram coordinates and it was indicated that the proposed method is effective for support of ultrasound scanning procedures.

Research on a Single Camera Location Model and its Application

Digital cameras have been widely used in the areas of road transportation, railway transportation as well as security system. To address the position of digital camera in these fields this paper proposed a geometry calibration method based on feature point extraction of arbitrary target. Under the meaning of the questions, this paper first defines four kinds of coordinate system, that is the world coordinate system. The camera's optical center of the coordinate system is the camera coordinate system, using the same point in different coordinate system of the coordinate transformation to determine the relationship between world coordinate system and camera coordinate. And thus determine the camera's internal parameters and external parameters, available transformation matrix and translation vector indicated by the camera's internal parameters of the external parameters and the establishment of a single camera location model. According to the model, using the camera's external parameters to be on the target circle center point in the image plane coordinates.

A Distributed Active-Vision Network-Space Approach for the Navigation of a Car-Like Wheeled Robot

The navigation for a car-like wheeled robot (CLWR) in a distributed active-vision network-space system (DAVNSS) subject to three fuzzy variable-structure decentralized controls (FVSDCs) is developed. The scheme also includes trajectory tracking and obstacle avoidance. Two distributed wireless charge-coupled-device (CCD) cameras individually driven by two stepping motors, viz., active CCD1 and active CCD2 (or simply ACCD1 and ACCD2), are constructed to capture the dynamic pose of the CLWR and the obstacle. The proposed control system includes quad processors with multiple sampling rates. First, a personal computer (PC) is employed to receive the image of the CLWR or obstacle from ACCD1 or ACCD2 by a wireless transmitter and then to plan three reference commands for CLWR, ACCD1, and ACCD2. Next, a six-step image-processing routine and the calibration between the world coordinate and the image plane coordinate using multilayer perceptrons (MLPs) are established. Finally, experiments are performed to validate the proposed control system.

A patch algorithm for fast registration of distortions

This note describes an algorithm for registration of non-linear distortions in two-dimensional (2D) images. The algorithm solves an integrated form of linearized image matching equation over a set of patches in the image domain. The distortion field is represented as a linear combination of multinomials in the image-plane coordinates. Prior to registration, the images are convolved with band-pass filters that are implemented at a coarse-to-fine hierarchy of spatial scales (specified at input by the user); and some heuristics are given for choosing these scales. Images registered at a coarse scale are used as input approximations for deriving images registered at a subsequent finer scale of resolution. With this prescribed hierarchy, the algorithm is non-iterative; it is also robust to noise, and can in principle accommodate arbitrary local or global distortions. The results indicate that non-linear registration does improve the match between images.

Object Plane Deformation Due to Refraction in Two-Dimensional Underwater Motion Analysis

The purpose of this study was twofold: (a) to investigate the effects of selected experimental factors on the magnitude of the object plane deformation due to refraction, and (b) to discuss their practical implications in an effort to improve the applicability of the 2-D DLT method in the underwater motion analysis. The RMS and maximum object plane reconstruction errors of various experimental conditions were computed systematically. To isolate the error due to refraction from the experimental errors, the comparator coordinates (image plane coordinates) of the control points were computed based on a theoretical refraction model rather than actual digitizing. It was concluded from a series of object plane reconstruction that among the distance and angle factors of the experimental setting in the 2-D underwater motion analysis, the camera-to-interface distance and the interface-to-control-object distance are the two major factors affecting the magnitude of the object plane deformation. The other factors revealed only minor effects. The advantages of the 2-D DLT method over the traditional multiplier method in underwater motion analysis. such as oblique projection and multiple camera setup. were further discussed. Possible ways to reduce the maximum reconstruction error were also explored.

A method of detecting lines using local polar coordinates

The Hough transform is known to be an efficient method to detect lines from noisy images. This transform, however, implies building a two-dimensional voting array using all pixels of the image, which requires much computing time. In this context, the authors propose a method to rapidly detect lines using local polar coordinates with the pole being a point within the image; the method was proved efficient through experiments. With the proposed method, only the deviation angle is used as a parameter in the linear equations, which means that a one-dimensional voting array will suffice, and thus there is a substantial reduction of computing load. In so doing, the correspondence between the parameter plane and the image plane is transparent while the position of a line on the image plane (coordinates of the pole) is known as soon as voting is completed. Besides, short lines can be detected as well, while the amount of computation is independent of the parameter's division. In addition to these features, the proposed method does not involve processing related to lines' direction or pixels' continuity, which means robustness toward noise or image defects; thus, a new high-speed method was developed without impairing the performance of the Hough transform.

Experiments on estimating egomotion and structure parameters using long monocular image sequences

This paper presents a simple but robust model based approach to estimating the kinematics of a moving camera and the structure of the objects in a stationary environment using long, noisy, monocular image sequences. Both batch and recursive algorithms are presented and the problem due to occlusion is addressed. The approach is based on representing the constant translational velocity and constant angular velocity of the camera motion using nine rectilinear motion parameters, which are 3-D vectors of the position of the rotation center, linear and angular velocities. The structure parameters are 3-D coordinates of the salient feature points in the inertial coordinate system. Due to redundancies in parameterization, the total number of independent parameters to be estimated is 3M+7, whereM is the number of feature points. The image plane coordinates of these feature points in each frame are first detected and matched over the frames. These noisy image coordinates serve as the input to our algorithms. Due to the nonlinear nature of perspective projection, a nonlinear least squares method is formulated for the batch algorithm, and a conjugate gradient method is then applied to find the solution. A recursive method using an Iterated Extended Kalman Filter (IEKF) for incremental estimation of motion and structure is also presented. Since the plant model is simple in our formulation, closed form solutions for the state and covariance transition equations are easily derived. Experimental results for simulated imagery as well as several real image sequences are included.

Surface Profiling by Analysis of White-light Interferograms in the Spatial Frequency Domain

Abstract We describe a scanning white-light interferometer for high-precision surface structure analysis. Interferograms for each of the image points in the field of view of the instrument are generated simultaneously by scanning the object in a direction perpendicular to the object surface, while recording detector data in digital memory. These interferograms are then transformed into the spatial frequency domain and the surface height for each point is obtained by examination of the complex phase as a function of frequency. The final step is the creation of a complete three-dimensional image constructed from the height data and corresponding image plane coordinates. The measurement repeatability is better than 0·5 nm r.m.s. for a surface height range of 100 μm.

Estimating the kinematics and structure of a rigid object from a sequence of monocular images

The problem considered involves the use of a sequence of noisy monocular images of a three-dimensional moving object to estimate both its structure and kinematics. The object is assumed to be rigid, and its motion is assumed to be smooth. A set of object match points is assumed to be available, consisting of fixed features on the object, the image plane coordinates of which have been extracted from successive images in the sequence. Structure is defined as the 3-D positions of these object feature points, relative to each other. Rotational motion occurs about the origin of an object-centered coordinate system, while translational motion is that of the origin of this coordinate system. In this work, which is a continuation of the research done by the authors and reported previously (ibid., vol.PAMI-8, p.90-9, Jan. 1986), results of an experiment with real imagery are presented, involving estimation of 28 unknown translational, rotational, and structural parameters, based on 12 images with seven feature points.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Performance bounds for estimating three-dimensional motion parameters from a sequence of noisy images

The problem considered here involves the use of a sequence of monocular images of a three-dimensional moving object to estimate both its structure and kinematics. The object is assumed to be rigid, and its motion is assumed to be smooth. A set of object match points is assumed to be available, consisting of fixed features on the object, the image plane coordinates of which have been extracted from successive images in the sequence. The measured data are the noisy image plane coordinates of this set of object match points, taken from each image in the sequence. In previous papers [ IEEE Trans. Pattern Anal. Mach. Intell.PAMI-8, 90 ( 1986); in Proceedings of the IEEE Workshop on Motion: Representation and Analysis ( Institute of Electrical and Electronics Engineers, New York, 1986), p. 95; in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition ( Institute of Electrical and Electronics Engineers, New York, 1986), p. 176] we discussed model-based approaches for motion and structure estimation from a long sequence of images. We examine here the performance of such techniques by using a Cramer–Rao lower bounds on the estimation error variance. This method permits a priori prediction of estimation accuracy as a function of a number of factors, including the number of images in the sequence, the time at which each image is made, the number of feature points used, the image-plane noise level, and the type of motion that is involved. Theoretical performance predictions are compared with the statistics of Monte Carlo simulation, and it is shown that the actual estimation accuracy is close to the Cramer–Rao bounds in most cases. These results also show that noisy sequences with fewer than four images often do not contain enough information to permit accurate estimation of motion and structure parameters. This conclusion is consistent with the observed instability of so-called two-frame estimation methods in the presence of noise.

Evaluation of magnetic shear in off-disk center active regions

We analyze the changes that projection effects produce in the evaluation of magnetic shear in off-disk center active regions by comparing angular shear calculated in image plane and heliographic coordinates. We describe the procedure for properly evaluating magnetic shear by transforming the observed vector magnetic field into the heliographic system and then apply this procedure to evaluate magnetic shear along the magnetic neutral line in an active region that was observed on 1984 April 24 at a longitude offset of -45°. In particular, we show that the number of ‘critically sheared’ pixels along an east-west directed segment of the neutral line in the leader sunspot group changes from 16 in the image plane magnetogram to 14 in the heliographic magnetogram. We also show that the critical shear as calculated in the image plane served as a good predictor for the location of flaring activity since the flare ribbons of the great flare of April 24 bracketed the inversion line where the critical shear was located. These results indicate that for this particular region, projection effects did not significantly affect the evaluation of critical shear.

Fourier space separability in space-variant imaging

A modification of the equation of one-dimensional space-variant imaging is considered in which the output image is known both as a function of an image-plane coordinate and an object-position coordinate. Under this condition, the Fourier transform of the image intensity with respect to both variables equals the product of the object Fourier transform and a transfer function. A similar relation also occurs for observation of either the integrated image intensity or the intensity at a point versus an object-scan coordinate. Consequently, for the special situation considered, the transfer-function concept may be extended to the space-variant case. The implications of factorization of the transfer function are also considered.