Perspective Projection Camera Research Articles

Blind 3D Video Stabilization with Spatio-Temporally Varying Motion Blur

Video stabilization is a challenging task that attempts to compensate for the overall frame shake during video acquisition. Existing three-dimensional video stabilization methods aim at modeling camera perspective projection through either data-driven training or explicit motion estimation. However, the above methods are difficult to effectively solve the issue of shaky videos with abrupt object movements, resulting in local motion blur in the direction of the movement. This phenomenon is prevalent in real-world scenarios featuring foreground blind motion scenes. Unfortunately, directly combining stabilization and deblurring methods poses challenges when dealing with this situation. In the video, the intensity of motion blur undergoes continuous changes, and the direct combination method inadequately utilizes spatiotemporal information, providing insufficient clues for cross-frame compensation. To alleviate this problem, the Cross-frame-temporal Module framework is proposed to address blind motion blur induced by various conditions, which utilizes cross-frame temporal features to estimate depth maps and camera motion. In this framework, a Blur Transform Network (BTNet) is designed to adapt to spatially varying motion blur, which transforms local regions according to the impact of blur intensities to adapt to the effects of non-uniform motion blur; furthermore, our Temporal-Aware Network (TANet) further suppresses motion blur by leveraging cross-frame temporal features. In addition, the limited availability of pair-training video data containing motion blur limits the application of this approach in practice. The Cross-frame-temporal Module framework adopts an un-pretrained in-test training strategy. Extensive experimental results have demonstrated that our method outperforms state-of-the-art methods.

Робота системи технічного зору по елементах поверхні космічного апарата в умовах її неповної видимості

Вирішується задача відстеження переміщення реперних об’єктів на зображеннях, що отримуються системою технічного зору, призначеною для вимірювання відносного положення і орієнтації некооперованих космічних апаратів на малих відстанях у складі системи забезпечення автоматичного стикування. Дана задача виникає при формуванні вихідних даних для так званої задачі PnP, в якій положення та орієнтація тіла у просторі визначаються за зображеннями контрольних точок поверхні цього тіла, отриманими камерою перспективної проєкції. При цьому положення контрольних точок відносно системи координат цільового тіла мають бути відомими. Оскільки цільовим тілом виступає абстрактний некооперований космічний апарат, для визначення контрольних точок можуть використовуватися лише наявні видимі неоднорідності його поверхні. Тому виникає запитання, як вибирати контрольні точки, щоб вони були прив’язані до поверхні тіла і могли бути знайдені з високою точністю на зображеннях за різних зовнішніх умов зйомки. Одним із варіантів вирішення є використання для прив’язки дрібнорозмірних конструктивних елементів некооперованого космічного апарата, які з великою ймовірністю будуть присутні на його поверхні. Тоді прив’язка контрольної точки здійснюється до відомої форми поверхні відповідного елемента, а визначення її положення на зображенні виконується через розпізнавання відповідного образу. У даній статті розглядається випадок використання системи технічного зору з монокулярною камерою. Розглядається етап роботи на близьких відстанях, коли в полі зору камери знаходиться ділянка поверхні стикування з розміщеними на ній елементами, що використовуються як реперні. Запропоновано метод знаходження на зображеннях положення точок прив’язки окремого класу реперних об’єктів при виконанні періодичної зйомки.

An improved projector calibration method for structured-light 3D measurement systems

In a structured-light three-dimensional measurement system, understanding the optical configuration of the projector and suppressing the eccentricity error caused by the camera perspective projection are critical to realize high-precision measurement. In this paper, we analyze the special offset optical structure in commercial projectors, where a larger diameter lens is used to ensure the quality of the projected image, and the position of the principal point has been shifted. Meanwhile, a projector calibration strategy that makes the camera’s optical axis perpendicular to the target plane is proposed to avoid the pollution of perspective projection distortion. The sub-pixel correspondence based on homography transformation is performed by relying on the phase value of the fringe, and bundle adjustment optimization is used to further improve the accuracy and robustness of projector calibration. The experimental results demonstrate that the proposed calibration method can improve the calibration accuracy by 52.52%.

A standard expression of underwater binocular vision for stereo matching

A camera in an underwater stereo vision system conforms to a multi-layer refraction model, and the perspective projection camera model is invalid. Because the imaging light is refracted, the corresponding pixel does not conform to the traditional polar constraint, and the stereo matching methods must conduct a two-dimensional search. In this paper, a standard expression is proposed to reduce the search space to one dimension. Based on a light field, any specific underwater binocular stereo vision is converted to a standard setup in which the axes of two cameras are parallel and the image is expressed as the light field, the direction information image and the position image. We found and proved that the direction information image conforms to the rule of corresponding rows. With this standard expression, the stereo matching is efficiently conducted and the 3D information can be computed from the disparity map and the position image. In the experiment, an underwater binocular stereo vision system is developed and the proposed method is compared with the traditional method. The results confirm that the proposed method has several advantages of conciseness, correctness, efficiency, and generality.

3D Human Pose Estimation via Explicit Compositional Depth Maps

In this work, we tackle the problem of estimating 3D human pose in camera space from a monocular image. First, we propose to use densely-generated limb depth maps to ease the learning of body joints depth, which are well aligned with image cues. Then, we design a lifting module from 2D pixel coordinates to 3D camera coordinates which explicitly takes the depth values as inputs, and is aligned with camera perspective projection model. We show our method achieves superior performance on large-scale 3D pose datasets Human3.6M and MPI-INF-3DHP, and sets the new state-of-the-art.

Design of Spatial Posture Measurement System Based on Camera Perspective Projection Model

Aiming at the requirement of high precision, fast speed and high safety and reliability in ground zero gravity test of space mechanism, the design principle and system composition of space position and attitude measurement system based on camera perspective projection model are elaborated in detail, which effectively reduces the hardware failure rate and meets the requirements of space mechanism products. Interview test requirements.

An Improved Pose Estimation Method Based on Projection Vector With Noise Error Uncertainty

Aiming at the problem of anomalous and non-independent distribution of the image errors in the feature-based visual pose estimation, a method of monocular visual pose estimation based on the uncertainty of noise error established by projection vector is proposed. First, by using the covariance matrix to describe the uncertainty of the feature point direction and integrating the uncertainty of the feature point direction into the pose estimation, characteristic point measurement error with different degrees of directional uncertainty can be adapted that can makes the algorithm robust. Then, by introducing the projection vector and combining the depth information of each feature point to represent the collinearity error, the model nonlinear problem caused by the camera perspective projection can be eliminated that can make the algorithm have global convergence. Finally, we use global convergence theorem to prove the global convergence of the proposed algorithm. The results show that the proposed method has good robustness and convergence while adapting to different degrees of error uncertainty, which can meet practical engineering applications.

A projective chirp based stair representation and detection from monocular images and its application for the visually impaired

• Introducing a projective chirp model for staircasess structure representation. • Proposing an iterative algorithm for staircases detection from monocular images. • Validating performances of the proposed method on benchmark and artificial datasets. • Deployment of a staircase alarm system supporting blind people in indoor environment. The most prominent characteristic of a stair is that it has rigid form with periodic pattern of its steps. In this work, we exploit this periodic characteristic in view of geometrical rules. As a stair consists of equidistant nosing lines, under a perspective projection of camera, the projection of these lines on an image follows a projective chirplet transform. We propose to detect a stair by finding a group of lines that best satisfies a projective chirp model. The most advantage of the proposed techniques is that some missed noising lines and thus whole stair could be recovered. We validate the proposed method on both artificial and real datasets. The experimental results show a higher detection rate on different datasets. Finally, a real application alarming the visually impaired about stairs in indoor environments has been conducted and obtained 88.37% of accuracy. The implementations and datasets are made publicly available.

Analysis of Fish-Eye Lens Camera Self-Calibration

The fish-eye lens camera offers the advantage of efficient acquisition of image data through a wide field of view. However, unlike the popular perspective projection camera, a strong distortion effect appears as the periphery of the image is compressed. Such characteristics must be precisely analyzed through camera self-calibration. In this study, we carried out a fish-eye lens camera self-calibration while considering different types of test objects and projection models. Self-calibration was performed using the V-, A-, Plane-, and Room-type test objects. In the fish-eye lens camera, the V-type test object was the most advantageous for ensuring the accuracy of the principal point coordinates and focal length, because the correlations between parameters were relatively low. On the other hand, the other test objects were advantageous for ensuring the accuracy of distortion parameters because of the well-distributed image points. Based on the above analysis, we proposed, an accurate fish-eye lens camera self-calibration method that applies the V-type test object. The RMS-residuals of the proposed method were less than 1 pixel.

Vision-Based Calibration of Dual RCM-Based Robot Arms in Human-Robot Collaborative Minimally Invasive Surgery

This letter reports the development of a vision-based calibration method for dual remote center-of-motion (RCM) based robot arms in a human-robot collaborative minimally invasive surgery (MIS) scenario. The method does not require any external tracking sensors and directly uses images captured by the endoscopic camera and the robot encoder readings as calibration data, which leads to a minimal and practical system in the operating room. By taking advantage of the motion constraints imposed by the RCM-based kinematics of the robotic surgical tools and cameras, we can find unique relationships between the endoscope and the surgical tool using camera perspective projection geometry without the geometric information of the tool. A customized vision-based centerline detection algorithm is also proposed, which provides robust estimation of centerline positions for a variety of settings. We validate the method through simulations and an experimental study in a simulated MIS scenario, in which the first generation da Vinci surgical system controlled by the open source da Vinci research kit electronics and cisst/SAW software environment is used.

Sparse Line-Optical Flow Field Computing Method Based on Lines Matching

This paper brings out a new sparse line-optical flow field computing method. Firstly, we establish a lines matching algorithm based on Kalman Filter (KF). In this algorithm, we map lines in an image into Hough space, after that we employ KF to predict the position in the following frame in order to match lines in image sequence. Secondly, we present the concept of sparse line-optical flow field of images and propose the calculation method of it. By using the camera perspective projection model and the optical flow Identity, we can get the sparse line-optical flow field. Simulations is made in the following step, and results show that the lines matching algorithm works well and the accuracy of the calculation method proposed in this paper is as good as that of the classic Horn algorithm, while the calculating time-cost of it is only 1/30 Horn algorithms.

High-speed measurement algorithm for the position of holes in a large plane

CMM is widely used to measure the position of the holes on the top surface of an engine block. However, CMM requires a perfect environment and cannot be applied in online measurements. Moreover, using CMM to measure the position of holes in a large plane takes more than 10min, thus lowering its efficiency. To solve this problem, this paper presents a high-speed measurement algorithm for the position of holes in a large plane based on a flexible datum and the feature neighborhood model. First, two area CCD cameras that grab the images of the reference holes of the block are used to establish the flexible datum. Second, different mapping models are built in the neighborhood of the center of different holes. These black-box mapping models ignore the intermediate process of camera perspective projection. Finally, the feature points in the different scales of the neighborhood of the hole centers solve the mapping results. The mapping results are then weighted by using the multi-scale weighting algorithm. The calibration target is designed, but the feature points in the target are minimal. Thus, a new method to create several more feature points is designed. Compared with the measurement result of CMM, the maximum position error of the measurement system is 0.025mm. The relative error is better than 0.025% and the standard deviation of the measurement data is less than 0.010mm. With a conference level of 95%, the system measurement uncertainty is better than ±0.020mm. The measuring time is less than 3min. The position measurement scheme features high automation and high efficiency, and can be used in the position measurement of online engine block holes.

Rectification of Perspective Text Images on Rectangular Planes

Natural images often contain useful information about the scene such as text or company logos placed on a rectangular shaped plane. The 2D images captured from such objects by a camera are often distorted, because of the effects of the perspective projection camera model. This distortion makes the acquisition of the text information difficult. In this study, we detect the rectangular object on which the text is written, then the image is restored by removing the perspective distortion. The Hough transform is used to detect the boundary lines of the rectangular object and a bilinear transformation is applied to restore the original image.

Improved robust and accurate camera calibration method used for machine vision application

Camera calibration is an important step for vision-based measurement applications. A well-known flexible camera calibration method is analyzed that uses the checkerboard pattern plane and in which the camera can be moved freely. When using a perspective projection camera model, characteristics of both the objective plane and the image plane are utilized and accurate results can be obtained. However, the method's results may fail when the rotation angles of the planar pattern are small, and the distortion coefficients obtained under the perspective projection model can not be used for a real-time vision application. We solve the ill-conditioned equations using the genetic algorithm, and the correct camera parameters are always obtained. We compute the distortion coefficients of the inverse projection model, which can be used for general vision applications. The influence of the corner detection precision is taken into consideration. Simulation shows that the best results may be obtained when the planar pattern is placed in a close range and its rotation angle is small. Simulations and real-world experiments illustrate that the improved calibration algorithm can always obtain robust and accurate results.

EM enhancement of 3D head pose estimated by point at infinity

Head pose estimation is a key task for visual surveillance, HCI and face recognition applications. In this paper, a new approach is proposed for estimating 3D head pose from a monocular image. The approach assumes the full perspective projection camera model. Our approach employs general prior knowledge of face structure and the corresponding geometrical constraints provided by the location of a certain vanishing point to determine the pose of human faces. To achieve this, eye-lines, formed from the far and near eye corners, and mouth-line of the mouth corners are assumed parallel in 3D space. Then the vanishing point of these parallel lines found by the intersection of the eye-line and mouth-line in the image can be used to infer the 3D orientation and location of the human face. In order to deal with the variance of the facial model parameters, e.g. ratio between the eye-line and the mouth-line, an EM framework is applied to update the parameters. We first compute the 3D pose using some initially learnt parameters (such as ratio and length) and then adapt the parameters statistically for individual persons and their facial expressions by minimizing the residual errors between the projection of the model features points and the actual features on the image. In doing so, we assume every facial feature point can be associated to each of features points in 3D model with some a posteriori probability. The expectation step of the EM algorithm provides an iterative framework for computing the a posterori probabilities using Gaussian mixtures defined over the parameters. The robustness analysis of the algorithm on synthetic data and some real images with known ground-truth are included.

Projecting registration error for accurate registration of overlapping range images

In this paper, we propose a novel algorithm for the automatic registration of two overlapping range images. Since it is relatively difficult to compare the registration errors of different point matches, we project them onto a virtual image plane for more accurate comparison using the classical pin-hole perspective projection camera model. While the traditional ICP algorithm is more interested in the points in the second image close to the sphere centred at the transformed point, the novel algorithm is more interested in the points in the second image as collinear as possible to the transformed point. The novel algorithm then extracts useful information from both the registration error and projected error histograms for the elimination of false matches without any feature extraction, image segmentation or the requirement of motion estimation from outliers corrupted data and, thus, has an advantage of easy implementation. A comparative study based on real images captured under typical imaging conditions has shown that the novel algorithm produces good registration results.

Structure from Motion Using Sequential Monte Carlo Methods

In this paper, the structure from motion (SfM) problem is addressed using sequential Monte Carlo methods. A new SfM algorithm based on random sampling is derived to estimate the posterior distributions of camera motion and scene structure for the perspective projection camera model. Experimental results show that challenging issues in solving the SfM problem, due to erroneous feature tracking, feature occlusion, motion/structure ambiguity, mixed-domain sequences, mismatched features, and independently moving objects, can be well modeled and effectively addressed using the proposed method.

Calibration, pose and novel views from single images of constrained scenes

We exploit the common constraint of having a right-angle corner of two rectangular planes in the scene in order to calibrate a perspective projection camera and compute its pose relative to the coordinate system defined by the corner. No metric information about the corner is assumed. The camera is constrained to have its image x- and y-axes to be orthogonal with the same scale factor, which is valid for most real-world cameras. We then reproject the image of the corner to an arbitrary viewpoint. We can also compute the metric properties of the scene to scale. We report experimental results with subjectively acceptable quality. The approach shows the power of exploiting constraints that are abundant in typical architectural scenes.

A lie theoretic approach to structure and motion in computer vision 1

There has been an increasing interest in applying computer vision in robot control such as vision guided navigation, manipulation, and object recognition. A combination of a robot's rigid body motion and a camera's perspective projection has brought us a new geometric subject to study, the so called multiview geometry in computer vision literature. In this paper, we propose a new approach to this subject based on mathematical tools commonly used in robotics such as linear algebra, differential geometry and Lie group theory. Based on the geometric model of a vision system, main results of multiview geometry are outlined, including reconstruction theory of camera motion, scene structure and camera self-calibration. Potential applications of multiview geometry in robot control are also presented.