Uncalibrated Camera Research Articles

Automatic Human Facial Expression Recognition Based on Integrated Classifier From Monocular Video with Uncalibrated Camera

An automatic recognition framework for human facial expressions from a monocular video with an uncalibrated camera is proposed. The expression characteristics are first acquired from a kind of deformable template, similar to a facial muscle distribution. After associated regularization, the time sequences from the trait changes in space-time under complete expressional production are then arranged line by line in a matrix. Next, the matrix dimensionality is reduced by a method of manifold learning of neighborhood-preserving embedding. Finally, the refined matrix containing the expression trait information is recognized by a classifier that integrates the hidden conditional random field (HCRF) and support vector machine (SVM). In an experiment using the Cohn–Kanade database, the proposed method showed a comparatively higher recognition rate than the individual HCRF or SVM methods in direct recognition from two-dimensional human face traits. Moreover, the proposed method was shown to be more robust than the typical Kotsia method because the former contains more structural characteristics of the data to be classified in space-time

3D Reconstruction from a Single Still Image Based on Monocular Vision of an Uncalibrated Camera

we propose a framework of combining Machine Learning with Dynamic Optimization for reconstructing scene in 3D automatically from a single still image of unstructured outdoor environment based on monocular vision of an uncalibrated camera. After segmenting image first time, a kind of searching tree strategy based on Bayes rule is used to identify the hierarchy of all areas on occlusion. After superpixel segmenting image second time, the AdaBoost algorithm is applied in the integration detection to the depth of lighting, texture and material. Finally, all the factors above are optimized with constrained conditions, acquiring the whole depthmap of an image. Integrate the source image with its depthmap in point-cloud or bilinear interpolation styles, realizing 3D reconstruction. Experiment in comparisons with typical methods in associated database demonstrates our method improves the reasonability of estimation to the overall 3D architecture of image’s scene to a certain extent. And it does not need any manual assist and any camera model information.

Novel projector calibration method for monocular structured light system based on digital image correlation

Monocular structured light system, which consists of one camera and one digital light processing (DLP) projector, is a simple and effective tool to acquire accurate three-dimensional (3D) shapes. But such high precision is only possible if the projector is accurately calibrated. Many of the existing projector calibration methods have complicated procedures or low calibration precision. In this paper, an accurate speckle-based feature points matching method is proposed to estimate the image coordinates of feature points in projection chip (digital micro-mirror device or DMD) plane. The method relies on an uncalibrated camera and makes use of affine transformation theory and two-dimensional digital image correlation (2D-DIC) method to reach sub-pixel matching precision. Considering the skew of the projection chip, a full-parameter calibration model is presented to describe the projector. A novel system is developed and series of validation tests were carried out to demonstrate the performance of the proposed method.

Part-aware trajectories association across non-overlapping uncalibrated cameras

This paper focuses on the problem of multi-person tracking across non-overlapping uncalibrated cameras using data association method. The problem is extremely difficult as we have very limited cues to associate persons between cameras. To tackle the problem, our system consists of firstly building multiple trajectories from each camera independently, and then finding associations of trajectories between every two cameras of interest, where the later is the most challenging process. Our contributions are mainly two folds: First, we introduce a method to explore the human part configurations on every trajectory to describe the inter-camera spatial-temporal constraints for trajectories association. Second, we formulate trajectories association across non-overlapping cameras as a multi-class classification problem via the Markov Random Field (MRF) to effectively utilize domain priors such as group activity between persons. With the proposed part-aware correspondences and pair-wise group activity constraints of trajectories, we can achieve robust multi-person tracking. Experimental results on a benchmark dataset validates the effectiveness of our proposed approach.

Visual Servo Regulation of Wheeled Mobile Robots With an Uncalibrated Onboard Camera

In this paper, a visual servo regulation strategy is designed for an uncalibrated camera system mounted on a wheeled mobile robot subject to nonholonomic motion constraint, which can drive the mobile robot to the target pose with exponential convergence. Unlike existing methods, the proposed approach can work well without the camera intrinsic parameters being known in advance. Specifically, a novel fundamental matrix-based algorithm is first proposed to rotate the robot to point toward the desired position, with the camera intrinsic parameters estimated simultaneously by employing the fundamental matrix and a projection homography matrix. Subsequently, by utilizing the obtained camera intrinsic parameters, a straight-line motion controller is developed to drive the robot to the desired position, with the orientation of the robot always facing the target position. Another pure rotation controller is finally adopted to correct the orientation error. The exponentially convergent properties of the visual servo errors are proven with mathematical analysis. The performance of the proposed uncalibrated visual servo regulation method is further validated by both simulation and experimental results.

Online multi‐person tracking with two‐stage data association and online appearance model learning

This study addresses the automatic multi‐person tracking problem in complex scenes from a single, static, uncalibrated camera. In contrast with offline tracking approaches, a novel online multi‐person tracking method is proposed based on a sequential tracking‐by‐detection framework, which can be applied to real‐time applications. A two‐stage data association is first developed to handle the drifting targets stemming from occlusions and people's abrupt motion changes. Subsequently, a novel online appearance learning is developed by using the incremental/decremental support vector machine with an adaptive training sample collection strategy to ensure reliable data association and rapid learning. Experimental results show the effectiveness and robustness of the proposed method while demonstrating its compatibility with real‐time applications.

From adaptive control to variable structure systems – seeking harmony

From adaptive control to variable structure systems – seeking harmony

3D Human Model Reconstruction from Sparse Uncalibrated Views.

Using a two-stage algorithm, the proposed technique can tackle the challenges of reconstructing high-quality 3D models of humans wearing regular clothes from sparse uncalibrated cameras. The proposed algorithm based on nonrigid dense correspondences (NRDC) requires fewer images than previous methods because it does not require an initial sparse matching. The authors validated the proposed algorithm using images from an existing dataset and images captured by a cell phone camera.

3D Reconstruction of Human Motion from Monocular Image Sequences.

This article tackles the problem of estimating non-rigid human 3D shape and motion from image sequences taken by uncalibrated cameras. Similar to other state-of-the-art solutions we factorize 2D observations in camera parameters, base poses and mixing coefficients. Existing methods require sufficient camera motion during the sequence to achieve a correct 3D reconstruction. To obtain convincing 3D reconstructions from arbitrary camera motion, our method is based on a-priorly trained base poses. We show that strong periodic assumptions on the coefficients can be used to define an efficient and accurate algorithm for estimating periodic motion such as walking patterns. For the extension to non-periodic motion we propose a novel regularization term based on temporal bone length constancy. In contrast to other works, the proposed method does not use a predefined skeleton or anthropometric constraints and can handle arbitrary camera motion. We achieve convincing 3D reconstructions, even under the influence of noise and occlusions. Multiple experiments based on a 3D error metric demonstrate the stability of the proposed method. Compared to other state-of-the-art methods our algorithm shows a significant improvement.

Evaluation of Interest Point Matching Methods for Projective Reconstruction of 3D Scenes

This work evaluates the application of different state-of-the-art methods for interest point matching, aiming the robust and efficient projective reconstruction of three-dimensional scenes. Projective reconstruction refers to the computation of the structure of a scene from images taken with uncalibrated cameras. To achieve this goal, it is essential the usage of an effective point matching algorithm. Even though several point matching methods have been proposed in the literature, their impacts in the projective reconstruction task have not yet been carefully studied. Our evaluation uses as criterion the estimated epipolar, reprojection and reconstruction errors, as well as the running times of the algorithms. Specifically, we compare five different techniques: SIFT, SURF, ORB, BRISK and FREAK. Our experiments show that binary algorithms such as, ORB and BRISK, are so accurate as float point algorithms like SIFT and SURF, nevertheless, with smaller computational cost.

Passivity‐based adaptive 3D visual servoing without depth and image velocity measurements for uncertain robot manipulators

SummaryIn this work, we consider the 3D visual tracking problem for a robot manipulator with uncertainties in the kinematic and dynamic models. The visual feedback is provided by a fixed and uncalibrated camera located above the robot workspace. The Cartesian motion of the robot end effector can be separated into a 1D motion parallel to the optical axis of the camera and a 2D motion constrained on a plane orthogonal to this axis. Thus, the control design can be simplified, and the overall visual servoing system can be partitioned in two almost‐independent subsystems. Adaptive visual servoing schemes, based on a kinematic approach, are developed for image‐based look‐and‐move systems allowing for both depth and planar tracking of a reference trajectory, without using image velocity and depth measurements. In order to include the uncertain robot kinematics and dynamics in the presented solution, we develop a cascade control strategy based on an indirect/direct adaptive method. The stability and convergence properties are analyzed in terms of Lyapunov‐like functions and the passivity‐based formalism. Numerical simulations including hardware‐in‐the‐loop results, obtained with a robot manipulator and a web camera, are presented to illustrate the performance and feasibility of the proposed control scheme. Copyright © 2016 John Wiley & Sons, Ltd.

Object Detection and Tracking-Based Camera Calibration for Normalized Human Height Estimation

This paper presents a normalized human height estimation algorithm using an uncalibrated camera. To estimate the normalized human height, the proposed algorithm detects a moving object and performs tracking-based automatic camera calibration. The proposed method consists of three steps: (i) moving human detection and tracking, (ii) automatic camera calibration, and (iii) human height estimation and error correction. The proposed method automatically calibrates camera by detecting moving humans and estimates the human height using error correction. The proposed method can be applied to object-based video surveillance systems and digital forensic.

Straightening Curved Traffic Lanes in Video

<p>Video based vehicle sensing from uncalibrated cameras provides<br />useful traffic information to modern Intelligent Transportation Systems<br />(ITS). This requires vehicle tracking and Turning Movement<br />Counts (TMCs). Traffic scenes contain vehicle motion constrained<br />along possibly curved traffic lanes which can be normalized into<br />straight lines. A simple interpolation scheme is presented which<br />projects to image sequences where vehicles always appear to move<br />straight horizontally for simplified vehicle tracking.</p>

Camera self-calibration from translation by referring to a known camera.

This paper presents a novel linear method for camera self-calibration by referring to a known (or calibrated) camera. The method requires at least three images, with two images generated by the uncalibrated camera from pure translation and one image generated by the known reference camera. We first propose a method to compute the infinite homography from scene depths. Based on this, we use two images generated by translating the uncalibrated camera to recover scene depths, which are further utilized to linearly compute the infinite homography between an arbitrary uncalibrated image, and the image from the known camera. With the known camera as reference, the computed infinite homography is readily decomposed for camera calibration. The proposed self-calibration method has been tested with simulation and real image data. Experimental results demonstrate that the method is practical and accurate. This paper proposes using a "known reference camera" for camera calibration. The pure translation, as required in the method, is much more maneuverable, compared with some strict motions in the literature, such as pure rotation. The proposed self-calibration method has good potential for solving online camera calibration problems, which has important applications, especially for multicamera and zooming camera systems.

The Exponential Stabilization of Uncertain Chained Form Systems of Mobile Robots Based on Visual Servoing

A system for a type of nonholonomic wheeled mobile robots equipped with an uncalibrated camera fixed to the ceiling is investigated. Based on the visual feedback and the state-input transformation, models of uncertain chained form systems are presented for the robot-camera systems. Then, new smooth time-varying feedback controllers are proposed to exponentially stabilize the uncertain chained system by using state-scaling and control theories for two cases. The exponential stabilities of the closed-loop uncertain systems are rigorously proved. Simulation results demonstrate the effectiveness of the proposed strategies.

Automated Mobile System for Accurate Outdoor Tree Crop Enumeration Using an Uncalibrated Camera.

This paper demonstrates an automated computer vision system for outdoor tree crop enumeration in a seedling nursery. The complete system incorporates both hardware components (including an embedded microcontroller, an odometry encoder, and an uncalibrated digital color camera) and software algorithms (including microcontroller algorithms and the proposed algorithm for tree crop enumeration) required to obtain robust performance in a natural outdoor environment. The enumeration system uses a three-step image analysis process based upon: (1) an orthographic plant projection method integrating a perspective transform with automatic parameter estimation; (2) a plant counting method based on projection histograms; and (3) a double-counting avoidance method based on a homography transform. Experimental results demonstrate the ability to count large numbers of plants automatically with no human effort. Results show that, for tree seedlings having a height up to 40 cm and a within-row tree spacing of approximately 10 cm, the algorithms successfully estimated the number of plants with an average accuracy of 95.2% for trees within a single image and 98% for counting of the whole plant population in a large sequence of images.

Online Multi-Target Tracking With Unified Handling of Complex Scenarios.

Complex scenarios, including miss detections, occlusions, false detections, and trajectory terminations, make the data association challenging. In this paper, we propose an online tracking-by-detection method to track multiple targets with unified handling of aforementioned complex scenarios, where current detection responses are linked to the previous trajectories. We introduce a dummy node to each trajectory to allow it to temporally disappear. If a trajectory fails to find its matching detection, it will be linked to its corresponding dummy node until the emergence of its matching detection. Source nodes are also incorporated to account for the entrance of new targets. The standard Hungarian algorithm, extended by the dummy nodes, can be exploited to solve the online data association implicitly in a global manner, although it is formulated between two consecutive frames. Moreover, as dummy nodes tend to accumulate in a fake or disappeared trajectory while they only occasionally appear in a real trajectory, we can deal with false detections and trajectory terminations by simply checking the number of consecutive dummy nodes. Our approach works on a single, uncalibrated camera, and requires neither scene prior knowledge nor explicit occlusion reasoning, running at 132 frames/s on the PETS09-S2L1 benchmark sequence. The experimental results validate the effectiveness of the dummy nodes in complex scenarios and show that our proposed approach is robust against false detections and miss detections. Quantitative comparisons with other methods on five benchmark sequences demonstrate that we can achieve comparable results with the most existing offline methods and better results than other online algorithms.

STAR IDENTIFICATION ALGORITHM FOR UNCALIBRATED, WIDE FOV CAMERAS

A novel method is proposed for star identification via uncalibrated cameras with wide fields of view (FOVs). In this approach some of the triangles created by the stars in the FOV are selected for pattern recognition. The triangles are selected considering the sensitivity of their interior angles to the calibration error. The algorithm is based on the intersection between sets of triangles that are found in the database for each selected triangle of the image. By this method, most of the image stars contribute to pattern recognition and thereby it is very robust against the noise and the calibration error. The algorithm is performed on 150 night sky images, which are taken by an uncalibrated camera in FOV of 114° ± 12° with a success rate of 94% and no false positives. Based on the identification approach, an adaptive method is also developed for calibrating and obtaining the projection function of an uncalibrated camera.

One global optimization method in network flow model for multiple object tracking

In this paper, we address the task of automatically tracking a variable number of objects in the scene of a monocular and uncalibrated camera. We propose a global optimization method in network flow model for multiple object tracking. This approach extends recent work which formulates the tracking-by-detection into a maximum-a posteriori (MAP) data association problem. We redefine the observation likelihood and the affinity between observations to handle long term occlusions. Moreover, an improved greedy algorithm is designed to solve min-cost flow, reducing the amount of ID switches apparently. Furthermore, a linear hypothesis method is proposed to fill up the gaps in the trajectories. The experiment results demonstrate that our method is effective and efficient, and outperforms the state-of-the-art approaches on several benchmark datasets.

Pose estimation of soccer players using multiple uncalibrated cameras

Fully automatic algorithm for estimating the 3D human pose from multiple uncalibrated cameras is presented. Unlike the state-of-the-art methods which use the estimated pose of previous frames to re...