Pre-calibrated Cameras Research Articles

IN-PROCESS PORTABLE PHOTOGRAMMETRY USING OPTICAL TARGETS FOR LARGE SCALE INDUSTRIAL METROLOGY

Portable photogrammetry methods are being used more and more as a 3D technique for large scale industrial metrology applications. Optical targets are placed on an object and images are taken around it, where measuring traceability is provided by precise off-process pre-calibrated digital cameras and scale bars. According to the 2D target image coordinates, target 3D coordinates and camera views are jointly computed. Numerical approaches such as bundle adjustment have usually been adopted for computing the 3D scene, with high computation time, leading in practice to time consuming and user dependent iterative review and re-processing measuring procedures until an adequate set of images is taken, limiting the potential of photogrammetry for fast, easy-to-use, and precise measurements. In a previous work, a new efficient computing procedure was presented for solving the bundle adjustment problem in portable photogrammetry. In this paper, further steps are presented towards increasing the computational efficiency of the in-process approach in portable photogrammetry. Measurement of raw part surface geometry prior to its machining is adopted as the industrial case for evaluation, with large scale measuring volumes (100 m3). An increase of efficiency in the in-process computing capability is demonstrated on a consumer grade desktop PC, with a total in-process computing time ranging 1 second per image for all the images taken at all the computed industrial scenarios. Additionally, a method for the self-calibration of camera and lens distortion is integrated into the in-process approach, enabling the use of low cost non-specialized digital cameras for precise industrial metrology, ranging at the same performance reported for portable photogrammetry with precise off-process pre-calibrated cameras, with a relative precision of 1/10,000 (e.g. 0.1 mm error in 1 m). Keywords: metrology; precision; machine tool; image; photogrammetry; calibration

Deep Depth from Uncalibrated Small Motion Clip.

We propose a novel approach to infer a high-quality depth map from a set of images with small viewpoint variations. In general, techniques for depth estimation from small motion consist of camera pose estimation and dense reconstruction. In contrast to prior approaches that recover scene geometry and camera motions using pre-calibrated cameras, we introduce in this paper a self-calibrating bundle adjustment method tailored for small motion which enables computation of camera poses without the need for camera calibration. For dense depth reconstruction, we present a convolutional neural network called DPSNet (Deep Plane Sweep Network) whose design is inspired by best practices of traditional geometry-based approaches. Rather than directly estimating depth or optical flow correspondence from image pairs as done in many previous deep learning methods, DPSNet takes a plane sweep approach that involves building a cost volume from deep features using the plane sweep algorithm, regularizing the cost volume, and regressing the depth map from the cost volume. The cost volume is constructed using a differentiable warping process that allows for end-to-end training of the network. Through the effective incorporation of conventional multiview stereo concepts within a deep learning framework, the proposed method achieves state-of-the-art results on a variety of challenging datasets.

Self-Calibrated In-Process Photogrammetry for Large Raw Part Measurement and Alignment before Machining.

Photogrammetry methods are being used more and more as a 3D technique for large scale metrology applications in industry. Optical targets are placed on an object and images are taken around it, where measuring traceability is provided by precise off-process pre-calibrated digital cameras and scale bars. According to the 2D target image coordinates, target 3D coordinates and camera views are jointly computed. One of the applications of photogrammetry is the measurement of raw part surfaces prior to its machining. For this application, post-process bundle adjustment has usually been adopted for computing the 3D scene. With that approach, a high computation time is observed, leading in practice to time consuming and user dependent iterative review and re-processing procedures until an adequate set of images is taken, limiting its potential for fast, easy-to-use, and precise measurements. In this paper, a new efficient procedure is presented for solving the bundle adjustment problem in portable photogrammetry. In-process bundle computing capability is demonstrated on a consumer grade desktop PC, enabling quasi real time 2D image and 3D scene computing. Additionally, a method for the self-calibration of camera and lens distortion has been integrated into the in-process approach due to its potential for highest precision when using low cost non-specialized digital cameras. Measurement traceability is set only by scale bars available in the measuring scene, avoiding the uncertainty contribution of off-process camera calibration procedures or the use of special purpose calibration artifacts. The developed self-calibrated in-process photogrammetry has been evaluated both in a pilot case scenario and in industrial scenarios for raw part measurement, showing a total in-process computing time typically below 1 s per image up to a maximum of 2 s during the last stages of the computed industrial scenes, along with a relative precision of 1/10,000 (e.g., 0.1 mm error in 1 m) with an error RMS below 0.2 pixels at image plane, ranging at the same performance reported for portable photogrammetry with precise off-process pre-calibrated cameras.

Rojective distortion correction algorithm at low altitude photographing

An algorithm for projective geometric distortion correction of images at low altitude shooting with a virtual camera is considered. The algorithm is based on the orientation of the virtual camera so that its optical axis is collinear with the normal to the plane of shooting and evaluating a homography matrix linking the coordinates of corresponding pixels of the real and virtual cameras. The homography matrix evaluation is possible based on information from additional sensors – an angular orientation sensor and a rangefinder, or altimeter. Their signals allow one to estimate the camera angular orientation and the distance to the observed plane. The impact of the angular orientation error, estimated using a triaxial microelectromechanical accelerometer, on a criterion-based correction quality is investigated. The experimental results with pre-calibrated cameras Logitech C270 and uEye RE 5240 show that when the roll and pitch errors of the camera orientation are no more than 1° then the correction algorithm with the virtual camera provides a quality index value of at least 0.97.

Estimation of optimal pivot point for remote center of motion alignment in surgery.

The determination of an optimal pivot point ([Formula: see text]) is important for instrument manipulation in minimally invasive surgery. Such knowledge is of particular importance for robotic-assisted surgery where robots need to rotate precisely around a specific point in space in order to minimize trauma to the body wall while maintaining position control. Remote center of motion (RCM) mechanisms are commonly used, where the RCM point is manually and visually aligned. If not positioned appropriately, this misalignment might lead to intolerably high forces on the body wall with increased risk of postoperative complications or instrument damage. An automated method to align the RCM with the [Formula: see text] was developed and tested. Computer vision and a lightweight calibration procedure are used to estimate the optimal pivot point. One or two pre-calibrated cameras viewing the surgical scene are employed. The surgeon is asked to make short pivoting movements, applying as little torque as possible, with an instrument of choice passing through the insertion point while camera images are being recorded. The physical properties of an instrument rotating around a pivot point are exploited in a random sample consensus scheme to robustly estimate the ideal position of the RCM in the image planes. Triangulation is used to estimate the RCM position in 3D. Experiments were performed on a specially designed mockup to test the method. The position of the pivot point is estimated with an average error less than 1.85 mm using two webcams placed from approximately 30 cm to 1 m away from the scene. The entire procedure was completed in a few seconds. In automated method to estimate the ideal position of the RCM was shown to be reliable. The method can be implemented within a visual servoing approach to automatically place the RCM point, or the results can be displayed on a screen to provide guidance to the surgeon. Further work includes the development of an image-guided alignment method and validation with in vivo experiments.

Video sensor network for real-time traffic monitoring and surveillance

Sensor networks and associated infrastructures become ever more important to the traffic monitoring and control because of the increasing traffic demands in terms of congestion and safety. These systems allow authorities not only to monitor the traffic state at the detection sites, but also to obtain real-time related information (e.g. traffic loads). This study presents a real-time vision system for automatic traffic monitoring based on a network of autonomous tracking units (ATUs) that capture and process images from one or more pre-calibrated cameras. The proposed system is flexible, scalable and suitable for a broad field of applications, including traffic monitoring of tunnels at highways and aircraft parking areas at airports. Another objective of this work is to test and evaluate different image processing and data fusion techniques in order to be incorporated to the final system. The output of the image processing unit is a set of information for each moving object in the scene, such as target ID, position, velocity and classification, which are transmitted to a remote traffic control centre, with remarkably low bandwidth requirements. This information is analysed and used to provide real-time output (e.g. alerts, electronic road signs, ramp meters etc.) as well as to extract useful statistical information (traffic loads, lane changes, average velocity etc.).

Image sequence based automatic multi-camera system calibration techniques

An image sequence-based, fully automatic and rather flexible procedure for the calibration of stationary multi-camera systems for 3-D observation of dynamic events is presented and analysed. While conventional close-range camera calibration techniques are either based on a stable point-field with known reference coordinates or on a temporarily stationary point-field with only approximately known 3-D coordinates, which is imaged from different locations and under different orientations with one single camera, the presented technique is based on stationary cameras and moving targets, making use of the image sequence acquisition nature of most solid state cameras. In the simplest version, only one easily detectable marker has to be tracked through image sequences of multiple pre-calibrated cameras, thus avoiding the necessity of homologous feature identification for the establishment of multi-view correspondences; 3-D coordinates of the marker position are not required. This single-marker method does not allow for the determination of the interior orientation. In an extended version allowing for full camera orientation and calibration, a reference bar of known length is moved through object space, with the problem of feature identification and establishment of multi-view correspondences being reduced to the tracking of two targets. The method can only be used with multi-camera systems and is most useful for 3-D motion analysis applications, but may be adapted to a wide range of other applications. The advantages of the method over conventional self-calibration techniques are the trivial establishment of multi-view correspondences, the fact that no temporarily stable target field has to be constructed, and the fact that each camera has to be set up only once. After an explanation of the technique, its performance is examined in detail based on extensive computer simulations, and the practical effectiveness is shown in a pilot study on industrial robot calibration. Based on these studies, recommendations are given concerning the number of reference bar locations, preferable reference bar orientation schemes and the achievable accuracy potential.