Camera Calibration Technique Research Articles

Automatic welding seam tracking and real-world coordinates identification with machine learning method

In the automatic welding process, the precision of the welding position significantly influences welding quality, relying heavily on the accurate identification of the welding seam. However, in complex and unstructured welding environments, such as those with strong arc lights, welding splashes, and thermal-induced deformations, various disturbances present a significant challenge to identify the welding seam. This paper introduces a cost-effective welding seam tracking model constructed using a laser line and a fixed viewpoint single RGB camera. The model's primary objective is to detect a chain of welding-seam points in both captured images and real-time camera feeds. The paper presents the techniques for camera calibration, laser detection, and the use of artificial intelligence to establish a real-world coordinate frame. The process begins with camera calibration using the OpenCV library to filter noise and undistort images. Subsequently, the thresholding technique is employed for color image segmentation, isolating the laser sensor light on the workpiece. An algorithm is then introduced to detect welding spots. Finally, the image coordinates are converted to world coordinates using two machine learning methods: Random Forest Regression and Gradient Boosting Regression. Experimental demonstrations were conducted at different positions on the object to assess algorithm precision. The results indicate average deviations of 0.83 mm, 0.74 mm, and 0.77 mm for X, Y, and Z coordinates, respectively, which fall below the expected standard deviation of 3 mm. Despite the overall consistency of the program, slight noise from environmental lighting may still exist. In conclusion, the proposed welding seam tracking method and coordinate conversion program have proven effective and cost-efficient, with potential for future improvements.

Research and Implementation of Robotic Arm Positioning and Grasping Based on Visual Guidance

Abstract This paper proposes a robotic arm vision guidance system based on the ROS platform, which combines the Hikrobot Robotics 2D vision sensor with the Xinsong GCR5 robotic arm to successfully realise vision-guided robotic arm positioning and grasping. Camera calibration, hand-eye calibration, and deep learning techniques are used to establish an accurate transformation relationship between the base coordinate system of the robotic arm and the camera coordinate system to achieve efficient target goal recognition and localisation. Adopting the MoveIt framework and RRT-connect algorithm, the robotic arm can flexibly plan the movement path and successfully complete the target grasping, which provides the theoretical basis and practical experience for robotic arm intelligent grasping.

An Advanced Solution Based on Machine Learning for Remote EMDR Therapy

For this work, a preliminary study proposed virtual interfaces for remote psychotherapy and psychology practices. This study aimed to verify the efficacy of such approaches in obtaining results comparable to in-presence psychotherapy, when the therapist is physically present in the room. In particular, we implemented several joint machine-learning techniques for distance detection, camera calibration and eye tracking, assembled to create a full virtual environment for the execution of a psychological protocol for a self-induced mindfulness meditative state. Notably, such a protocol is also applicable for the desensitization phase of EMDR therapy. This preliminary study has proven that, compared to a simple control task, such as filling in a questionnaire, the application of the mindfulness protocol in a fully virtual setting greatly improves concentration and lowers stress for the subjects it has been tested on, therefore proving the efficacy of a remote approach when compared to an in-presence one. This opens up the possibility of deepening the study, to create a fully working interface which will be applicable in various on-field applications of psychotherapy where the presence of the therapist cannot be always guaranteed.

Stereovision-based vibration measurement of stay cable using synchronized multi-camera setup and video motion magnification

This paper proposes a new framework for 3D noncontact stereovision-based displacement measurement in conjunction with Phase-based Video Motion Magnification (PVMM) to capture in-plane and out-of-plane vibrations of stay cables. In the proposed method, cable vibrations are captured by a pair of synchronized digital cameras, then PVMM method is applied to magnify micro vibrations of the cable (ambient responses) recorded by video cameras. Next, a camera calibration technique is proposed to convert the 2D image coordinates to 3D global coordinates, followed by the stereo calibration between the two cameras to synchronize the target points. Afterwards, a Centroid-based Bounding-Box Tracking (CBBT) technique is proposed and used to track the movements of the cable in each video frame. As a result, time-history displacements of the cable are extracted from the magnified videos. Finally, natural frequencies and damping ratios of the cable are extracted from the in-plane and out-of-plane displacements using Wavelet-screening Random Decrement Technique (WS-RDT). The proposed methodology has been validated through a couple of wind-tunnel tests of cable vibrations. The results have demonstrated a good agreement between the proposed method and conventional accelerometer-based measurement results.

Research on conflict detection model for taxi‐in process on the apron based on aircraft wingtip keypoint detection

AbstractApron safety is critical to aviation operations and aprons are prone to wingtip scraping accidents. However, few studies have been conducted on wingtip conflicts. In this paper, a class of aircraft conflict detection models based on wingtip keypoint detection is devised to solve the problem of wingtip scraping that may occur during the aircraft taxi‐in process on the apron. Firstly, a concise camera calibration technique is designed for the conversion between the pixel coordinate and the airport actual coordinate, which allows for the acquisition of high accuracy activity posture within the apron. Secondly, the static and dynamic safety protection zone delineation methods for wingtip conflict detection are given, taking the apron standard operating procedures and aircraft kinematic model into consideration. Then, the corresponding conflict detection models and algorithms are designed for the possible scraping between the vehicle and aircraft wingtips and between the aircraft and aircraft wingtips during the aircraft taxi‐in process on the apron. Finally, HRNet is selected to detect the keypoint of the wingtip by comparison, and the corresponding models and algorithm validation are carried out based on the apron physical sandbox. The verification results indicate that the models have strong accuracy and real‐time performance.

A New Camera Calibration Technique for Serious Distortion

A new camera calibration technique based on serious distortion is proposed, which only requires the camera to observe the plane pattern in an arbitrary azimuth. It uses the geometrical imaging principle and radial distortion model to acquire radial lens distortion coefficient and the image coordinate (u0, v0), and then solves the linear equation aiming at the other parameters of the camera. This method has the following characteristics: Firstly, the position of the camera and the plane is arbitrary, and the technique needs only a single observation for plane pattern. Secondly, it is suitable for camera calibration with serious distortion. Thirdly, it does not need expensive ancillary equipment, accurate movement, or lots of photos observed from different orientations. Having been authenticated by computer emulation and actual experiment, the results of the proposed technique have proved to be satisfactory. The research has also paved a new way in camera calibration for further studies.

A computer vision framework using Convolutional Neural Networks for airport-airside surveillance

Modern airports often have large and complex airside environments featuring multiple runways, with changing configurations, numerous taxiways for effective circulation of flights and tens, if not hundreds, of gates. With inherent uncertainties in gate push-back and taxiway routing, efficient surveillance and management of airport-airside operations is a highly challenging task for air traffic controllers. An increase in air traffic may lead to gate delays, taxiway congestion, taxiway incursions as well as significant increase in the workload of air traffic controllers. With the advent of Digital Towers, airports are increasingly being equipped with surveillance camera systems. This paper proposes a novel computer vision framework for airport-airside surveillance, using cameras to monitor ground movement objects for safety enhancement and operational efficiency improvement. The framework adopts Convolutional Neural Networks and camera calibration techniques for aircraft detection and tracking, push-back prediction, and maneuvering monitoring. The proposed framework is applied on video camera feeds from Houston Airport, USA (for maneuvering monitoring) and Obihiro Airport, Japan (for push-back prediction). The object detection models of the proposed framework achieve up to 73.36% average precision on Houston airport and 87.3% on Obihiro airport. The framework estimates aircraft speed and distance with low error (up to 6 meters), and aircraft push-back is predicted with an average error of 3 min from the time an aircraft arrives with the error-rate reducing until the aircraft’s actual push-back event.

Comparison of Camera Calibration and Measurement Accuracy Techniques for Phase Measuring Deflectometry

Phase measuring deflectometry (PMD) is a competitive method for specular surface measurement that offers the advantages of a high dynamic range, non-contact process, and full field measurement; furthermore, it can also achieve high accuracy. Camera calibration is a crucial step for PMD. As a result, a method based on the calibration of the entrance pupil center is introduced in this paper. Then, our proposed approach is compared with the most popular photogrammetric method based on Zhang’s technique (PM) and Huang’s modal phase measuring deflectometry (MPMD). The calibration procedures of these three methods are described, and the measurement errors introduced by the perturbations of degrees of freedom in the PMD system are analyzed using a ray tracing technique. In the experiment, a planar window glass and an optical planar element are separately measured, and the measurement results of the use of the three methods are compared. The experimental results for the optical planar element (removing the first 6 terms of the Zernike polynomial) show that our method’s measurement accuracy reached 13.71 nm RMS and 80.50 nm PV, which is comparable to accuracy values for the interferometer.

Directionally Decomposing Structured Light for Projector Calibration.

Intrinsic projector calibration is essential in projection mapping (PM) applications, especially in dynamic PM. However, due to the shallow depth-of-field (DOF) of a projector, more work is needed to ensure accurate calibration. We aim to estimate the intrinsic parameters of a projector while avoiding the limitation of shallow DOF. As the core of our technique, we present a practical calibration device that requires a minimal working volume directly in front of the projector lens regardless of the projector's focusing distance and aperture size. The device consists of a flat-bed scanner and pinhole-array masks. For calibration, a projector projects a series of structured light patterns in the device. The pinholes directionally decompose the structured light, and only the projected rays that pass through the pinholes hit the scanner plane. For each pinhole, we extract a ray passing through the optical center of the projector. Consequently, we regard the projector as a pinhole projector that projects the extracted rays only, and we calibrate the projector by applying the standard camera calibration technique, which assumes a pinhole camera model. Using a proof-of-concept prototype, we demonstrate that our technique can calibrate projectors with different focusing distances and aperture sizes at the same accuracy as a conventional method. Finally, we confirm that our technique can provide intrinsic parameters accurate enough for a dynamic PM application, even when a projector is placed too far from a projection target for a conventional method to calibrate the projector using a fiducial object of reasonable size.

A Technology of Surface Defects for the Solar Wafer by Dual Vision

For overcoming the defects of the existing method of surface defect for the solar wafer.A automatic measurement technology by dual vision is presented. Frist the principle and flow of the automatice measurement technology are introduced. Second the technique of camera calibration, technique of center extraction of edge and technique of automatic matching are researched. At the end, using the technology realized the rapid defects detection including crack, cavern, smudge and broken so on and so forth which can be concluded that the application of this technology helps to enhance the inspection efficiency.

Identification of geometric parameters of a parallel robot by using a camera calibration technique

This work reports a novel method to estimate the geometrical parameters of a 2-(3-RRPS) parallel robot intended for manufacturing tasks. The method uses camera calibration techniques, and it is based on the concept of vertex space. The advantage of this technique is that the system does not require complex electronic instrumentation, and only uses a CCD camera as a main sensor and planar patterns, which makes it portable, accurate and low cost. To ensure the quality of the measurements, a methodology for characterization of the measurement system is included. The applicability and the advantages of using the proposed method are shown by means of the estimation of the geometrical dimensions of a spatial parallel manipulator with a relatively complex kinematic architecture. Experiments are conducted and show a significant improvement in manipulator accuracy when the parameters estimated with this technique are used.

Camera calibration based on color-coded phase-shifted fringe

Aiming at the low adaptability of blurring noise of target feature points in traditional calibration methods, a calibration method based on the color-coded phase-shifted fringe is proposed. Using a liquid crystal display panel as the calibration target, horizontal and vertical color-coded phase-shifted stripes are displayed in sequence; the orthogonal phase-shifted stripes are obtained by separating color channels; based on the phase-shifteg theory, the intersections of the orthogonal phase truncation lines are calculated as the feature points. After changing the target position multiple times and extracting feature points, the plane-based camera calibration technique is applied to realize the calibration of both the single camera and the binocular system. Furthermore, color-coded phase-shift circles are added to four corners of the target pattern to automatically extract and sort feature points. Accordingly, the efficiency of calibration is promoted. The experimental results indicate that when the target image is blurred, the reprojection error of the single-camera calibration is 0.15 pixels, and the standard deviation of the binocular system measurement after calibration is 0.1 mm.

A Tool for Annotating Homographies from Hockey Broadcast Video

In order to develop solutions for automatic ice rink localization from broadcast video, a dataset with ground truth homographies is required. Hockey broadcast video does not tend to provide camera parameters for each frame, which means that they must be gathered manually. A novel tool for collecting ground truth transforms through point correspondences between each frame and an overhead view of the ice rink is presented in this paper. Through collaboration with the users of the tool, we have added features to improve accuracy and efficiency, especially in frames with few lines on the playing surface visible. A dataset of 4,262 frames has been collected, which will be used for research into automatic camera calibration techniques.

Geometry-Based Camera Calibration Using Closed-Form Solution of Principal Line.

Camera calibration is a crucial prerequisite in many applications of computer vision. In this paper, a new geometry-based camera calibration technique is proposed, which resolves two main issues associated with the widely used Zhang's method: (i) the lack of guidelines to avoid outliers in the computation and (ii) the assumption of fixed camera focal length. The proposed approach is based on the closed-form solution of principal lines with their intersection being the principal point while each principal line can concisely represent relative orientation/position (up to one degree of freedom for both) between a special pair of coordinate systems of image plane and calibration pattern. With such analytically tractable image features, computations associated with the calibration are greatly simplified, while the guidelines in (i) can be established intuitively. Experimental results for synthetic and real data show that the proposed approach does compare favorably with Zhang's method, in terms of correctness, robustness, and flexibility, and addresses issues (i) and (ii) satisfactorily.

Low-cost experimental apparatus for motion tracking based on image processing and camera calibration techniques

The present work focuses on the use of a low-cost equipment to monitor static and dynamic tests as an alternative to expensive commercial devices. First, we describe a procedure for cameras calibration and image processing in order to establish a less invasive (than other common instrumentation, such as strain gauges) and much cheaper motion capture method (than commercial tracking systems). Following, two cameras and one personal computer (with an image acquisition board) are used as the monitoring apparatus for tracking the movements of a flexible cable under harmonic excitation at its top end. The experimental results are then compared with numerical simulations, showing a fairly good agreement and the same level of precision as that obtained with one of the most used motion capture system in laboratories. The proposed experimental methodology correctly identifies the displacements, frequencies, and dynamic geometric behavior of the flexible model in all directions. Although commercial solutions are faster, since data processing and cameras calibration take place in real time, the acquisition cost of the suggested equipment is most affordable for small industries, educational institutions and projects with restrict budget on kinesiology, physiology, dentistry, facial recognition and mechanics, among others.

Post-earthquake building safety evaluation using consumer-grade surveillance cameras

This paper demonstrates the possibility of evaluating the safety of a building right after an earthquake using consumergrade surveillance cameras installed in the building. Two cameras are used in each story to extract the time history of interstory drift during the earthquake based on camera calibration, stereo triangulation, and image template matching techniques. The interstory drift of several markers on the rigid floor are used to estimate the motion of the geometric center using the least square approach, then the horizontal interstory drift of any location on the floor can be estimated. A shaking table collapse test of a steel building was conducted to verify the proposed approach. The results indicate that the accuracy of the interstory drift measured by the cameras is high enough to estimate the damage state of the building based on the fragility curve of the interstory drift ratio. On the other hand, the interstory drift measured by an accelerometer tends to underestimate the damage state when residual interstory drift occurs because the low frequency content of the displacement signal is eliminated when high-pass filtering is employed for baseline correction.

Versatile image-based measurements of granular flows and water wave propagation in experiments of tsunamis generated by landslides

Landslides falling into water bodies can generate destructive waves, which can be classified as tsunamis. An experimental facility to study this phenomenon has been set up. It consists of a landslide generator releasing gravel at high-speed into a wave basin. A non-intrusive system has been designed ad-hoc to be able to measure the high velocity and the geometry of the landslide as well as the generated waves characteristics. The measurement system employs the treatment of images captured by a high-speed camera which records the launched granular material illuminated by a laser sheet. A grid of laser sheets marks the basin water surface. The water has been filled by a small amount of kaolin to properly reflect the laser light at water surface. Thus, by filming with high definition cameras the perturbed water surface and successively processing the resulting images, it has been possible to measure the generated waves. The measurement framework employs a versatile camera calibration technique which allows accurate measurements in presence of: (1) high lens distortions; (2) pronounced non-parallelism condition between camera sensor and plane of measurement coincident with the laser sheet. The maximum resolution of the measurement tool is 0.01 mm, while the maximum uncertainty due to systematic error has been estimated to be 15% for the worst-case scenario. This work improves the suitability of image-based measuring systems in granular flows and free surface hydraulics experiments.Graphic

Application of 3D-PTV to mass exchange in an open-channel flow past a lateral embayment

This work presents the design and application of a Lagrangian measurement and analysis methodology, which is employed to study the flow and passive tracer exchange between a main channel and a lateral cavity in a laboratory experiment. For this purpose, a 3D-PTV system is implemented for which a static and dynamic experimental validation show that the use of a multiplane camera calibration technique and a recent trajectory linking strategy allow one to track the neutrally buoyant particles accurately in time. The resulting 3D particle trajectories are then used to quantify the 3D flow field and study the entrainment mechanisms between the main flow and the cavity using an Eulerian and a Lagrangian methodology, respectively. Analysis of the flow velocities at the geometrical interface between the main flow and the cavity indicates that the inflow is mainly concentrated at the downstream end of the cavity opening, closer to the bottom, while the particles tend to leave the cavity in the upstream part of the interface more uniform over the water depth. A vertical profile of the mass exchange coefficient is quantified based on the transverse velocity components at the interface, which confirms that the exchange intensity varies significantly with depth. More importantly, however, a novel Lagrangian trajectory classification strategy is proposed to study the transport of particles more in detail and overcome problems related to the time-dependent and 3D shape of the hydrodynamic boundary between the main channel and the cavity. Compared to the common Eulerian approach, this enables to refine the definition of mass exchange and exclude those particles that do not add to the net exchange. Subsequently a Lagrangian definition of the (depth-averaged) mass exchange coefficient is proposed, for which the current (preliminary) results indicate its potential to reliably quantify mass exchange.

Accurate and Practical Calibration of Multiple Pan-Tilt-Zoom Cameras for Live Broadcasts

We aim to realize novel services for live broadcasts, including visualizing the 3D position of a subject, using multiple pan-tilt-zoom (PTZ) cameras. For this purpose, a camera calibration technique with high accuracy over a wide pan-tilt-zoom range and high operability is required. Herein, we propose an accurate and practical method to calibrate multiple PTZ cameras for live broadcasts. This method models the PTZ camera system and accurately pre-estimates unknown parameters in the model. After the pre-estimation, accurate camera parameters can be calculated in real time using the pan-tilt angles and zoom values acquired from the sensors in the camera system. The proposed method accurately estimates the unknown parameters for a high-magnification zoom lens by making repeated initial guesses and nonlinear optimizations for high accuracy, and consists of a two-stage camera calibration for operability. We demonstrated the effectiveness of the proposed method by conducting experiments on computer graphics data and real data. We also applied it to a live sports graphics system.

Off-Line Camera-Based Calibration for Optical See-Through Head-Mounted Displays

In recent years, the entry into the market of self contained optical see-through headsets with integrated multi-sensor capabilities has led the way to innovative and technology driven augmented reality applications and has encouraged the adoption of these devices also across highly challenging medical and industrial settings. Despite this, the display calibration process of consumer level systems is still sub-optimal, particularly for those applications that require high accuracy in the spatial alignment between computer generated elements and a real-world scene. State-of-the-art manual and automated calibration procedures designed to estimate all the projection parameters are too complex for real application cases outside laboratory environments. This paper describes an off-line fast calibration procedure that only requires a camera to observe a planar pattern displayed on the see-through display. The camera that replaces the user’s eye must be placed within the eye-motion-box of the see-through display. The method exploits standard camera calibration and computer vision techniques to estimate the projection parameters of the display model for a generic position of the camera. At execution time, the projection parameters can then be refined through a planar homography that encapsulates the shift and scaling effect associated with the estimated relative translation from the old camera position to the current user’s eye position. Compared to classical SPAAM techniques that still rely on the human element and to other camera based calibration procedures, the proposed technique is flexible and easy to replicate in both laboratory environments and real-world settings.