Camera Coordinate Research Articles

Genetic algorithm-based optimal computed torque control of a vision-based tracker robot: Simulation and experiment

In this paper, an optimal control scheme based on the application of genetic algorithm (GA) is applied to a tracker robot with machine vision capability. First, dynamic equations governing the robot system are extracted and then, kinematic relationships are acquired according to camera specifications. Then, the obtained open-loop model of the robot is simulated and its validity is evaluated through an experimental work. The computed torque control algorithm with conventional tuning technique is initially used to control the robotic system. However, it is found that tuning the mentioned robot controller via the conventional technique is not so effective due to the unseen hard non-linearity presented in the robot controller such as saturation of the actuators. Consequently, a more advanced GA-based optimal control scheme is proposed in this investigation. Thus, GA is used to obtain the optimum values of control constants based on a suitable cost function. Accordingly, a simulation study is conducted to highlight the superiority of the GA-based controller compared to the former algorithm. Besides, the linking approach of the vision subsystem to the robot controller is further described from both mathematical and experimental standpoints. Finally, the proposed visual tracker robot is developed and its performance to track a moving object on a circular as well as a butterfly-shape trajectory in the camera coordinates is investigated. The experimental results clearly reveal the effectiveness of the visual robot incorporating the optimal controller to follow the target trajectories and confirm the stability of the control system.

The influence of software algorithms on photogrammetric micro-feature measurement’s uncertainty

In the 3D reconstruction field, photogrammetry is a powerful and robust measurement technology for lots of dimensional ranges, from macro and aerial photogrammetry to close-range and micro-photogrammetry, emerging recently with good results also in the survey of small object with sub-millimeter features. A fundamental issue is still the calibration, for the estimation of both internal and external parameters, mostly in the case of very small dimensional ranges. In previous works, some aspects of the photogrammetric process have been taken into account and analyzed by 3D comparisons with a reference model, while the present work is firstly focused on the outputs of the camera’s alignment phase, in which the computation of internal and external parameters is carried out. By repetition of the process in the same conditions, it has been noticed that alignment results, in terms of camera coordinates and internal calibration parameters, are affected by a sort of variability. Thus, the aim of this work is to estimate the uncertainty component over the 3D reconstruction accuracy attributable to the repeatability of the photogrammetric software. For this purpose, several factors, linked to the shooting geometry and to the alignment process, have been considered.

An Adaptive Camera and Lidar Joint Calibration Algorithm

In large mechanism intelligent control system, cameras and lidars are usually mounted to obtain environmental information. In order to fuse the information of camera and lidar, joint calibration is essential to place their data under the same coordinate system. To solve the problem of camera and 3D lidar joint calibration, an adaptive joint calibration method is proposed. Only one planar calibration board is needed for the calibration. In the nonlinear optimization procedure, the camera extrinsic parameters and optimized together with the rotation matrix and translation vector from lidar to camera coordinate. Compared with the method of fixed camera parameters, average calibration error decreases by 22.50%. By using weighed lidar data, calibration error further decreases by 8.80% in average.

A computationally efficient method for hand\u2013eye calibration

PurposeSurgical robots with cooperative control and semiautonomous features have shown increasing clinical potential, particularly for repetitive tasks under imaging and vision guidance. Effective performance of an autonomous task requires accurate hand–eye calibration so that the transformation between the robot coordinate frame and the camera coordinates is well defined. In practice, due to changes in surgical instruments, online hand–eye calibration must be performed regularly. In order to ensure seamless execution of the surgical procedure without affecting the normal surgical workflow, it is important to derive fast and efficient hand–eye calibration methods.MethodsWe present a computationally efficient iterative method for hand–eye calibration. In this method, dual quaternion is introduced to represent the rigid transformation, and a two-step iterative method is proposed to recover the real and dual parts of the dual quaternion simultaneously, and thus the estimation of rotation and translation of the transformation.ResultsThe proposed method was applied to determine the rigid transformation between the stereo laparoscope and the robot manipulator. Promising experimental and simulation results have shown significant convergence speed improvement to 3 iterations from larger than 30 with regard to standard optimization method, which illustrates the effectiveness and efficiency of the proposed method.

Modeling and evaluation of hand-eye coordination of surgical robotic system on task performance.

Robotic-assisted minimally invasive surgery changes the direct hand and eye coordination in traditional surgery to indirect instrument and camera coordination, which affects the ergonomics, operation performance, and safety. A camera, two instruments, and a target, as the descriptors, are used to construct the workspace correspondence and geometrical relationships in a surgical operation. A parametric model with a set of parameters is proposed to describe the hand-eye coordination of the surgical robot. From the results, optimal values and acceptable ranges of these parameters are identified from two tasks. A 90° viewing angle had the longest completion time; 60° instrument elevation angle and 0° deflection angle had better performance; there is no significant difference among manipulation angles and observing distances on task performance. This hand-eye coordination model provides evidence for robotic design, surgeon training, and robotic initialization to achieve dexterous and safe manipulation in surgery.

Estimation of Antenna Pose in the Earth Frame Using Camera and IMU Data from Mobile Phones.

The poses of base station antennas play an important role in cellular network optimization. Existing methods of pose estimation are based on physical measurements performed either by tower climbers or using additional sensors attached to antennas. In this paper, we present a novel non-contact method of antenna pose measurement based on multi-view images of the antenna and inertial measurement unit (IMU) data captured by a mobile phone. Given a known 3D model of the antenna, we first estimate the antenna pose relative to the phone camera from the multi-view images and then employ the corresponding IMU data to transform the pose from the camera coordinate frame into the Earth coordinate frame. To enhance the resulting accuracy, we improve existing camera-IMU calibration models by introducing additional degrees of freedom between the IMU sensors and defining a new error metric based on both the downtilt and azimuth angles, instead of a unified rotational error metric, to refine the calibration. In comparison with existing camera-IMU calibration methods, our method achieves an improvement in azimuth accuracy of approximately 1.0 degree on average while maintaining the same level of downtilt accuracy. For the pose estimation in the camera coordinate frame, we propose an automatic method of initializing the optimization solver and generating bounding constraints on the resulting pose to achieve better accuracy. With this initialization, state-of-the-art visual pose estimation methods yield satisfactory results in more than 75% of cases when plugged into our pipeline, and our solution, which takes advantage of the constraints, achieves even lower estimation errors on the downtilt and azimuth angles, both on average (0.13 and 0.3 degrees lower, respectively) and in the worst case (0.15 and 7.3 degrees lower, respectively), according to an evaluation conducted on a dataset consisting of 65 groups of data. We show that both of our enhancements contribute to the performance improvement offered by the proposed estimation pipeline, which achieves downtilt and azimuth accuracies of respectively 0.47 and 5.6 degrees on average and 1.38 and 12.0 degrees in the worst case, thereby satisfying the accuracy requirements for network optimization in the telecommunication industry.

High-precision method of binocular camera calibration with a distortion model.

A high-precision camera calibration method for binocular stereo vision system based on a multi-view template and alternative bundle adjustment is presented in this paper. The proposed method could be achieved by taking several photos on a specially designed calibration template that has diverse encoded points in different orientations. In this paper, the method utilized the existing algorithm used for monocular camera calibration to obtain the initialization, which involves a camera model, including radial lens distortion and tangential distortion. We created a reference coordinate system based on the left camera coordinate to optimize the intrinsic parameters of left camera through alternative bundle adjustment to obtain optimal values. Then, optimal intrinsic parameters of the right camera can be obtained through alternative bundle adjustment when we create a reference coordinate system based on the right camera coordinate. We also used all intrinsic parameters that were acquired to optimize extrinsic parameters. Thus, the optimal lens distortion parameters and intrinsic and extrinsic parameters were obtained. Synthetic and real data were used to test the method. The simulation results demonstrate that the maximum mean absolute relative calibration errors are about 3.5e-6 and 1.2e-6 for the focal length and the principal point, respectively, under zero-mean Gaussian noise with 0.05 pixels standard deviation. The real result shows that the reprojection error of our model is about 0.045 pixels with the relative standard deviation of 1.0e-6 over the intrinsic parameters. The proposed method is convenient, cost-efficient, highly precise, and simple to carry out.

Devising a method for the active coordination of video cameras in optical navigation based on the multi-agent approach

Here we present results of research into multi-agent approach to solving the problem of constructing a wireless Ad Hoc network to transmit video stream during optical navigation. When obtaining data from video cameras in real time, there is a problem of realization of effective coordination of video data. To improve the algorithms of capturing an object, it is necessary to improve the architecture of devices that utilize network metadata for the analysis of scenes from video surveillance. As a result of exploring the ways of solving the specified problems, we proposed a method for active coordination of video cameras based on the multi-agent approach (MAC-on-MAA). The application of this method increases efficiency of network cameras through their reconfiguration. We examined the problem of determining an adequate level of illuminance normalization of a video frame based on analysis of the modules throughput of multi-agent system for encoding transmitted video data. It was revealed that the time of analysis of video stream from a network decreases when ignoring the decoding of certain domains of graphic elements in the received video frames. The devised MAC-on-MAA method allows effective updating of routing tables in the reactive protocol AODV. We obtained analytic expressions for the assessment of quantitative characteristics of energy consumption by mobile agents. Protection of communication channels of video surveillance network is improved through the integration of steganocontainers in the free bits of RGB characteristics of video frames pixels.

2-D and 3-D lens model for computer vision

The lenses are modified in its design to meet the refractive index used for various applications like automated vehicle driving, micro-scopes, telescope etc. These application work well along with correct and effective modeling and implementations (Adelson and Wang in IEEE Trans Pattern Anal Mach Intell 14(2):99–106, 12; Chavan and Mishra in Int J Math Sci Eng Appl 1(1):199–218, 13). In this paper one such modeling and its simulation is presented as 2-D and 3-D model of lens which can be used in camera for computer vision system. The model gives the results considering physical parameters as constants, (camera coordinates and image coordinates), and are based on the angle of the object with axes and focal distance. This model has been implemented using ‘C’ and results are plotted.

Marker’s position estimation under uncontrolled environment for augmented reality

The optical tracking information in manufacturing can provide valuable support and time saving for autonomous operations, but ill environment conditions prevent a better performance of vision systems. In this work, a method for estimating object position under semi-controlled environment where lighting conditions change dynamically is proposed. This method incorporates regression analysis that combines light measurement and an augmented reality (AR) system. Augmented reality (AR) combines virtual objects with real environment. Furthermore, every AR application uses a video camera to capture an image including a marker in order to place a virtual object, which gives user an enriched environment. Using a tracking system to estimate the marker’s position with respect to the camera coordinate frame is needed to positioning a virtual object. Most research studies on tracking system for AR are under controlled environment. The problem is that tracking systems for markers are sensitive to variations in lighting conditions in the real environment. To solve this problem, a method is proposed to better estimate a marker position based on regression analysis, where lighting conditions are taken into account. Our approach improves the accuracy of the marker position estimation under different lighting conditions. The experimental data obtained under a laboratory context with changes on light condition are fitted with this approach with an accuracy of 99 %.

Location Tracking And Detection Of Human Using Mobile Robot And Wireless Sensor Networks

In order to avoid dangerous environmental disasters, robots are being recognized as good entrants to step in as human rescuers. Robots has been gaining interest of many researchers in rescue matters especially which are furnished with advanced sensors. In distributed wireless robot system main objective for a rescue system is to track the location of the object continuously. This paper provides a novel idea to track and locate human in disaster area using stereo vision system and ZigBee technology. This system recursively predict and updates 3D coordinates in a robot coordinate camera system of a human which makes the system cost effective. This system is comprised of ZigBee network which has many advantages such as low power consumption, self- healing low data rates and low cost.

Development of a calibrating algorithm for Delta Robot’s visual positioning based on artificial neural network

Delta robot with vision system can automatically control the end-actuator to accurately grasp moving objects on the conveyor belt. Establishment of the mapping relationship between the image feature space and the robot working space form a closed-loop chain for transformational link between the robot coordinate, camera coordinate and conveyor belt coordinate. The vision system calibration is a basic problem of robot vision research and implementation. The artificial neural networks (ANN) which has learning ability, adaptive ability and nonlinear function approximation ability can establish the nonlinear relationship between space points and pixel points to complete accurate calibration of the vision system. The convergence speed of calibration algorithm affects the real-time visual servo system. The calibration precision, generalization ability and calibration space of algorithm influence the robot grasping accuracy. Therefore, a new calibration technique for delta robot’s vision system was presented in this paper. The algorithm combines ANN with Faugeras vision system calibration technology. The setting of the initial value, network structure and the choice of the activation function is based on the model of Faugeras vision system calibration algorithm, which makes the actual output of the network closer to the target output. Experiments proved that this algorithm has higher calibration accuracy and generalization ability compared with the conventional calibration algorithm, as well as faster convergence speed compared with the conventional artificial neural network structure in the case of high calibration accuracy.

4D ANIMATION RECONSTRUCTION FROM MULTI-CAMERA COORDINATES TRANSFORMATION

Reservoir dredging issues are important to extend the life of reservoir. The most effective and cost reduction way is to construct a tunnel to desilt the bottom sediment. Conventional technique is to construct a cofferdam to separate the water, construct the intake of tunnel inside and remove the cofferdam afterwards. In Taiwan, the ZengWen reservoir dredging project will install an Elephant-trunk Steel Pipe (ETSP) in the water to connect the desilting tunnel without building the cofferdam. Since the installation is critical to the whole project, a 1:20 model was built to simulate the installation steps in a towing tank, i.e. launching, dragging, water injection, and sinking. To increase the construction safety, photogrammetry technic is adopted to record images during the simulation, compute its transformation parameters for dynamic analysis and reconstruct the 4D animations. In this study, several Australis© coded targets are fixed on the surface of ETSP for auto-recognition and measurement. The cameras orientations are computed by space resection where the 3D coordinates of coded targets are measured. Two approaches for motion parameters computation are proposed, i.e. performing 3D conformal transformation from the coordinates of cameras and relative orientation computation by the orientation of single camera. Experimental results show the 3D conformal transformation can achieve sub-mm simulation results, and relative orientation computation shows the flexibility for dynamic motion analysis which is easier and more efficiency.

4D ANIMATION RECONSTRUCTION FROM MULTI-CAMERA COORDINATES TRANSFORMATION

Abstract. Reservoir dredging issues are important to extend the life of reservoir. The most effective and cost reduction way is to construct a tunnel to desilt the bottom sediment. Conventional technique is to construct a cofferdam to separate the water, construct the intake of tunnel inside and remove the cofferdam afterwards. In Taiwan, the ZengWen reservoir dredging project will install an Elephant-trunk Steel Pipe (ETSP) in the water to connect the desilting tunnel without building the cofferdam. Since the installation is critical to the whole project, a 1:20 model was built to simulate the installation steps in a towing tank, i.e. launching, dragging, water injection, and sinking. To increase the construction safety, photogrammetry technic is adopted to record images during the simulation, compute its transformation parameters for dynamic analysis and reconstruct the 4D animations. In this study, several Australis© coded targets are fixed on the surface of ETSP for auto-recognition and measurement. The cameras orientations are computed by space resection where the 3D coordinates of coded targets are measured. Two approaches for motion parameters computation are proposed, i.e. performing 3D conformal transformation from the coordinates of cameras and relative orientation computation by the orientation of single camera. Experimental results show the 3D conformal transformation can achieve sub-mm simulation results, and relative orientation computation shows the flexibility for dynamic motion analysis which is easier and more efficiency.

TH‐EF‐BRB‐08: Robotic Motion Compensation for Radiation Therapy: A 6DOF Phantom Study

Purpose:The high accuracy of frame‐based stereotactic radiosurgery (SRS), which uses a rigid frame fixed to the patient's skull, is offset by potential drawbacks of poor patient compliance and clinical workflow restrictions. Recent research into frameless SRS has so far resulted in reduced accuracy. In this study, we investigate the use of a novel 6 degree‐of‐freedom (6DOF) robotic head motion cancellation system that continuously detects and compensates for patient head motions during a SRS delivery. This approach has the potential to reduce invasiveness while still achieving accuracies better or equal to traditional frame‐based SRS.Methods:A 6DOF parallel kinematics robotics stage was constructed, and controlled using an inverse kinematics‐based motion compensation algorithm. A 6DOF stereoscopic infrared (IR) marker tracking system was used to monitor real‐time motions at sub‐millimeter and sub‐degree levels. A novel 6DOF calibration technique was first applied to properly orient the camera coordinate frame to match that of the LINAC and robotic control frames. Simulated head motions were measured by the system, and the robotic stage responded to these 6DOF motions automatically, returning the reflective marker coordinate frame to its original position.Results:After the motions were introduced to the system in the phantom‐based study, the robotic stage automatically and rapidly returned the phantom to LINAC isocenter. When errors exceeded the compensation lower threshold of 0.25 mm or 0.25 degrees, the system registered the 6DOF error and generated a cancellation trajectory. The system responded in less than 0.5 seconds and returned all axes to less than 0.1 mm and 0.1 degree after the 6DOF compensation was performed.Conclusion:The 6DOF real‐time motion cancellation system was found to be effective at compensating for translational and rotational motions to current SRS requirements. This system can improve frameless SRS by automatically returning patients to isocenter with high 6DOF accuracy.

Golf Green Slope Estimation Using a Cross Laser Structured Light System and an Accelerometer

In this paper, we propose a method combining an accelerometer with a cross structured light system to estimate the golf green slope. The cross-line laser provides two laser planes whose functions are computed with respect to the camera coordinate frame using a least square optimization. By capturing the projections of the cross-line laser on the golf slope in a static pose using a camera, two 3D curves’ functions are approximated as high order polynomials corresponding to the camera coordinate frame. Curves’ functions are then expressed in the world coordinate frame utilizing a rotation matrix that is estimated based on the accelerometer’s output. The curves provide some important information of the green such as the height and the slope’s angle. The curves estimation accuracy is verified via some experiments which use OptiTrack camera system as a ground-truth reference.

Research on displacement measurement of disk vibration based on machine vision technique

The paper proposes a new measurement of displacement based on machine vision technique. It ingeniously makes good use of principle of camera calibration and realizes non-contact measurement of disk vibration. There are mainly two parts during the whole process: first, the intrinsic parameters of the camera used are measured by means of highly precise calibration target, and then the extrinsic parameters of disk are obtained by camera. The displacement variation of the feature points on Z axis will be calculated by extrinsic parameters and the vibration displacement can be acquired by conversion of world coordinates with camera coordinates. Compared with previous methods, it can simultaneously obtain vibration displacement of many feature points on the disk, increase the efficiency and precision of measurement with simple and economic apparatus and wide range of application, thus a convenient, feasible and non-contact displacement measurement.

Direction Estimation for Pedestrian Monitoring System in Smart Cities: An HMM Based Approach

The paper proposes a novel approach for direction estimation of a moving pedestrian as perceived in a 2-D coordinate of field camera. The proposed direction estimation method is intended for pedestrian monitoring in traffic control systems. Apart from traffic control, direction of motion estimation is also very important in accident avoidance system for smart cars, assisted living systems, in occlusion prediction for seamless tracking in visual surveillance, and so on. The proposed video-based direction estimation method exploits the notion of perspective distortion as perceived in monocular vision of 2-D camera co-ordinate. The temporal pattern of change in dimension of pedestrian in a frame sequence is unique for each direction; hence, the dimensional change-based feature is used to estimate the direction of motion; eight discrete directions of motion are considered and the hidden Markov model is used for classification. The experiments are conducted over CASIA Dataset A , CASIA Dataset B , and over a self-acquired dataset: NITR Conscious Walk Dataset . The balanced accuracy of direction estimation for these experiments yields satisfactory results with accuracy indices as 94.58%, 90.87%, and 95.83%, respectively. The experiment also justifies with suitable test conditions about the characteristic features, such as robustness toward improper segmentation, partial occlusion, and changing orientation of head or body during walk of a pedestrian. The proposed method can be used as a standalone system or can be integrated with existing frame-based direction estimation methods for implementing a pedestrian monitoring system.

Calibration of a Stereo Radiation Detection Camera Using Planar Homography

This paper proposes a calibration technique of a stereo gamma detection camera. Calibration of the internal and external parameters of a stereo vision camera is a well-known research problem in the computer vision society. However, few or no stereo calibration has been investigated in the radiation measurement research. Since no visual information can be obtained from a stereo radiation camera, it is impossible to use a general stereo calibration algorithm directly. In this paper, we develop a hybrid-type stereo system which is equipped with both radiation and vision cameras. To calibrate the stereo radiation cameras, stereo images of a calibration pattern captured from the vision cameras are transformed in the view of the radiation cameras. The homography transformation is calibrated based on the geometric relationship between visual and radiation camera coordinates. The accuracy of the stereo parameters of the radiation camera is analyzed by distance measurements to both visual light and gamma sources. The experimental results show that the measurement error is about 3%.

차선검출 기반 카메라 포즈 추정

Abstract : When a camera installed on a vehicle is used, estimation of the camera pose including tilt, roll, and pan angle with respect to the world coordinate system is important to associate camera coordinates with world coordinates. Previous approaches using huge calibration patterns have the disadvantage that the calibration patterns are costly to make and install. And, previous approaches exploiting multiple vanishing points detected in a single image are not suitable for automotive applications as a scene where multiple vanishing points can be captured by a front camera is hard to find in our daily environment. This paper proposes a camera pose estimation method. It collects multiple images of lane markings while changing the horizontal angle with respect to the markings. One vanishing point, the cross point of the left and right lane marking, is detected in each image, and vanishing line is estimated based on the detected vanishing points. Finally, camera pose is estimated from the vanishing line. The proposed method is based on the fact that planar motion does not change the vanishing line of the plane and the normal vector of the plane can be estimated by the vanishing line. Experiments with large and small tilt and roll angle show that the proposed method outputs accurate estimation results respectively. It is verified by checking the lane markings are up right in the bird’s eye view image when the pan angle is compensated.