Monocular Vision System Research Articles

Development of a monocular vision system for robotic drilling

Robotic drilling for aerospace structures demands a high positioning accuracy of the robot, which is usually achieved through error measurement and compensation. In this paper, we report the development of a practical monocular vision system for measurement of the relative error between the drill tool center point (TCP) and the reference hole. First, the principle of relative error measurement with the vision system is explained, followed by a detailed discussion on the hardware components, software components, and system integration. The elliptical contour extraction algorithm is presented for accurate and robust reference hole detection. System calibration is of key importance to the measurement accuracy of a vision system. A new method is proposed for the simultaneous calibration of camera internal parameters and hand-eye relationship with a dedicated calibration board. Extensive measurement experiments have been performed on a robotic drilling system. Experimental results show that the measurement accuracy of the developed vision system is higher than 0.15 mm, which meets the requirement of robotic drilling for aircraft structures.

An Acceleration Command Approach to Robotic Stereo Image-based Visual Servoing

This paper presents a new stereo image-based visual servoing (IBVS) controller for a six degrees of freedom (DOF) robot manipulator based on acceleration command. An eye-in-hand stereo vision system is utilized to capture the images of the object and to estimate the depth of the object. A proportional derivative (PD) controller is developed considering the acceleration command. The augmented image-based visual servoing (AIBVS) controller helps the system achieve smoother and more linear image feature trajectories and decrease the risk that the features leave the field of view (FOV). The developed control method also enhances the stability characteristics of the system and the robot's end-effector 3D behaviour. Both simulation and experimental results on a 6 DOF DENSO 6242G robot validate the effectiveness of the proposed stereo AIBVS controller. The comparison with IBVS with monocular vision system demonstrates the improved performance of the proposed system.

Study on Stereo Vision for 3D Reconstruction of Welding Seam

The Monocular stereo vision sensor based on structured light is designed and invented for the detection of the weld seam. By incorporating the structured light, the corresponding points matching problem in the stereo vision has been solved effectively. Firstly, the built monocular stereo vision system is used to obtain the image information of seam surface. Then the image coordinate of the seam feature points in the two-dimensional plane can be identified and extracted, through the responding image processing and feature analysis of the seam image collected. After that, the three-dimensional coordinates are calculated by using the theory of projective geometry according to the positional relation between the CCD camera and structured light projected in the seam surface. By this way, the seam feature information such as the three-dimensional geometry sizes, the groove type, the angle, and the interval can be obtained. Finally, the algorithm performance is tested by the experiment of obtaining the three-dimensional coordinate for seams center points, and the results have proved the detection accuracy and reliability of the visual collection system.

Visual Measurement of the Racket Trajectory in Spinning Ball Striking for Table Tennis Player

A high-speed monocular vision system is established to track the racket trajectory for estimating the spinning ball initiated by a human. A new fast and robust method is proposed to extract the feature sequence of a racket's pose during the process of striking. Considering the complexity of the working environment and fast movement of the racket, a novel corner extraction algorithm with good robustness and high efficiency is presented, which mainly consists of two parts, i.e., target segmentation and line detection. Then PnP-based algorithm is adopted to get the rough pose of the racket, which is then optimized by orthogonal iteration algorithm to guarantee the orthogonality of the racket's orientation matrix. In order to classify the spinning mode of the ball with a higher accuracy, the racket's movement is represented by the sequence of normal vector of the racket's surface. Experiments are thoroughly conducted to verify the effectiveness of the proposed method.

3D monocular robotic ball catching

A new method to catch a thrown ball with a robot endowed with an eye-in-hand monocular visual system integrated into a gripper is proposed. As soon as the thrown ball is recognized by the visual system, the camera carried by the robot end-effector is forced to follow a baseline in the space so as to acquire an initial dataset of visual measurements from several points of view, providing a first estimate of the catching point through a linear estimation algorithm. Hereafter, additional measurements are acquired to constantly refine the previous estimate by exploiting a nonlinear estimation algorithm. During the robot trajectory, the translational components of the camera are controlled in such a way as to follow the planned path to intercept the ball, while the rotational components are forced to keep the ball into the field of view. Experimental results performed on a common industrial robotic system prove the effectiveness of the presented solution.

Low-Cost, High-Performance Monocular Vision System for Air Bearing Table Attitude Determination

Attitude determination is one of the most important prerequisites for the implementation of an air bearing table. In this context, computer vision is shown to be an enabling technology; a cost-effective monocular camera vision system, including hardware setup and navigation software, has been developed to determine the attitude of the air bearing table. Both infrared light-emitting diodes and filter are selected to simplify image processing, thereby maximizing the attitude update rate. A fiducial marker system uses five infrared light-emitting diodes with four in the same plane and the fifth outside the plane. This noncoplanar design not only improves the attitude determination accuracy, but also provides an element of robustness via a fiducial marker fault diagnosis and process method in case one of the light-emitting diodes is obscured by the top balance mass or is faulty. The static experimental results show that the system can provide attitude accuracy of 0.06 deg and angular velocity accuracy of with an attitude update rate of at least 10 Hz. Finally, a yaw-axis maneuver is performed to demonstrate the system performance under dynamic conditions. The developed system could be reused to support testing for other scientific space missions, which represents a major added value of this work.

Obstacle detection of indoor robots based on monocular vision

In this paper,a new monocular vision system was proposed to improve obstacle detection capability of indoor mobile robot.In this system,firstly,the Hue,Saturation,Intensity(HSI) color space conversion of images was performed.Secondly,a small target threshold selection method was proposed to segment the images,which enhanced the precision of the image segmentation.Thirdly,the target scene matching method and target projection matching method were used to calculate the change of the target pixel and projection so as to judge whether the target is obstacles or ground graphs.The experimental results show that the monocular vision system is effective and feasible,and this system can be applied to the navigation for small indoor mobile robots.

Study on Measurement with Monocular Vision for Posture of Moving Objects Suspended

An independent hardware system based on monocular vision has been built to measure the posture of moving objects suspended. The infrared light source and infrared filter were used to reduce image noise and interference. The six feature points distributing on the different surfaces of the object suspended were determined by PnP problem theory. According to the perspective projection constraints of the 3D coordinates and image coordinates, three posture angles were calculated as the object suspended moved. A verification test has been carried out at the three-axis-turntable to verify the static and dynamic characteristics of the measure system. The results show that the monocular vision system constructed has a good stability of higher measurement accuracy and certain dynamic instantaneity, and may be applied to measure the dynamic posture of the objects suspended.

Precise self‐localization of a walking robot on rough terrain using parallel tracking and mapping

Purpose – The purpose of this paper is to describe a novel application of the recently introduced concept from computer vision to self‐localization of a walking robot in unstructured environments. The technique described in this paper enables a walking robot with a monocular vision system (single camera) to obtain precise estimates of its pose with regard to the six degrees of freedom. This capability is essential in search and rescue missions in collapsed buildings, polluted industrial plants, etc.

Based on the"Huang Di nei jing"'s Qingzhi Theory to Adjust Chinese Enterprise Gerent��s Rage Mood

Enterprise gerent in China are suffering from depression disease due to excessive psychological pressure, that leads to they became psychosomatic in an unhealthy state or major illness. Analysis through the business life cycle stage, as well as the use of mathematical language to describe the relationship between the health benefits and seriously ill loss, pointed out that the operators must master precautions for sudden major diseases. Now, many gan diseases are due to long-term anger, therefore the prevention method is to control their emotions and let them not angry. According to the five elements theory of traditional Chinese medicine Huang Di nei jing, we put forward taking the theory of emotions as an important emotion regulation guide through Qingzhi regulation, freshing sweet taste, listenning to music and taking Chinese herbal medicine, the four kinds of treatment programs including Qingzhi regulation、eating delicious food and so on to restrain the angry mood. Also a general risk decision model is established in view of mathematical decision theory for operators or doctor to select the appropriate program to restrain the angry mood, thus opreators can be prevented from psychosomatic major diseases through the view of psychology. Enterprise gerent is a kind of special and scarce human resources, growing in the three-dimensional space in the natural environment including the society's incentive mechanism, constrainting mechanism and market mechanism of [1] . Though they are leading leaders pushing forward enterprise promoting and development, their psychology bears comprehensive pressure from all sides,not only having to manage the normal operation of the enterprise, but also taking care of the elderly and children,2011 Chinese entrepreneurs survey report shows that the number of enterprise operators who feels stressful rises year after year, especially job satisfaction and depression showed very significant negative correlation, due to excessive mental stress they become high-risk population of various diseases [2] , Now even a great many billionaires have choosen suicide to end their life, such as Jiu-Zhi-Tang group ’s 41 years old director Wei Dong，Wan Chang Technology’s 68-year-old director Gao Qingchang and so on have died in succession by jumping off a building. At present it is already verified before suicide that they were suffering from different degree of depression disease. From clinical perspective, the generation of entrepreneurs’ depression is often because of Worried and indignant , caused by many years' excess psychological pressure and a bad emotion expression for people individual getting angry. Angry mood is a intense sentiment with hostile accumulation due to countering things going against people's own wishes and lasting frustration,which blocks blood flow inside body,shows changes in expression, behavior and physiology and leads to various diseases.They usually reduce work pressure by relaxation training, biofeedback, rehabilitation system of human tracking study and holiday way to regulate the poor psychological [3] . However, relaxation training requires the presence of the expert guidance, while the majority Chinese entrepreneurs affected by cultural. The way of releasing poor emotional method select enduring alone and generally do not want to communicate with others. Biofeedback measurement and treatment require specialized equipment, such as the use of psychological measurement equipment, floating therapy equipment, ZCEA smart EAplus meridian oxygen therapy instrument, multi-sensory music therapy instrument and fully automatic brain function diagnosis and treatment instrument, such as those belonging to the sub-health rehabilitation equipment in China nor popular [4] . Human tracking rehabilitation research system research is how to use a monocular vision system and use the Mean-Shift algorithm to track improved real-time human body, nor the increasing use of these methods in China

Extended Kalman Filter-Based Methods for Pose Estimation Using Visual, Inertial and Magnetic Sensors: Comparative Analysis and Performance Evaluation

In this paper measurements from a monocular vision system are fused with inertial/magnetic measurements from an Inertial Measurement Unit (IMU) rigidly connected to the camera. Two Extended Kalman filters (EKFs) were developed to estimate the pose of the IMU/camera sensor moving relative to a rigid scene (ego-motion), based on a set of fiducials. The two filters were identical as for the state equation and the measurement equations of the inertial/magnetic sensors. The DLT-based EKF exploited visual estimates of the ego-motion using a variant of the Direct Linear Transformation (DLT) method; the error-driven EKF exploited pseudo-measurements based on the projection errors from measured two-dimensional point features to the corresponding three-dimensional fiducials. The two filters were off-line analyzed in different experimental conditions and compared to a purely IMU-based EKF used for estimating the orientation of the IMU/camera sensor. The DLT-based EKF was more accurate than the error-driven EKF, less robust against loss of visual features, and equivalent in terms of computational complexity. Orientation root mean square errors (RMSEs) of 1° (1.5°), and position RMSEs of 3.5 mm (10 mm) were achieved in our experiments by the DLT-based EKF (error-driven EKF); by contrast, orientation RMSEs of 1.6° were achieved by the purely IMU-based EKF.

Real-Time Navigation for a Humanoid Robot Using Particle Filter

This paper presents a practical real-time visual navigation system, including a vision system, a particle filter (PF) based localization system, and a path planning system, for humanoid robots in an indoor environment. A neural network (NN) converter system is used to solve the image distortion problem. The monocular vision system detects objects of interest in the scene, calculating their position in the image, and converting the position in the image to real world coordinates. The PF localization system estimates the current position by the robot’s motion model and corrects the estimated position by using feedback from the data gathered by the vision system. The path planning system determines the next motion based on the result of the localization system. This paper uses a tree-like path planning method which not only guides the robot to the destination but also avoids obstacles at the same time. The navigation method allows a user to assign several different target destinations to the robot simultaneously. The proposed method is implemented on a humanoid robot “ROBOTIS DARwIn-OP”, an open platform humanoid robot. The effectiveness of the system is demonstrated in an empirical evaluation.

光电成像探测中目标方位测量方法

In order to obtain the image and the position information of the target, an azimuth measuring method for moving targets based on monocular vision system is investigated. According to the principle of projection imaging, the mapping relationship between the 3-D position and the 2-D image of the target is derived, and the target azimuth measuring mathematical model is established based on monocular vision system. A detection combination algorithm is proposed using image subtraction and KLT method for moving targets under still background and changing background, and the sub-pixel orientation method is applied to the positioning of target feature points. The experimental and simulation results show that the method can extract the target feature points effectively, achieving a relatively accurate target positioning, and the target azimuth information is measured through monocular vision method, with the error within 8%.

CCD非可视区域下空间方位角检测方法

The traditional monocular vision system has certain limitations when space azimuth is measured. It is not feasible for the condition under which the object is without the scope of the CCD field. Therefore, a kind of rotation matrix method to measure the objects space azimuth under CCD non-vision area is proposed. According to the principle of Zhang Zhengyou calibration method, the rotation matrix method, through processing with feature points on the cooperative target, works out rotation axiss space azimuth of the cooperative target. Then the objects space azimuth under CCD non-visual area is indirectly obtained. Experiments indicate that the rotation matrix method has certain feasibility and stability, and the root mean square error of the objects space azimuth calculated by this method is lower than 0.06.

Intelligent Vision System for Door Sensing Mobile Robot

Wheeled Mobile Robots find numerous applications in the Indoor man made structured environments. In order to operate effectively, the robots must be capable of sensing its surroundings. Computer Vision is one of the prime research areas directed towards achieving these sensing capabilities. In this paper, we present a Door Sensing Mobile Robot capable of navigating in the indoor environment. A robust and inexpensive approach for recognition and classification of the door, based on monocular vision system helps the mobile robot in decision making. To prove the efficacy of the algorithm we have designed and developed a ‘Differentially’ Driven Mobile Robot. A wall following behavior using Ultra Sonic range sensors is employed by the mobile robot for navigation in the corridors. Field Programmable Gate Arrays (FPGA) have been used for the implementation of PD Controller for wall following and PID Controller to control the speed of the Geared DC Motor. DOI: http://dx.doi.org/10.11591/ijra.v1i4.1262 Full Text: PDF

A Monocular Vision System Based on the Equidistance Pseudo Binocular Feature Recognition

Monocular vision system for medical robot must have high accuracy. In traditional monocular vision system, localization algorithm is realized by image space transformation, which leads to a trivial calculation process and a complex camera calibration method, therefore, this paper presents a new target characteristics identification method. Through the establishment of an equidistance pseudo binocular image acquisition way, searching the image and space coordinate transformation relation, avoid the complicated space matrix conversion and improve the position precision. Using Pointer method for image processing and customized backtracking boundary conditions for image searching, this paper realizes the curve fitting of target features boundary, and pattern recognition. Finally, an experiment is completed to guide the pharmacy movement of the Staubli TX40 arm, whose software runs within 200 ms and location error is less in 0.2 mm, verifies the validity of the algorithm.

An Onboard Monocular Vision System for Autonomous Takeoff, Hovering and Landing of a Micro Aerial Vehicle

In this paper, we present an onboard monocular vision system for autonomous takeoff, hovering and landing of a Micro Aerial Vehicle (MAV). Since pose information with metric scale is critical for autonomous flight of a MAV, we present a novel solution to six degrees of freedom (DOF) pose estimation. It is based on a single image of a typical landing pad which consists of the letter H surrounded by a circle. A vision algorithm for robust and real-time landing pad recognition is implemented. Then the 5 DOF pose is estimated from the elliptic projection of the circle by using projective geometry. The remaining geometric ambiguity is resolved by incorporating the gravity vector estimated by the inertial measurement unit (IMU). The last degree of freedom pose, yaw angle of the MAV, is estimated from the ellipse fitted from the letter H. The efficiency of the presented vision system is demonstrated comprehensively by comparing it to ground truth data provided by a tracking system and by using its pose estimates as control inputs to autonomous flights of a quadrotor.

Vision-based Terrain Referenced Navigation for Unmanned Aerial Vehicles using Homography Relationship

A vision-based terrain referenced navigation (TRN) system is addressed for autonomous navigation of unmanned aerial vehicles (UAVs). A typical TRN algorithm blends inertial navigation data with measured terrain information to estimate vehicle's position. In this paper, however, we replace the low-cost inertial navigation system (INS) with a monocular vision system. The homography decomposition algorithm is utilized to estimates the relative translational motion using features on the ground with simple assumptions. A numerical integration point-mass filter based on Bayesian estimation is employed to combine the translation information obtained from the vision system with the measured terrain height. Numerical simulations are constructed to evaluate the performance of the proposed method. The results show that the precise autonomous navigation of unmanned aircrafts is achieved by the vision-based TRN algorithm.

On-Board Lane Detection System for Intelligent Vehicle Based on Monocular Vision

Abstract The objective of this research is to develop a monocular vision system that can locate the positions of the road lane in real time. First, Canny approach is used to obtain edge map from the road image acquired from monocular camera mount on vehicle; Second, a matching process is conducted to normalize the candidates of road line; Third, a searching method is used for reinforce potential road lines while degraded those impossible ones; Forth, a linking condition is used to further enhance the confidence of the potential lane lines, and a K-means cluster algorithm is employed to localize the lane lines; Finally, a on board system is designed for experiment. The proposed system is shown to work well under various conditions on the roadway. Besides, the computation cost is inexpensive and the system’s response is almost real time.

Optimized EIF-SLAM algorithm for precision agriculture mapping based on stems detection

Precision agricultural maps are required for agricultural machinery navigation, path planning and plantation supervision. In this work we present a Simultaneous Localization and Mapping (SLAM) algorithm solved by an Extended Information Filter (EIF) for agricultural environments (olive groves). The SLAM algorithm is implemented on an unmanned non-holonomic car-like mobile robot. The map of the environment is based on the detection of olive stems from the plantation. The olive stems are acquired by means of both: a range sensor laser and a monocular vision system. A support vector machine (SVM) is implemented on the vision system to detect olive stems on the images acquired from the environment. Also, the SLAM algorithm has an optimization criterion associated with it. This optimization criterion is based on the correction of the SLAM system state vector using only the most meaningful stems – from an estimation convergence perspective – extracted from the environment information without compromising the estimation consistency. The optimization criterion, its demonstration and experimental results within real agricultural environments showing the performance of our proposal are also included in this work.