Uncalibrated Monocular Camera Research Articles

Visual teach and generalise (VTAG)—Exploiting perceptual aliasing for scalable autonomous robotic navigation in horticultural environments

Nowadays, most agricultural robots rely on precise and expensive localisation, typically based on global navigation satellite systems (GNSS) and real-time kinematic (RTK) receivers. Unfortunately, the precision of GNSS localisation significantly decreases in environments where the signal paths between the receiver and the satellites are obstructed. This precision hampers deployments of these robots in, e.g., polytunnels or forests. An attractive alternative to GNSS is vision-based localisation and navigation. However, perceptual aliasing and landmark deficiency, typical for agricultural environments, cause traditional image processing techniques, such as feature matching, to fail. We propose an approach for an affordable pure vision-based navigation system which is not only robust to perceptual aliasing, but it actually exploits the repetitiveness of agricultural environments. Our system extends the classic concept of visual teach and repeat to visual teach and generalise (VTAG). Our teach and generalise method uses a deep learning-based image registration pipeline to register similar images through meaningful generalised representations obtained from different but similar areas. The proposed system uses only a low-cost uncalibrated monocular camera and the robot’s wheel odometry to produce heading corrections to traverse crop rows in polytunnels safely. We evaluate this method at our test farm and at a commercial farm on three different robotic platforms where an operator teaches only a single crop row. With all platforms, the method successfully navigates the majority of rows with most interventions required at the end of the rows, where the camera no longer has a view of any repeating landmarks such as poles, crop row tables or rows which have visually different features to that of the taught row. For one robot which was taught one row 25 m long our approach autonomously navigated the robot a total distance of over 3.5 km, reaching a teach-generalisation gain of 140.

A small family service robot system with uncalibrated monocular camera for visual servoing tracking fast moving family targets in short range

Small family service robots are essential applications in service robots, and their accurate tracking of moving targets is the primary condition for robot-human interaction. Compared with long-range target tracking, the limited family space leads to the short distance between the robot and the target, which often leads to the problem of large circumferential relative angular velocity between the robot and the moving target. It dramatically reduces the ability of the robot to track the fast target motion at close range and at different scales. In this paper, a novel family service robot system based on monocular RGB and robot terminal calculation is proposed to track fast-moving targets in close range. Based on the construction of the robot body with two rotation directions, the neural network model is used for visual detection and recognition of the target object. Combined with the historical information of the target, the new relative azimuth information of the target object is calculated to control the changes in the pitch angle and yaw angle of the robot body and realize the servo tracking of the robot vision on the moving target. Experimental results confirm the visual servoing tracking ability of the tiny family service robot system for indoor moving objects, which will positively promote the application of small intelligent service robots.

Visual Servoing of a Cable-Driven Soft Robot Manipulator With Shape Feature

Soft continuum robot is of superiority in the applications in unstructured environments due to its high flexibility and safe interaction ability. Accurate shape control is regarded as one of the prerequisites to improve its practicability. Considering the situation where the 3D position signals are not available, this letter investigates the vision-based shape control scheme of a soft robot manipulator. The curvature features used to depict the reference image shape of the robot backbone curve are solved through nonlinear optimization with given constraints of two endpoints in pixel coordinates. Thereafter, an adaptive image-based visual servoing controller is designed to track the reference image shape features with an uncalibrated monocular camera. The proposed algorithm was tested on an eight-cable driving soft robot prototype and proved its validity in convergence to the desired image shape.

Unified Visual Servoing Tracking and Regulation of Wheeled Mobile Robots With an Uncalibrated Camera

In this paper, a vision-based strategy is proposed for the unified tracking and regulation control of a wheeled mobile robot (WMR) equipped with an uncalibrated monocular camera. Specifically, an online implementable method based on the projection homography technique is first proposed to estimate the partial camera intrinsic parameters. By exploiting the obtained camera intrinsic parameters, the orientation and scaled position information of the WMR is extracted from the current, the reference, and the desired images to construct the error system. Then, an adaptive continuous controller is designed to address both the trajectory tracking and regulation problems with input saturation for the WMR. The system uncertainties regarding the distance information and the camera intrinsic parameters are fully taken into consideration as well as the nonholonomic constraint. Moreover, the Lyapunov-based stability analysis is utilized to prove that the proposed controller can achieve asymptotic tracking and regulation in the presence of system uncertainties. Simulation and experimental results validate the effectiveness of the proposed strategy.

An adaptive observer framework for accurate feature depth estimation using an uncalibrated monocular camera

This paper presents a novel solution to the problem of depth estimation using a monocular camera undergoing known motion. Such problems arise in machine vision where the position of an object moving in three-dimensional space has to be identified by tracking motion of its projected feature on the two-dimensional image plane. The camera is assumed to be uncalibrated, and an adaptive observer yielding asymptotic estimates of focal length and feature depth is developed that precludes prior knowledge of scene geometry and is simpler than alternative designs. Experimental results using real camera imagery are obtained with the current scheme as well as the extended Kalman filter, and performance of the proposed observer is shown to be better than the extended Kalman filter-based framework.

Estimating Dynamics On-the-Fly Using Monocular Video For Vision-Based Robotics

Estimating the physical parameters of articulated multibody systems (AMBSs) using an uncalibrated monocular camera poses significant challenges for vision-based robotics. Articulated multibody models, especially ones including dynamics, have shown good performance for pose tracking, but require good estimates of system parameters. In this paper, we first propose a technique for estimating parameters of a dynamically equivalent model (kinematic/geometric lengths as well as mass, inertia, damping coefficients) given only the underlying articulated model topology. The estimated dynamically equivalent model is then employed to help predict/filter/gap-fill the raw pose estimates, using an unscented Kalman filter. The framework is tested initially on videos of a relatively simple AMBS (double pendulum in a structured laboratory environment). The double pendulum not only served as a surrogate model for the human lower limb in flight phase, but also helped evaluate the role of model fidelity. The treatment is then extended to realize physically plausible pose-estimates of human lower-limb motions, in more-complex uncalibrated monocular videos (from the publicly available DARPA Mind's Eye Year 1 corpus). Beyond the immediate problem-at-hand, the presented work has applications in creation of low-order surrogate computational dynamics models for analysis, control, and tracking of many other articulated multibody robotic systems (e.g., manipulators, humanoids) using vision.