Field Robotics Applications Research Articles

CLONeR: Camera-Lidar Fusion for Occupancy Grid-Aided Neural Representations

Recent advances in neural radiance fields (NeRFs) achieve state-of-the-art novel view synthesis and facilitate dense estimation of scene properties. However, NeRFs often fail for outdoor, unbounded scenes that are captured under very sparse views with the scene content concentrated far away from the camera, as is typical for field robotics applications. In particular, NeRF-style algorithms perform poorly: (1) when there are insufficient views with little pose diversity, (2) when scenes contain saturation and shadows, and (3) when finely sampling large unbounded scenes with fine structures becomes computationally intensive. This paper proposes <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">CLONeR</i> , which significantly improves upon NeRF by allowing it to model large unbounded outdoor driving scenes that are observed from sparse input sensor views. This is achieved by decoupling occupancy and color learning within the NeRF framework into separate Multi-Layer Perceptrons (MLPs) trained using LiDAR and camera data, respectively. In addition, this paper proposes a novel method to build differentiable 3D Occupancy Grid Maps (OGM) alongside the NeRF model, and leverage this occupancy grid for improved sampling of points along a ray for volumetric rendering in metric space. Through extensive quantitative and qualitative experiments on scenes from the KITTI dataset, this paper demonstrates that the proposed method outperforms state-of-the-art NeRF models on both novel view synthesis and dense depth prediction tasks when trained on sparse input data.

Real-Time 6-DOF Pose Estimation of Known Geometries in Point Cloud Data

The task of tracking the pose of an object with a known geometry from point cloud measurements arises in robot perception. It calls for a solution that is both accurate and robust, and can be computed at a rate that aligns with the needs of a control system that might make decisions based on it. The Iterative Closest Point (ICP) algorithm is widely used for this purpose, but it is susceptible to failure in practical scenarios. We present a robust and efficient solution for pose-from-point cloud estimation called the Pose Lookup Method (PLuM). PLuM is a probabilistic reward-based objective function that is resilient to measurement uncertainty and clutter. Efficiency is achieved through the use of lookup tables, which substitute complex geometric operations such as raycasting used in earlier solutions. Our results show millimetre accuracy and fast pose estimation in benchmark tests using triangulated geometry models, outperforming state-of-the-art ICP-based methods. These results are extended to field robotics applications, resulting in real-time haul truck pose estimation. By utilising point clouds from a LiDAR fixed to a rope shovel, the PLuM algorithm tracks a haul truck effectively throughout the excavation load cycle at a rate of 20 Hz, matching the sensor frame rate. PLuM is straightforward to implement and provides dependable and timely solutions in demanding environments.

Attention-Driven Active Sensing With Hybrid Neural Network for Environmental Field Mapping

In environmental monitoring programs, mobile robots have been widely deployed for remote sensing, with the end objective of monitoring and mapping out environmental fields. Complex characteristics and correlations in natural phenomena make it challenging to establish a reliable framework for mobile sensing and field mapping. Furthermore, constraints of onboard resources will limit the ability of mobile robots to cover a large area. This article focuses on the active sensing problem in environmental field mapping and particularly exploits the use of intrinsic interactions among multivariate spatiotemporal data. A novel deep neural network of a hybrid CNN-RNN model is employed to learn the monitored multivariate spatiotemporal field. Specifically, a set of attention mechanisms is designed and embedded in the network, which is able to adaptively capture parameterwise dependencies among the monitored heterogeneous parameters and spatial correlations in geolocations of a surveyed field. The weights of inferred attention facilitate explicit interpretation of the driving parameters and geolocations. Some subregions of interest in the surveyed field are specified by their spatial attention distribution and are actively sensed by following the proposed coverage path planner. Experiments are carried out using a real-world dataset with multisource environmental imagery from a remote sensing program. Experimental results are obtained, which demonstrate the superior mapping performance of the proposed systematical methodology compared to baseline methods. Furthermore, the proposed model is able to quantitatively reveal the driving monitored parameters and geolocations in a regression process. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —This article was motivated by the need for a practical and systematic approach for reconstruction and planning to execute robotic active sensing (AS) in environmental field mapping. Field robotic applications are not maneuverable in comparison with indoor scenarios due to severe conflict between the need for long execution endurance in the field and the very limited onboard resources. Traditional AS planners normally use statistical model-based informative metrics, which may lead to model misspecification in real-world phenomena. The developed framework in this article yields a novel attention-driven metric to guide AS and mapping. It relies on an attention-based hybrid neural network that reveals the driving variables in terms of the heterogeneities and complexities in a natural environment. The high-priority regions are maximized in a coverage path depending on the inferred spatial attention distribution while maintaining the travel cost of the sensing robots within an available energy budget. Experiments using a remote sensing dataset validate the reliable performance of the proposed framework, in environmental field mapping.

CHORD: Distributed Data-Sharing via Hybrid ROS 1 and 2 for Multi-Robot Exploration of Large-Scale Complex Environments

A well-structured and reliable communication system is key to the successful operations of multi-robot systems. In this letter, we present our design and implementation of a multi-robot communication architecture CHORD (Collaborative High-bandwidth Operations with Radio Droppables) based on two popular robotics middleware, ROS 1 and ROS 2. We discuss the benefit and best practices of combining two different frameworks that share the same spirit and show its performance from large-scale real-world experiments. The proposed system is developed as part of Team CoSTAR's effort for the DARPA Subterranean (SubT) Challenge. The system has been field-proved and demonstrated in the Urban Circuit event, where team CoSTAR won first place. To our knowledge, this work is the first real-world demonstration of a ROS 2-based multi-robot system in such large-scale extreme environments. From the significant improvement of the communication performance and the ease of transition from existing ROS 1 systems, this work encourages wider adoption of ROS 2 in field robotics applications.

Intuitive Bilateral Teleoperation of a Cable-driven Parallel Robot Controlled by a Cable-driven Parallel Robot

Intuitive remote multiple degrees of freedom (DOF) robot control has distinct advantages in field robotics applications, especially for a cable-driven parallel robot (CDPR) capable of operating in a wide workspace with a heavy payload. This study proposes a novel intuitive teleoperation method for a CDPR, in which a remote CDPR is teleoperated by a local CDPR driven by a human. The proposed system comprises a master CDPR in the operating room, a slave CDPR in the workplace, wireless communication between the master and slave CDPRs, and an admittance control to realize the haptic force to the master CDPR. The master CDPR transmits the Cartesian posture command, based on an operator???s manipulation, to the slave CDPR, and the slave CDPR???s wrench force obtained through the cable tension measurement is fed back to the master CDPR. For validation, two CDPRs in different locations (approximately 7.46 km apart) are utilized; a mini-size CDPR is used as the master device and a grand-size CDPR is used as the slave robot. The experimental results demonstrate that the operator can fully control the slave robot???s six DOF motions with spatial force feeling in hand, and satisfactory synchronized movements of the two CDPRs at a distance could be performed. Also, a preliminary experimental result of the proposed wrench feedback method is discussed. The proposed method has practical benefits in field robotics applications in unmanned and hazardous workspaces, such as nuclear power plants and space, which require long-distance remote manipulation at different locations.

Learning-Aided 3-D Occupancy Mapping With Bayesian Generalized Kernel Inference

In this paper, we consider the problem of building descriptive three-dimensional (3-D) maps from sparse and noisy range sensor data. We expand our previously proposed method leveraging Bayesian kernel inference for prediction of occupancy in locations not directly observed by a range sensor. In this paper, we show that our kernel inference approach generalizes previous “counting sensor model” approaches from discrete occupancy grids to continuous maps. Our approach enables prediction about occupancy in regions unobserved by the range sensor based on local measurements, and smoothly transitions to a prior in regions lacking sufficient data for reliable inference. Furthermore, we demonstrate quantitatively using simulated data that the mapping performance of our method can be improved by considering rays as continuous observations, rather than sampling discrete free-space point observations along rays. Though the maps produced by our method are in principle continuous, discretizing space affords us several computational advantages, including the ability to apply recursive Bayesian updates, that allow us to perform inference very efficiently, even on large datasets. To demonstrate this advantage, we present experimental results applying this method to large-scale lidar data collected with a ground robot, showing real-time performance. Other field robotics applications, including underwater 3-D mapping with sonar, are explored qualitatively.

Motion Cost Characterisation of an Omnidirectional WMR on Uneven Terrains

Abstract Electrically powered robots are seeing increased adoption for field robotics applications. These applications typically require the platforms be deployed for extended periods, or traverse large distances. Power modelling therefore becomes important, as knowing the energy costs of operation is necessary for the estimation of maximum range and operating time, and the generation of energy-efficient plans. In this paper, we present a physics-based power model for the Swagbot platform--an omnidirectional electric field robot--which can be calibrated from a combination of GPS and proprioceptive data gathered throughout the course of normal operations. This model is then experimentally verified via endurance testing performed on cattle properties featuring uneven terrain. The model was demonstrated to predict the energy consumption for a pasture survey and a weed detection task to within 5% of the measured value.

Energy Aware Mission Planning for WMRs on Uneven Terrains

Field robotics applications typically require platforms to be deployed for extended periods of time, traversing large distances over often uneven terrains. To best leverage these platforms, it is important to know the energy costs of their operation, to maximise range and operating time. This is particularly the case for electrically powered robots, which share a single power source for actuation, sensing, and computation. In this paper we build upon our prior work—derivation of a physics-based power model for an omnidirectional wheeled mobile robot— to develop an energy-aware rough terrain motion planner, which enables efficient missions to be defined and carried out whilst considering the impact of the terrain slope and slippage characteristics on the energy cost of motion. We develop an effective pipeline for the generation of such missions on-the-fly, and compare the performance of the proposed energy-aware approach against a 3D Euclidean distance metric, experimentally verifying the increased energy efficiency of the resulting plans.

Registration of three‐dimensional scanning LiDAR sensors: An evaluation of model‐based and model‐free methods

AbstractRegistration, also know as extrinsic calibration, is the process of determining the position and orientation of a sensor relative to a known frame of reference. For ranging sensors such as light detection and ranging (LiDAR) used in field robotic applications, the quality of the registration determines the utility of the range measurements. This paper makes two contributions. The first is the introduction of a new method, termed maximum sum of evidence (MSoE) for registering three‐dimensional LiDAR sensors to moving platforms. This method is shown to produce more accurate registration solutions than two leading methods for these sensors, the adaptive structure registration filter (ASRF) and Rényi quadratic entropy (RQE). The second contribution of the paper is to study the accuracy of the MSoE registration against these two other approaches. One of these, like the MSoE, requires a truth model of the environment. The other, a model‐free method, seeks the registration that minimizes the RQE of a compound point cloud. The main finding of this investigation is that while the model‐based methods prove more accurate than the model‐free approach, the results of all three methods are fit for their intended field robotic applications. This leads us to conclude that registration based on RQE is preferable in many, if not all, field robotic applications for reasons of convenience, since a truth model of the environment is not required.

Data-driven robotic sampling for marine ecosystem monitoring

Robotic sampling is attractive in many field robotics applications that require persistent collection of physical samples for ex-situ analysis. Examples abound in the earth sciences in studies involving the collection of rock, soil, and water samples for laboratory analysis. In our test domain, marine ecosystem monitoring, detailed understanding of plankton ecology requires laboratory analysis of water samples, but predictions using physical and chemical properties measured in real-time by sensors aboard an autonomous underwater vehicle (AUV) can guide sample collection decisions. In this paper, we present a data-driven and opportunistic sampling strategy to minimize cumulative regret for batches of plankton samples acquired by an AUV over multiple surveys. Samples are labeled at the end of each survey, and used to update a probabilistic model that guides sampling during subsequent surveys. During a survey, the AUV makes irrevocable sample collection decisions online for a sequential stream of candidates, with no knowledge of the quality of future samples. In addition to extensive simulations using historical field data, we present results from a one-day field trial where beginning with a prior model learned from data collected and labeled in an earlier campaign, the AUV collected water samples with a high abundance of a pre-specified planktonic target. This is the first time such a field experiment has been carried out in its entirety in a data-driven fashion, in effect “closing the loop” on a significant and relevant ecosystem monitoring problem while allowing domain experts (marine ecologists) to specify the mission at a relatively high level.

Calibration and Evaluation of Parameters in A 3D Proximity Rotating Scanner

This paper presents the procedures enabling the calibration and evaluation of intrinsic parameters in a Velodyne multi-beam laser scanner. As the device will be utilized in field robotics applications, both the evaluated parameters and the calibration are generally aimed to improve the performance of the scanner with advanced mapping algorithms such as robot evidence grids or octree. The measured parameters are compared with the data provided by the manufacturer. A novel calibration method based on conditional adjustment for correlated measurements is proposed and compared with factory calibration. DOI: http://dx.doi.org/10.5755/j01.eee.21.1.7299

Automatic Calibration of Spinning Actuated Lidar Internal Parameters

Actuated lidar, where a scanning lidar is combined with an actuation mechanism to scan a three‐dimensional volume rather than a single line, has been used heavily in a wide variety of field robotics applications. Common examples of actuated lidar include spinning/rolling and nodding/pitching configurations. Due to the construction of actuated lidar, the center of rotation of the lidar mirror may not coincide with the center of rotation of the actuation mechanism. To triangulate a precise point cloud representation of the environment, the centers of rotation must be brought into alignment using a suitable calibration procedure. We refer to this problem as estimating the internal parameters of actuated lidar. In this work, we focus on spinning/rolling lidar and present a fully automated algorithm for calibration using generic scenes without the need for specialized calibration targets. The algorithm is evaluated on a range of real and synthetic data and is shown to be robust, accurate, and have a large basin of convergence.

Modular and versatile platform for the benchmarking of modern actuators for robots

This work presents a test platform for the assessment and benchmarking of modern actuators which have been specifically developed for the new field and service robotics applications. This versatile platform has been designed for the comparative analysis of actuators of dissimilar technology and operating conditions. It combines a modular design to adapt to linear and rotational actuators of different sizes, shapes and functions, as well as those with different load capacities, power and displacement. This test platform emulates the kinematics of robotic joints while an adaptive antagonist-load actuator allows reproducing the variable dynamic loads that actuators used in real robotics applications will be subjected to. A data acquisition system is used for monitoring and analyzing test actuator performance. The test platform combines hardware and software in the loop to allow actuator performance characterization. The use of the proposed test platform is demonstrated through the characterization and benchmarking of three controllable impedance actuators recently being incorporated into modern robotics.

Monocular odometry in country roads based on phase‐derived optical flow and 4‐DOF ego‐motion model

PurposePositioning is a key task in most field robotics applications but can be very challenging in GPS‐denied or high‐slip environments. The purpose of this paper is to describe a visual odometry strategy using only one camera in country roads.Design/methodology/approachThis monocular odometery system uses as input only those images provided by a single camera mounted on the roof of the vehicle and the framework is composed of three main parts: image motion estimation, ego‐motion computation and visual odometry. The image motion is estimated based on a hyper‐complex wavelet phase‐derived optical flow field. The ego‐motion of the vehicle is computed by a blocked RANdom SAmple Consensus algorithm and a maximum likelihood estimator based on a 4‐degrees of freedom motion model. These as instantaneous ego‐motion measurements are used to update the vehicle trajectory according to a dead‐reckoning model and unscented Kalman filter.FindingsThe authors' proposed framework and algorithms are validated on videos from a real automotive platform. Furthermore, the recovered trajectory is superimposed onto a digital map, and the localization results from this method are compared to the ground truth measured with a GPS/INS joint system. These experimental results indicate that the framework and the algorithms are effective.Originality/valueThe effective framework and algorithms for visual odometry using only one camera in country roads are introduced in this paper.

A Mine on Its Own

This article presents a study of Gaussian process (GP) models applied to the problems of modeling and data fusion in the context of large-scale terrain modeling. The proposed model naturally provides a multiresolution representation of space, incorporates and handles uncertainties aptly, and copes with incompleteness of sensory information. These attributes are considered essential to support most field robotics applications, including autonomous mining. GP regression techniques are applied to estimate and interpolate (to fill gaps in occluded areas) elevation information across the field. GP approximation methods are introduced to enable the application of the proposed techniques to large data sets. To obtain a comprehensive model of complex terrain, typically, multiple sensory modalities and multiple data sets are required. The GP modeling approach is consequently extended to fuse multiple, multimodal data sets to obtain a best estimate of the elevation given the individual data sets. Two different GP-based concepts are applied to perform data fusion-heteroscedastic GPs and dependent GPs (DGPs). Thus, this article presents a report on an ongoing study of the use of GPs and several GPbased concepts to the problem of large-scale terrain modeling in the context of mining automation.

Locomotion Control of a Hydraulically Actuated Hexapod Robot by Robust Adaptive Fuzzy Control with Self-Tuned Adaptation Gain and Dead Zone Fuzzy Pre-compensation

Hydraulically actuated robotic mechanisms are becoming popular for field robotic applications for their compact design and large output power. However, they exhibit nonlinearity, parameter variation and flattery delay in the response. This flattery delay, which often causes poor trajectory tracking performance of the robot, is possibly caused by the dead zone of the proportional electromagnetic control valves and the delay associated with oil flow. In this investigation, we have proposed a trajectory tracking control system for hydraulically actuated robotic mechanism that diminishes the flattery delay in the output response. The proposed controller consists of a robust adaptive fuzzy controller with self-tuned adaptation gain in the feedback loop to cope with the parameter variation and disturbances and a one-step-ahead fuzzy controller in the feed-forward loop for hydraulic dead zone pre-compensation. The adaptation law of the feedback controller has been designed by Lyapunov synthesis method and its adaptation rate is varied by fuzzy self-tuning. The variable adaptation rate helps to improve the tracking performance without sacrificing the stability. The proposed control technique has been applied for locomotion control of a hydraulically actuated hexapod robot under independent joint control framework. For tracking performance of the proposed controller has also been compared with classical PID controller, LQG state feedback controller and static fuzzy controller. The experimental results exhibit a very accurate foot trajectory tracking with very small tracking error with the proposed controller.

Stereo vision three-dimensional terrain maps for precision agriculture

The combined interest in precision agriculture, information technology, and autonomous navigation has led to a growing interest in the generation of 3D maps of mobile equipment surroundings. This article proposes a method to create 3D terrain maps by combining the information captured with a stereo camera, a localization sensor, and an inertial measurement unit, all installed on a mobile equipment platform. The perception engine comprises a compact stereo camera that captures field scenes and generates 3D point clouds, which are transformed to geodetic coordinates and assembled in a global field map. The results showed that stereo perception can provide the level of detail and accuracy needed in the construction of 3D field maps for precision agriculture and field robotics applications.

Visual Servoing of Fully Actuated Mobile Robots with Significant Dynamics

A robust control design is proposed for image based visual servoing of fully actuated systems with non-trivial dynamics. The proposed control uses only target centroid information and is highly robust to observed variations to the target such as may be due to variable lighting and other difficulties often encountered in field robotics applications. The proposed design methodology is simulated for the case of a fully actuated autonomous hovercraft.