Global Coordinate Frame Research Articles

Planar Reconstruction of Indoor Scenes from Sparse Views and Relative Camera Poses

Planar reconstruction detects planar segments and deduces their 3D planar parameters (normals and offsets) from the input image; this has significant potential in the fields of digital preservation of cultural heritage, architectural design, robot navigation, intelligent transportation, and security monitoring. Existing methods mainly employ multiple-view images with limited overlap for reconstruction but lack the utilization of the relative position and rotation information between the images. To fill this gap, this paper uses two views and their relative camera pose to reconstruct indoor scene planar surfaces. Firstly, we detect plane segments with their 3D planar parameters and appearance embedding features using PlaneRCNN. Then, we transform the plane segments into a global coordinate frame using the relative camera transformation and find matched planes using the assignment algorithm. Finally, matched planes are merged by tackling a nonlinear optimization problem with a trust-region reflective minimizer. An experiment on the Matterport3D dataset demonstrates that the proposed method achieves 40.67% average precision of plane reconstruction, which is an improvement of roughly 3% over Sparse Planes, and it improves the IPAA-80 metric by 10% to 65.7%. This study can provide methodological support for 3D sensing and scene reconstruction in sparse view contexts.

Robust Control of Dual-Linear-Motor-Driven Gantry Stage for Coordinated Contouring Tasks Based on Feed Velocity

Contouring control is one of the most important tasks for the dual-linear-motor-driven gantry system (DLMDGS), whose control performance determines the in various precision machining equipment, mainly industrial machine tools. Nevertheless, there are some limitations in present results, including the insufficient consideration of system uncertainty and the approximation errors introduced by the computation of the path length. To overcome these issues, the effects of the rotational dynamic and uncertainties are studied and further refined in this article, and a robust control scheme with adaptive neural network identification is designed based on the desired contouring velocity. The method improves existing approaches and avoids the imperfection of the previous one, based on tracking control under global task coordinate frame. The effects of various environmental forces are considered and the B-spline wavelet neural network is employed to approximate unknown dynamic caused by the thrust ripples of the three motors. Some experiments on a real system verify the effectiveness and superiority of the proposed velocity-based contouring control method.

A Switched Systems Approach to Multiagent System Consensus: A Relay–Explorer Perspective

The rendezvous or position consensus problem is a fundamental topic within the multiagent system (MAS) literature and has numerous engineering applications. The majority of recent results that solve the position consensus problem rely on communication networks and each agent's ability to obtain position information via direct measurements and communication. Distributed coordination strategies for MASs that are network free and have agents that cannot directly measure position information are scarce. In this work, we develop a relay–explorer strategy to achieve position consensus at a common, desired location. In particular, a group of explorer agents, lacking global position sensors, use open-loop estimators of their position to independently dead reckon toward the desired location. To prevent the difference between the estimated and true position of each explorer from growing beyond a user-defined threshold, a mobile information service provider (relay agent) that is capable of measuring its position in the global coordinate frame, intermittently visits each explorer to provide position information as determined by a maximum dwell-time condition. The relay agent has a position estimator for each explorer, and each estimator is synchronized with the corresponding explorer. The relay agent uses these position estimators to locate each explorer, maneuver to them, and provide position feedback. The contribution of this work over our precursory results is the consideration of uncertain explorer agent dynamics, which are estimated online using recurrent neural networks and integral concurrent learning. The estimated dynamics serve as feed-forward model approximations in the position estimators used by the explorers and relay agent, which generate more accurate position estimates once a finite excitation condition is satisfied. The MAS is modeled as a switched system, and a Lyapunov-based analysis is used to derive a maximum dwell-time condition for each explorer, prove the MAS is exponentially regulated to the desired location, and show the error between the estimated and true explorer dynamics is uniformly ultimately bounded. Experiments and multiple simulation examples are provided to verify the theoretical development, explore the scalability and learning performance of the approach, and shed light on future extensions.

Event-Triggered Pose Synchronization in SE(3) for Cooperating Multi-Robot Teams

Consensus and synchronization are fundamental concepts for the coordination of cooperating multi-robot teams. Applications like cooperative manipulation may require not only synchronization of the position, but also of the orientation of the individual agents. The pose of the agent can be described within the special Euclidean group and common results for coordination have to be adapted. We propose a control framework for full-pose synchronization in SE(3) for a team of Euler-Lagrange agents, relying only on relative information in the absence of a global coordinate frame. The framework consists of an inner loop for feedback linearization and an outer loop for pose synchronization. The measurements are taken by external sensors and communicated to the respective agents via a common communication network. To deal with limited communication bandwidth, we propose an event-triggered update strategy for the relative measurements. Finally, the efficacy of the proposed control and triggering law is illustrated in simulations.

Online Cartesian Compliance Shaping of Redundant Robots in Assembly Tasks

This paper presents a universal approach to shaping the mechanical properties of the interaction between a collaborative robot and its environment through an end-effector Cartesian compliance shaping. More specifically, the focus is on the class of kinematically redundant robots, for which a novel redundancy reconfiguration scheme for online optimization of the Cartesian compliance of the end-effector is presented. The null-space reconfiguration aims to enable the more efficient and versatile use of collaborative robots, including robots with passive compliant joints. The proposed approach is model-based and gradient-based to enable real-time computation and reconfiguration of the robot for Cartesian compliance while ensuring accurate position tracking. The optimization algorithm combines two coordinate frames: the global (world) coordinate frame commonly used for end-effector trajectory tracking; and the coordinate frame fixed to the end-effector in which optimization is computed. Another attractive feature of the approach is the bound on the magnitude of the interaction force in contact tasks. The results are validated on a torque-controlled 7-DOF KUKA LWR robot emulating joint compliance in a quasi-static experiment (the robot exerts a force on an external object) and a peg-in-hole experiment emulating an assembly task.

GNSS-denied UAV indoor navigation with UWB incorporated visual inertial odometry

The localization of unmanned aerial vehicles (UAVs) in GPS-denied areas is an essential issue for indoor navigation. This paper presents a technique to improve the accuracy of visual inertial odometry (VIO) by combining the ultra-wideband (UWB) positioning technology. The proposed architecture is divided into two stages. In the initial stage, the constraint on UWB short-term position change is adopted to improve the pose estimation results of the VIO system. It is also used to mitigate the translation error caused by the vibration and lack of features during the flight. In the second stage, a loose coupling approach based on nonlinear optimization is utilized to fuse the local pose estimator of the VIO system with the global constraints from the UWB positioning. At the beginning of each operation, the alignment between the VIO and UWB frames is estimated to avoid the influence of coordinate transformation due to the VIO cumulative error. It is shown that our optimization-based fusion method is able to achieve a smooth localization trajectory under the global coordinate frame. In the experiments, the performance evaluation carried out in the real-world scenes has demonstrated the effectiveness of the proposed technique.

Simultaneous localization and formation using angle-only measurements in 2D

This paper solves the simultaneous localization and formation (SLAF) problem for a multi-agent system moving in 2D plane. The multi-agent system consists of leaders who have the knowledge of their absolute positions in the global coordinate frame, and followers who do not know their absolute positions but have angle-only measurements and communication with respect to their neighboring agents. The aim of SLAF is to simultaneously localize and control the followers such that a desired formation among the leaders and followers can be achieved by using locally available sensing and communication information. To handle the challenging situation where the formation becomes unlocalizable at some nongeneric configurations, a perturbation-based SLAF algorithm is proposed such that the SLAF task can be achieved with an asymptotic convergence. To meet different tasks’ requirements, three types of distributed SLAF algorithms are designed for the followers when the leaders are static, move with constant, or time-varying velocities, respectively. The effect of measurement noises, extension to other types of sensor measurements, requirement on agents’ coordinate frames, collision and collinearity avoidance are also discussed. To validate the theoretical results, simulation examples corresponding to the discussed scenarios are carried out.

Discrete-Time Optimal Control of Miniature Helical Swimmers in Horizontal Plane

Microswimmer and miniswimmer toward precision-targeted medicine have attracted extensive attention recently. We have developed an autonomous manipulation approach for magnetic-driven helical miniswimmer at low Reynolds number in the horizontal plane (<inline-formula> <tex-math notation="LaTeX">$H$ </tex-math></inline-formula>-plane). Different from our previous work which just makes the barycenter of miniswimmer on the reference path as well as takes the swimming direction not into consideration in planar path following, our control policy in this article can make the miniswimmer to follow the reference path and, at the same time, its swimming direction is also along the reference path. A robust tracking method is employed to locate the helical miniswimmer in real time. Due to different external disturbances, an angle compensating model in the global coordinate frame is developed by radial basis function (RBF) networks trained by backpropagation algorithms, which is used to express the swimming model of the helical miniswimmer facing the gravity and lateral disturbances. A discrete-time optimal controller is formulated based on the linear-quadratic feedback control. Simulations and experiments are conducted to quantitatively validate the autonomous manipulation, and the results show the control performance with submillimeter accuracy in the <inline-formula> <tex-math notation="LaTeX">$H$ </tex-math></inline-formula>-plane. <i>Note to Practitioners:</i> This article is motivated by the potential application of precision-targeted medicine using magnetic-driven microswimmer/miniswimmer. The formulated controller employs the error model in the horizontal plane to design the control law. Simulations and experiments validate the effectiveness of the proposed discrete-time optimal control scheme using magnetic-driven miniswimmers.

Robust Localization of Occluded Targets in Aerial Manipulation Via Range-Only Mapping

This letter studies the problem of target localization in aerial manipulation tasks. When an aerial robot flies close to a target to manipulate, the target would be occluded by the onboard robotic manipulator occasionally or for a long period of time. It is, however, necessary to continuously localize the target even when it is occluded. This is a unique problem in aerial manipulation and has not been addressed in other tasks such as ground manipulation. To solve this problem, we propose a new approach of target-centered range-only mapping based on RGB-D measurements. Different from the traditional map representation where the 3D locations of the feature points are stored in a global reference coordinate frame, we store the distances from the target to the environmental feature points only. It thus results in a coordinate-free target-centered range-only map. When the target is occluded, it can still be localized accurately based on the map. The proposed approach is verified by both synthetic and real datasets. The experimental results demonstrate the superior performance of our proposed approach to the state-of-the-art RGB-D ORB-SLAM3 method.

2D LiDAR based object detection and tracking on a moving vehicle

Environment perception is an important capability for fully autonomous operation of agricultural machinery. High-precision work requires perception systems that are able to locate objects with high and specified accuracy. In this paper, we present an object detection and tracking system in a global coordinate frame based on 2D LiDAR data and RTK-GNSS positioning. The LiDARs and the RTK-GNSS device are installed on a tractor that is moving while the data is recorded. The final mapping result is compared to an RTK-GNSS based ground truth measurement of the object positions. For a test setup with four objects on a clover field, the average object positioning accuracy of the LiDAR system was determined to be 0.135 m (CEP50) and the worst case accuracy was 0.178m (CEP50). The preliminary results from this study can be used as a basis for determining requirements for safety margins when using this system for autonomous driving.

Measurement of the positions of light sensors on a mobile frame being tracked with a lighthouse localization system

The laser lighthouse is a nearly 40 year old concept that only recently had a practical incarnation in the HTC Vive VR kit base station, a single beacon that sweeps a light sheet at a fixed frequency like a lighthouse. It can be placed on a flat surface in any position from where it has a line of sight to a mobile frame equipped with light sensors and some computing power to record and process time stamps at the passage of the light sheet. From the recorded timestamp and the positions of the sensors in the mobile frame, the position of the frame in the (global) coordinate frame of the lighthouse can be established. Surface applicable light sensors have been announced for this purpose, that can turn any object into a trackable object with the laser lighthouse system. The accuracy of the positions of the sensors in the mobile frame is cardinal to the accuracy of the calculated position of the mobile frame. This presents a challenge as the positions of sensors on the surface of an object are not easy to obtain in a 3D coordinate frame. Even if they are available in a CAD design, manufacturing tolerances might cause the positions of the sensors to deviate from their designed positions. In this first paper on this subject, a method is proposed to measure the position of the sensors in the mobile frame using (hand-) measurement of the distance between a small number of designated features of the mobile frame only. The positions of the light sensors in the mobile frame are derived from the timestamps recorded by the light sensors in a procedure that subjects the mobile frame to constrained rotations. The efficacy of the proposed method is tested and validated experimentally. The sensor positions obtained with the method improve the accuracy of the localization of the mobile frame.

Distributed bearing-based formation control of unmanned aerial vehicle swarm via global orientation estimation

Most existing formation control approaches for Unmanned Aerial Vehicle (UAV) swarm assume that global position and global coordinate frame are directly available for each agent. To extend the application domain, this paper proposes a distributed bearing-based formation control scheme, without any reliance on global position or global coordinate frame. The interactions among UAVs are described by a directed topology with two-leader structure. To address the issue of unavailable global coordinate frame, we first present a distributed orientation estimation law for each UAV to determine its orientation under the coordinate frame of the first leader. Based on the orientation estimation, we then design a bearing-based formation control law to globally asymptotically track target moving formations. Finally, simulation results are provided to validate the proposed method, which show that the translation, scale and orientation of the formation can be flexibly controlled via two leaders.

R-LOAM: Improving LiDAR Odometry and Mapping With Point-to-Mesh Features of a Known 3D Reference Object

LiDAR-based odometry and mapping is used in many robotic applications to retrieve the robot's position in an unknown environment and allows for autonomous operation in GPS-denied (e.g., indoor) environments. With a 3D LiDAR sensor, highly accurate localization becomes possible, which enables high quality 3D reconstruction of the environment. In this letter we extend the well-known LOAM framework by leveraging prior knowledge about a reference object in the environment to further improve the localization accuracy. This requires a known 3D model of the reference object and its known position in a global coordinate frame. Instead of only relying on the point features in the mapping module of LOAM, we also include mesh features extracted from the 3D triangular mesh of the reference object in the optimization problem. For fast correspondence computation of mesh features, we use the Axis-Aligned-Bounding-Box-Tree (AABB) structure. Essentially, our approach not only makes use of the previously built map for absolute localization in the environment, but also takes the relative position to the reference object into account, effectively reducing long-term drift. To validate the proposed concept, we generated datasets using the Gazebo simulation environment in exemplary visual inspection scenarios of an airplane inside a hangar and the Eiffel Tower. An actuated 3D LiDAR sensor is mounted via a 1-DoF gimbal on a UAV capturing 360° scans. We benchmark our approach against the state-of-the-art open-source LOAM framework. The results show that the proposed joint optimization using both point and mesh features yields a significant reduction in Absolute Pose Error (APE) and therefore improves the map and 3D reconstruction quality during long-term operations.

Accurate and dense point cloud generation for industrial Measurement via target-free photogrammetry

Industrial photogrammetry systems commonly require multiple coded targets to establish a global coordinate frame for relatively large objects. However, there are many industrial products that do not allow coded targets to be placed on their surfaces. We propose an accurate and dense point cloud generation approach for measuring large-sized, untextured objects. Unlike most existing industrial measurement methods, our photogrammetry approach is free of coded targets. We have three core contributions. First, a Rotation-Free Digital Image Correlation (RFDIC) method is proposed to improve the multi-view stereopsis matching precision. Second, based on the technique of structure from motion (SfM), a coarse-to-fine strategy is utilized to construct the multi-view geometry and accurately optimize camera poses. Third, the devices used are consumer-friendly and commonly available, i.e., only a digital projector and a camera are needed to obtain the dense points on the measured object surface. The experimental results show that the error of reconstructed length for a scale bar is less than 0.01 mm/m. Compared to the state-of-the-art commercial measurement system, the average error of point cloud reconstruction is about 0.055 mm, which meets the accuracy demand for industrial applications.

A novel geometry error measurement methodology for coaxiality evaluation

In this paper, we proposed a new device for geometry errors measurement and coaxiality evaluation, and the corresponding methodology for coaxiality evaluation from measurement data is presented, which allows to characterize multiple holes at a time. Unlike traditional measurement system a laser sensor is mounted onto out of the holes so that multi-hole surfaces can be “seen” by the senor when it rotates around a fixed axis. First the intersections (or ellipse profiles) of the sensor’s scanning plane and holes, are computed by fitting. Then, the center coordinates and profile points of the ellipse are computed and transformed to the 3D global coordinate frame. Finally the centerline of the hole is determined from the 3D profile points by using a weighted least-squares fitting algorithm. In addition, to reduce the effect of noises on the measurement result, error analysis and compensation techniques are studied to improve the measurement accuracy. A case study is presented to validate the measurement principle and data processing approach.

Angle-Displacement Rigidity Theory With Application to Distributed Network Localization

This article investigates the localization problem of a network in 2-D and 3-D spaces given the positions of anchor nodes in a global frame and internode relative measurements in local coordinate frames. It is assumed that the local frames of different nodes have different unknown orientations. First, an angle-displacement rigidity theory is developed, which can be used to localize all the free nodes by the known positions of the anchor nodes and local relative measurements (local relative position, distance, local relative bearing, angle, or ratio-of-distance measurements). Then, necessary and sufficient conditions for network localizability are given. Finally, a distributed network localization protocol is proposed, which can globally estimate the locations of all the free nodes of a network if the network is infinitesimally angle-displacement rigid. The proposed method unifies local-relative-position-based, distance-based, local-relative-bearing-based, angle-based, and ratio-of-distance-based distributed network localization approaches. The novelty of this article is that the proposed method can be applied in both generic and nongeneric configurations with an unknown global coordinate frame in both 2-D and 3-D spaces.

Reflection and transmission approximations for weak contrast orthorhombic media

Subsurface media are in general anisotropic, and this fact should be taken into account for analyzing reflection and transmission (R/T) coefficients. Orthorhombic (ORT) media are commonly regarded as a practical symmetry system to account for polar anisotropy and azimuthal anisotropy. We have focused on the model made up of two welded ORT half-spaces to analyze the R/T coefficients normalized by vertical energy flux. The two half-spaces have azimuthally nonaligned vertical symmetry planes and are parameterized in two local 3D Cartesian coordinate frames, respectively. The vertical coordinate planes in each local frame coincide with the vertical symmetry planes for the corresponding ORT half-spaces. Under the weak contrast assumption for the two half-spaces, this model is taken as the perturbed model in R/T approximations with the perturbation theory. The unperturbed model is also composed of two unperturbed ORT half-spaces with their different vertical symmetry plane orientations inheriting the counterparts for the perturbed half-spaces above and below the interface, respectively. The unperturbed ORT half-spaces above and below the interface have the same model parameters defined in the two local coordinate frames, respectively. With the perturbations respectively evaluated in the local coordinate frames above and below the interface, the azimuth angle that indicates the local frames’ azimuthal difference is decoupled from the model parameter contrasts. Compared with the traditional approximation method with the perturbation theory in a global coordinate frame, the proposed R/T approximations depend on fewer model parameter discontinuities. We also consider the isotropic background medium under the weak anisotropy assumption. Influences of S-wave singularity points are mitigated by introducing pseudowaves for approximations. Numerical tests are implemented to demonstrate the accuracy.

3-D Distributed Localization With Mixed Local Relative Measurements

This paper studies 3-D distributed network localization using mixed types of local relative measurements. Each node holds a local coordinate frame without a common orientation and can only measure one type of information (relative position, distance, relative bearing, angle, or ratio-of-distance measurements) about its neighboring nodes in its local coordinate frame. A novel rigidity-theory-based distributed localization is developed to overcome the challenge due to the absence of a global coordinate frame. The main idea is to construct displacement constraints for the positions of the nodes by using mixed local relative measurements. Then, a linear distributed localization algorithm is proposed for each free node to estimate its position by solving the displacement constraints. The algebraic condition and graph condition are obtained to guarantee the global convergence of the proposed distributed localization algorithm.

Multisensor Poisson Multi-Bernoulli Filter for Joint Target–Sensor State Tracking

In a typical multitarget tracking (MTT) scenario, the sensor state is either assumed known, or tracking is performed in the sensor's (relative) coordinate frame. This assumption does not hold when the sensor, e.g., an automotive radar, is mounted on a vehicle, and the target state should be represented in a global (absolute) coordinate frame. Then it is important to consider the uncertain location of the vehicle on which the sensor is mounted for MTT. In this paper, we present a multisensor low complexity Poisson multi-Bernoulli MTT filter, which jointly tracks the uncertain vehicle state and target states. Measurements collected by different sensors mounted on multiple vehicles with varying location uncertainty are incorporated sequentially based on the arrival of new sensor measurements. In doing so, targets observed from a sensor mounted on a well-localized vehicle reduce the state uncertainty of other poorly localized vehicles, provided that a common non-empty subset of targets is observed. A low complexity filter is obtained by approximations of the joint sensor-target state density minimizing the Kullback-Leibler divergence (KLD). Results from synthetic as well as experimental measurement data, collected in a vehicle driving scenario, demonstrate the performance benefits of joint vehicle-target state tracking.

3-D Path Following of Helical Microswimmers With an Adaptive Orientation Compensation Model

Controlling magnetic microswimmers toward 3-D manipulation tasks has received considerable attention. Although related studies on manipulating helical microswimmers have been developed, stable closed-loop controls and accuracy swimming models should be still investigated. This article addresses the problem of 3-D path following for magnetically driven helical microswimmers with an adaptive-compensation scheme. The orientation-compensation model in the global coordinate frame is learned by radial basis function (RBF) networks trained with backpropagation algorithms, which is used to express the motion of the helical microswimmer in the presence of the weight of the swimmer and lateral disturbances from the boundary effects. A proxy-based sliding-mode control (PSMC) approach is developed to design stable controllers based on the kinematic error model. The effects of variable parameters and boundary effects are also considered. Experimental results including different paths in 3-D space validated the path following with submillimeter accuracy using the helical microswimmer. Note to Practitioners —This article is motivated by the issue of the following predefined paths for magnetically driven helical microswimmers in 3-D space. The proposed closed-loop controller employs the error model in 3-D space to formulate the control law according to an orientation-compensation model learned by neural networks. It is demonstrated that the helical microswimmer is able to follow different paths in 3-D space with submillimeter accuracy using the proposed control scheme.