World Coordinate Frame Research Articles

Tightly Coupled, Graph-Based DVL/IMU Fusion and Decoupled Mapping for SLAM-Centric Maritime Infrastructure Inspection

In this article, we address the problem of simultaneous localization and mapping (SLAM)-centric maritime infrastructure inspection [using unmanned surface vehicles (USVs)] via novel approaches in tightly-coupled, graph-based DVL/IMU fusion and decoupled mapping. As our first contribution, we formalize the preintegration of linear velocity measurements, obtained by a Doppler velocity log (DVL), in combination with angular velocity measurements, obtained by an inertial measurement unit (IMU), as binary factors encoding relative position. To evaluate state estimation improvements imparted by DVL/IMU fusion, we implement our proposed factor within a state-of-the-art, graph-based lidar-visual-inertial (LVI) SLAM system as our second contribution. Accuracy and robustness improvements are demonstrated in simulation by comparing maximum <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">a posteriori</i> pose estimates with and without DVL/IMU fusion against ground truth poses. As our third contribution, we propose a map generation framework for downstream inspection applications decoupled from SLAM. In our framework, volumetric data (captured by sonar, lidar, etc.) is transformed into a common world coordinate frame using extrinsic calibrations and SLAM pose estimates as input. Our framework operates over the complete set of raw volumetric data, whereas SLAM systems (both online and offline) typically operate over a subset of down-sampled volumetric data. To address the processing of additional volumetric data, we present innovations in refined pose correction and staged filtering for user-controlled denoising. We experimentally evaluate our map generation framework against the LVI SLAM system adopted for this study using real-world data and demonstrate improvements to map quality metrics important to inspection.

A Hybrid Approach to 3D Shape Estimation of Catheters Using Ultrasound Images

Catheter-based cardiac interventions have substantially improved the efficacy of the treatment. However, catheter navigation still poses several challenges including its poor visualization and localisation. Therefore, various imaging methods have been used to localize the catheter and estimate its shape. The use of ultrasound imaging is superior to other modalities in many aspects, but suffers from poor spatial resolution. Hence, we present a hybrid approach involving deep learning and classical approach to 3D shape estimation of catheters. Our novel approach uses two stages. First, a UNet-3D model is proposed to estimate the catheter centroid in the ultrasound image. Then, an adaptive Kalman Filter (AKF) is used to fuse the points into the 3D world coordinate frame. Simulation studies, phantom and ex-vivo experimentation results demonstrate the robustness of the method to ultrasound noise and extreme configurations (sharp curves).

Online Cartesian Compliance Shaping of Redundant Robots in Assembly Tasks

This paper presents a universal approach to shaping the mechanical properties of the interaction between a collaborative robot and its environment through an end-effector Cartesian compliance shaping. More specifically, the focus is on the class of kinematically redundant robots, for which a novel redundancy reconfiguration scheme for online optimization of the Cartesian compliance of the end-effector is presented. The null-space reconfiguration aims to enable the more efficient and versatile use of collaborative robots, including robots with passive compliant joints. The proposed approach is model-based and gradient-based to enable real-time computation and reconfiguration of the robot for Cartesian compliance while ensuring accurate position tracking. The optimization algorithm combines two coordinate frames: the global (world) coordinate frame commonly used for end-effector trajectory tracking; and the coordinate frame fixed to the end-effector in which optimization is computed. Another attractive feature of the approach is the bound on the magnitude of the interaction force in contact tasks. The results are validated on a torque-controlled 7-DOF KUKA LWR robot emulating joint compliance in a quasi-static experiment (the robot exerts a force on an external object) and a peg-in-hole experiment emulating an assembly task.

Self-tuning anti-sway control for shipboard cranes providing combined world and deck-frame compensation

This paper presents a novel self-tuning anti-sway control system for shipboard cranes that provides full, six degree-of-freedom (DOF) motion compensation with respect to both the world (ocean) coordinate frame and the ship deck coordinate frame. The system can either manually or automatically switch between compensation modes during operation, and is capable of providing anti-sway action while tracking a time-varying operator input. Rather than requiring the operator to provide individual joint control, the entire system can track a simple three-axes Cartesian input from the operator, while simultaneously providing full anti-sway compensation in either coordinate frame.High-fidelity simulation case studies were performed with a nine-DOF shipboard knuckle boom crane, which included double pendulum dynamics between the hook and payload, in the presence of full six-DOF ship motion at sea-state six. The self-tuning anti-sway system showed at least a 92.70% reduction in payload tracking error when the corresponding compensation mode (either deck or world-frame) was activated, a significant reduction in undesired payload motion. Wind disturbances were added to the simulation using the Dryden wind model, where the system showed a maximum decrease in performance of only 0.73%; additional tests were run with self-tuning disabled, which resulted in a decrease in performance of up to 101%, indicating self-tuning is highly effective at minimizing the effect of wind disturbances.

Hand Gesture Recognition on a Resource-Limited Interactive Wristband.

Most of the reported hand gesture recognition algorithms require high computational resources, i.e., fast MCU frequency and significant memory, which are highly inapplicable to the cost-effectiveness of consumer electronics products. This paper proposes a hand gesture recognition algorithm running on an interactive wristband, with computational resource requirements as low as Flash < 5 KB, RAM < 1 KB. Firstly, we calculated the three-axis linear acceleration by fusing accelerometer and gyroscope data with a complementary filter. Then, by recording the order of acceleration vectors crossing axes in the world coordinate frame, we defined a new feature code named axis-crossing code. Finally, we set templates for eight hand gestures to recognize new samples. We compared this algorithm’s performance with the widely used dynamic time warping (DTW) algorithm and recurrent neural network (BiLSTM and GRU). The results show that the accuracies of the proposed algorithm and RNNs are higher than DTW and that the time cost of the proposed algorithm is much less than those of DTW and RNNs. The average recognition accuracy is 99.8% on the collected dataset and 97.1% in the actual user-independent case. In general, the proposed algorithm is suitable and competitive in consumer electronics. This work has been volume-produced and patent-granted.

Adaptive model predictive control for an omnidirectional mobile robot with friction compensation and incremental input constraints

In this paper, an adaptive model predictive control (MPC) scheme with friction compensation, subject to incremental control input constraints and parameter uncertainties, is proposed for a three-wheeled omnidirectional mobile robot (OMR). The proposed control framework is in a cascaded structure, wherein the outer-loop is kinematic-based control and the inner-loop is designed based on adaptive linear MPC. First, a complex nonlinear dynamic model of the OMR in the world coordinate frame is transformed and partially linearized into a reduced nonlinear model in the moving coordinate system. The nonlinearity of the reduced model only arises from Coulomb friction. Then an estimated system is established for the reduced nonlinear system, with an adaptive update law estimating the system uncertain parameters. To facilitate the linear MPC design, part of the control efforts is derived by feedback compensation of the Coulomb friction forces, resulting in a linear estimated system. The other part is designed by a constrained linear MPC. Feasibility and stability analyses are given for the proposed adaptive MPC scheme. Finally, experimental comparisons with model-based MPC are carried out to verify the effectiveness of the proposed control scheme.

Hyper resolution image mosaics with unbounded vertical field of view

This paper presents the framework for the creation of hyper resolution image mosaics (HRIM) with a full horizontal and unbounded vertical field of view. Additionally, the hyper-resolution images are mapped to a format compatible with a geo referencing frame. In this manner the images can be viewed using standard mapping services. The measurement device consists of eight cameras covering the full 360 degree horizontal field. The unlimited vertical field of view is obtained by moving the device, e.g., by sinking it down a mine shaft. The cameras are synchronized so that with each frame the full horizontal field of view is captured. The camera sensors are calibrated to an onboard inertial measurement unit (IMU) which allows the reconstruction of the position of each sensor in a world coordinate frame for each captured image using rigid transformations. Perspective projection onto an open manifold (cylinder) allows for the integration of all images into an HRIM. Constrained tensor-polynomial approximations are used to obtain geometric maps, i.e., a distortion lattice, from the image coordinate system onto an open cylindrical manifold. From this lattice binary masks are derived for each projected image. A multi band blending algorithm is implemented for the final composition. The HRIM are then tiled and stored on a GIS mapping server from where the HRIM can be viewed on any device running an HTML browser. It should be noted that this approach provides a significant advantage since viewing the large HRIM is not computationally intensive and can even be done on tablets. The framework has been applied to the remote inspection of deep vertical shafts.

Vision-Based Localization System Suited to Resident Underwater Vehicles

In recent years, we have seen significant interest in the use of permanently deployed resident robotic vehicles for commercial inspection, maintenance and repair (IMR) activities. This paper presents a concept and demonstration, through offshore trials, of a low-cost, low-maintenance, navigational marker that can eliminate drift in vehicle INS solution when the vehicle is close to the IMR target. The subsea localisation marker system is fixed on location on the resident field asset and is used in on-vehicle machine vision algorithms for pose estimation and facilitation of a high-resolution world coordinate frame registration with a high refresh rate. This paper presents evaluation of the system during trials in the North Atlantic Ocean during January 2019. System performances and propagation of position error is inspected and estimated, and the effect of intermittent visual based position update to Kalman filter and onboard INS solution is discussed. The paper presents experimental results of the commercial state-of-the-art inertial navigation system operating in the pure inertial mode for comparison.

Using Geometric Interval Algebra Modeling for Improved Three-Dimensional Camera Calibration

This paper addresses the problem of estimating camera calibration parameters by using a novel method based on interval algebra. Unlike existing solutions, which usually apply real algebra, our method is capable of obtaining highly accurate parameters even in scenarios where the input data for camera calibration are severely corrupted by noise or no artificial calibration target can be introduced on the scene. We introduce some key concepts regarding the usage of interval algebra on projective space, which might be used by other computer vision methods. To demonstrate the robustness and effectiveness of our method, we present results for camera calibration with varying levels of noise on the input data of a world coordinate frame (standard deviation of up to 0.5 m) and their corresponding projections onto an image plane (standard deviation of up to 10 pixels), which are significantly larger than noise levels considered by state-of-the-art methods.

Autonomous Vehicles: Autodriver Algorithm and Vehicle Dynamics

A given road can be expressed mathematically in a global (or world) coordinate frame. Following the road can be substituted by following the loci of its curvature center and turning at the right circle of curvature. Considering that a vehicle in motion is always in turn about an instantaneous rotation center relative to the ground, an autonomous vehicle capable of following a given path by coinciding the rotation center of vehicle at every moment on the curvature center of the road could be designed. The dynamic reactions of the vehicle influence its path of motion and make its rotation center to depart from the desired path of the curvature center of the road. In this study, the Autodriver algorithm control strategy to front-wheel-steering vehicles has been developed and a control loop is introduced to compensate the present errors generated by the differences of the desired locating on the road and the real position of the vehicle.

A New Rotor Position Measurement Method for Permanent Magnet Spherical Motors

This paper proposes a new high-precision rotor position measurement (RPM) method for permanent magnet spherical motors (PMSMs). In the proposed method, a LED light spot generation module (LSGM) was installed at the top of the rotor shaft. In the LSGM, three LEDs were arranged in a straight line with different distances between them, which were formed as three optical feature points (OFPs). The images of the three OFPs acquired by a high-speed camera were used to calculate the rotor position of PMSMs in the world coordinate frame. An experimental platform was built to verify the effectiveness of the proposed RPM method.

A Sensor-Fusing System for Spatial Circle and Trunk Parameter Estimation

This paper proposes a fusing system consisting of a camera, a line laser, and an inertial measurement unit (IMU) for spatial circle and trunk parameter estimation. The line laser provides a stripe on the object surface that is captured by the camera. When using the system to scan an object, the 3D coordinate points on the surface of the scanned object are calculated with respect to a world coordinate frame by fusing the system movement and laser point coordinates, where the system movement is estimated from the IMU data by solving a smoothing optimization problem. Based on the information from the scanning process, algorithms are derived for detecting the spatial circle’s diameter and the trunk’s parameters. We also performed experiments to verify the feasibility of the system in real applications.

Golf Green Slope Estimation Using a Cross Laser Structured Light System and an Accelerometer

In this paper, we propose a method combining an accelerometer with a cross structured light system to estimate the golf green slope. The cross-line laser provides two laser planes whose functions are computed with respect to the camera coordinate frame using a least square optimization. By capturing the projections of the cross-line laser on the golf slope in a static pose using a camera, two 3D curves’ functions are approximated as high order polynomials corresponding to the camera coordinate frame. Curves’ functions are then expressed in the world coordinate frame utilizing a rotation matrix that is estimated based on the accelerometer’s output. The curves provide some important information of the green such as the height and the slope’s angle. The curves estimation accuracy is verified via some experiments which use OptiTrack camera system as a ground-truth reference.

Image distortion correction for micromanipulation system based on SLM microscopic vision.

Stereo light microscope (SLM) simulates stereo imaging principle of human eyes. Microscopic vision system based on SLM has become an important visual tool for micro measurement, micromanipulation, and microinjection. We develop a micromanipulation system based on SLM and present an image distortion correction method. We mainly correct two kinds of image distortions: lateral and vertical distortion. Distortion correction consists of two steps. First, a linear fitting algorithm for each row or column of target points is developed, and the fitting errors are calculated. If the fitting errors are smaller than a given threshold, the linear fitting results are kept and used. Otherwise polynomial fitting procedure will be used. Second, the parallelism of straight lines is corrected. The results show that a line in world coordinate frame (WCF) is not necessarily a straight line in image coordinate frame (ICF), or two parallel lines in WCF may be not parallel in ICF. Distortion correction can restore the parallel and linear relationship. For distorted left and right images, the magnitude of distortion exceeds 6 pixels and 4 pixels in the horizontal direction, and 1.2 pixels and 1.7 pixels in the vertical direction, respectively. After corrected, for left and right image, distortion can be reduced to 0.8 pixels and 0.7 pixels in the horizontal direction, and 0.96 pixels and 1.3 pixels in the vertical direction, respectively. The results show that distortion parameters obtained from the proposed method can effectively correct distorted images.

Attitude estimation and control of a 3-DOF lab helicopter only based on optical flow

This paper proposes an attitude estimation method based on optical flow to solve the attitude tracking control problem for a three degree-of-freedom (3-DOF) lab helicopter. First, the relationship between optical flow and the motion of a general unmanned aerial vehicle is derived from the transformation between the image coordinate frame and the world coordinate frame. Then, an expression for the angular velocity of the 3-DOF helicopter is deduced only based on optical flow, and the attitude information is acquired by solving nonlinear equations. Finally, using visual feedback, a linear quadratic regulation (LQR) controller is designed for hovering and tracking, which consists of a feedforward controller and a LQR state feedback controller. Closed-loop experimental results on the lab helicopter demonstrate the effectiveness of the proposed estimation and control methods.

NDT スキャンマッチングに基づく車載レーザスキャナ観測点群の高精度マッピング

This paper presents a method of accurately refining 3D laser scan data from in-vehicle multilayer laser scanner. The vehicle estimates its own 3D pose (position and attitude) based on Normal-Distributions-Transform (NDT) scan-matching method. Based on the pose information, the vehicle pose when the laser scan data is captured is extrapolated and interpolated under the assumption that the vehicle moves at an almost constant linear and turning velocities. The extrapolated and interpolated pose of the vehicle is applied to mapping laser scan data into the voxel map represented on the world coordinate frame. Subsequently, the vehicle again estimates its own 3D pose based on NDT scan matching. Such an iteration enable to compensate the distortion of laser scan date and to accurately map the data on the world coordinate frame. Experimental results show the performance of the proposed method.

Analyzing Behavior of Differential Drive Mobile Robot by Using Statistical Tools

Now a day's robotics provides better service to people anywhere in the world in every sector of life. This standard does not come in a day. Day by day the research on 'analyzing behavior' on robotics by roboticists, engineers, physicists, and mathematician made this standard of life. This paper is presented an analytical approach to comparing the performance of mobile robot. A scenario has been considered in which robot is in world coordinate frame along with a definite boundary. The test run has been taken by varying the microcontrollers and microcomputer. Then analyzed the data’s by graphical and numerical methods that predict a slope which confirms that more the advanced device has been used the robot reached the goal earlier.

Precise calibration of linear camera equipped with cylindrical lenses using a radial basis function-based mapping technique.

The linear camera equipped with cylindrical lenses has prominent advantages in high-precision coordinate measurement and dynamic position-tracking. However, the serious distortion of the cylindrical lenses limits the application of this camera. To overcome this obstacle, a precise two-step calibration method is developed. In the first step, a radial basis function-based (RBF-based) mapping technique is employed to recover the projection mapping of the imaging system by interpolating the correspondence between incident rays and image points. For an object point in 3D space, the plane passing through the object point in camera coordinate frame can be calculated accurately by this technique. The second step is the calibration of extrinsic parameters, which realizes the coordinate transformation from the camera coordinate frame to world coordinate frame. The proposed method has three aspects of advantage. Firstly, this method (black box calibration) is still effective even if the distortion is high and asymmetric. Secondly, the coupling between extrinsic parameters and other parameters, which is normally occurred and may lead to the failure of calibration, is avoided because this method simplifies the pinhole model and only extrinsic parameters are concerned in the simplified model. Thirdly, the nonlinear optimization, which is widely used to refine camera parameters, is better conditioned since fewer parameters are needed and more accurate initial iteration value is estimated. Both simulative and real experiments have been carried out and good results have been obtained.

Pose Estimation for General Cameras Using Lines.

In this paper, we address the problem of pose estimation under the framework of generalized camera models. We propose a solution based on the knowledge of the coordinates of 3-D straight lines (expressed in the world coordinate frame) and their corresponding image pixels. Previous approaches used the knowledge of the coordinates of 3-D points (zero dimensional elements) and their corresponding images (zero dimensional elements). In this paper, pixels belonging to the image of 3-D lines are used. There is no need to establish correspondences between pixels and 3-D points. Correspondences are established between 3-D lines and their images. There is no need to identify individual pixels. The use of correspondences between pixels (that belong to the images of the 3-D lines) and 3-D lines facilitates the correspondence problem when compared to the use of world and image points. This is one of the contributions of this paper. The approach is both evaluated and validated using synthetic data and also real images.

A flexible calibration method using the planar target with a square pattern for line structured light vision system.

A flexible calibration approach for line structured light vision system is proposed in this paper. Firstly a camera model is established by transforming the points from the 2D image plane to the world coordinate frame, and the intrinsic parameters of camera can be obtained accurately. Then a novel calibration method for structured light projector is presented by moving a planar target with a square pattern randomly, and the method mainly involves three steps: first, a simple linear model is proposed, by which the plane equation of the target at any orientations can be determined based on the square’s geometry information; second, the pixel coordinates of the light stripe center on the target images are extracted as the control points; finally, the points are projected into the camera coordinate frame with the help of the intrinsic parameters and the plane equations of the target, and the structured light plane can be determined by fitting these three-dimensional points. The experimental data show that the method has good repeatability and accuracy.