Initial Motion Estimation Research Articles

Indoor Robot SLAM with Multi-Sensor Fusion

Abstract In order to solve the problem of large positioning error and incomplete mapping of SLAM based on two-dimensional lidar in indoor environment, a multi-sensor fusion SLAM algorithm for indoor robots was proposed. Aiming at the mismatch problem of the traditional ICP algorithm in the front end of the lidar SLAM, the algorithm adopts the PL-ICP algorithm that is more suitable for the indoor environment, and uses the extended Kalman filter to fuse the wheel odometer and IMU to provide the initial motion estimation value. Then, during the mapping phase, the pseudo 2D laser data converted from the 3D point cloud data obtained by the depth camera is fused with the data obtained from the 2D lidar to compensate for the lack of vertical field of view in the 2D lidar mapping. The final experimental results show that the fusion odometer data has improved the positioning accuracy by at least 33% compared to a single wheeled odometer, providing a higher initial iteration value for the PL-ICP algorithm. At the same time, fusion mapping compensates for the shortcomings of a single two-dimensional lidar mapping, and constructs an environmental map with more complete environmental information.

Minimal Solvers for Relative Pose Estimation of Multi-Camera Systems using Affine Correspondences

We propose three novel solvers for estimating the relative pose of a multi-camera system from affine correspondences (ACs). A new constraint is derived interpreting the relationship of ACs and the generalized camera model. Using the constraint, we demonstrate efficient solvers for two types of motions. Considering that the cameras undergo planar motion, we propose a minimal solution using a single AC and a solver with two ACs to overcome the degenerate case. Also, we propose a minimal solution using two ACs (a minimal number of one AC and one point correspondence) with known vertical direction, e.g., from an IMU. Since the proposed methods require significantly fewer correspondences than state-of-the-art algorithms, they can be efficiently used within RANSAC for outlier removal and initial motion estimation. The solvers are tested both on synthetic data and on three real-world scenes. It is shown that the accuracy of the estimated poses is superior to the state-of-the-art techniques. Source code is released at https://github.com/jizhaox/relative_pose_gcam_affine.

Relative Pose Estimation With a Single Affine Correspondence.

In this article, we present four cases of minimal solutions for two-view relative pose estimation by exploiting the affine transformation between feature points, and we demonstrate efficient solvers for these cases. It is shown that under the planar motion assumption or with knowledge of a vertical direction, a single affine correspondence is sufficient to recover the relative camera pose. The four cases considered are two-view planar relative motion for calibrated cameras as a closed-form and least-squares solutions, a closed-form solution for unknown focal length, and the case of a known vertical direction. These algorithms can be used efficiently for outlier detection within a RANSAC loop and for initial motion estimation. All the methods are evaluated on both synthetic data and real-world datasets. The experimental results demonstrate that our methods outperform comparable state-of-the-art methods in accuracy with the benefit of a reduced number of needed RANSAC iterations. The source code is released at https://github.com/jizhaox/relative_pose_from_affine.

Scene Flow Estimation From Sparse Light Fields Using a Local 4D Affine Model

In this paper, we address the problem of scene flow estimation from sparsely sampled video light fields. We first propose a local 4D affine model to represent scene flows, taking into account light field epipolar geometry. The model parameters are estimated per cluster in the 4D ray space. They are derived by fitting the model on initial motion and disparity estimates obtained by using 2D dense optical flow estimation techniques. We demonstrate that the model is very effective for estimating scene flows from 2D optical flows. The model regularizes the optical flows and disparity maps, and interpolates disparity variation values in occluded regions. The proposed model allows us to benefit from deep learning-based 2D optical flow estimation methods while ensuring scene flow geometry consistency in the 4 dimensions of the light field.

IMU-Assisted 2D SLAM Method for Low-Texture and Dynamic Environments

Generally, the key issues of 2D LiDAR-based simultaneous localization and mapping (SLAM) for indoor application include data association (DA) and closed-loop detection. Particularly, a low-texture environment, which refers to no obvious changes between two consecutive scanning outputs, with moving objects existing in the environment will bring great challenges on DA and the closed-loop detection, and the accuracy and consistency of SLAM may be badly affected. There is not much literature that addresses this issue. In this paper, a mapping strategy is firstly exploited to improve the performance of the 2D SLAM in dynamic environments. Secondly, a fusion method which combines the IMU sensor with a 2D LiDAR, based on framework of extended Kalman Filter (EKF), is proposed to enhance the performance under low-texture environments. In the front-end of the proposed SLAM method, initial motion estimation is obtained from the output of EKF, and it can be taken as the initial pose for the scan matching problem. Then the scan matching problem can be optimized by the Levenberg–Marquardt (LM) algorithm. For the back-end optimization, a sparse pose adjustment (SPA) method is employed. To improve the accuracy, the grid map is updated with the bicubic interpolation method for derivative computing. With the improvements both in the DA process and the back-end optimization stage, the accuracy and consistency of SLAM results in low-texture environments is enhanced. Qualitative and quantitative experiments with open-loop and closed-loop cases have been conducted and the results are analyzed, confirming that the proposed method is effective in low-texture and dynamic indoor environments.

2D–3D synchronous/asynchronous camera fusion for visual odometry

We propose a robust and direct 2D–3D registration method for camera synchronization. Once the cameras are synchronized—or for synchronous setups—we also propose a visual odometry framework that benefits from both 2D and 3D acquisitions. Our method does not require a precise set of 2D-to-3D correspondences, handles occlusions and works when the scene is only partially known. It is carried out through a 2D–3D based initial motion estimation followed by a constrained nonlinear optimization for motion refinement. The problems of occlusion and that of missing scene parts are handled by comparing the image-based reconstruction and 3D sensor measurements. The results of our experiments demonstrate that the proposed framework allows to obtain a good initial motion estimate and a significant improvement through refinement.

Comments on “Fast Motion Estimation Based on Content Property for Low-Complexity H.265/HEVC Encoder”

In this paper, we provide comments on the recent paper by Pan et al. that proposed an initial search point-based motion estimation skipping (ISP-MES) method. We found some discrepancies of the proposed method and its experimental results, especially in setting the ISPs. In this paper, we clarify these issues and, as a result, provide an enhanced results from the work of Pan et al.

Velocity estimation of glaciers with physically-based spatial regularization — Experiments using satellite SAR intensity images

Abstract Glaciers are an important climate indicator due to their sensitive dependence upon local and regional climate variables, which makes them worthwhile research subjects. A comprehensive description of the glaciers' interaction with the environment and their dynamical behavior requires complex physical models and the measurement of relevant parameters. In-situ data acquisitions are costly and often spatially sparse due to the large extent of glaciers; however, satellite-based sensors offer timely data with complete ground coverage, making them a good choice for continuous monitoring of glaciers. Synthetic aperture radar (SAR) allows a nearly weather-independent monitoring of glacier motion, which is beneficial for often cloudy regions like Alaska or Patagonia. This paper presents a new workflow for the automatic extraction of glacier surfaces from SAR intensity images and the determination of their velocities involving a fluid mechanics model. An initial motion estimation is obtained from intensity tracking on SAR image pairs and subsequently corrected by a physics-based spatial regularization. The surface velocity is approximated by the two-dimensional Navier–Stokes equation for incompressible fluids. The regularization is formulated as a data assimilation problem in which the final solution is a proper solution of the Navier–Stokes equation and simultaneously fitted to the observed velocity. This partial differential equation (PDE) constrained optimization is solved with adjoint models using finite element methods. The proposed method is evaluated on the Taku Glacier, AK, an outlet glacier of the Juneau Icefield. Our presented approach is independent from the type of sensor as long as initial velocity estimates can be obtained. The final results can be used as input to methods estimating ice volume and thickness.

Neural Mechanisms of Cortical Motion Computation Based on a Neuromorphic Sensory System.

The visual cortex analyzes motion information along hierarchically arranged visual areas that interact through bidirectional interconnections. This work suggests a bio-inspired visual model focusing on the interactions of the cortical areas in which a new mechanism of feedforward and feedback processing are introduced. The model uses a neuromorphic vision sensor (silicon retina) that simulates the spike-generation functionality of the biological retina. Our model takes into account two main model visual areas, namely V1 and MT, with different feature selectivities. The initial motion is estimated in model area V1 using spatiotemporal filters to locally detect the direction of motion. Here, we adapt the filtering scheme originally suggested by Adelson and Bergen to make it consistent with the spike representation of the DVS. The responses of area V1 are weighted and pooled by area MT cells which are selective to different velocities, i.e. direction and speed. Such feature selectivity is here derived from compositions of activities in the spatio-temporal domain and integrating over larger space-time regions (receptive fields). In order to account for the bidirectional coupling of cortical areas we match properties of the feature selectivity in both areas for feedback processing. For such linkage we integrate the responses over different speeds along a particular preferred direction. Normalization of activities is carried out over the spatial as well as the feature domains to balance the activities of individual neurons in model areas V1 and MT. Our model was tested using different stimuli that moved in different directions. The results reveal that the error margin between the estimated motion and synthetic ground truth is decreased in area MT comparing with the initial estimation of area V1. In addition, the modulated V1 cell activations shows an enhancement of the initial motion estimation that is steered by feedback signals from MT cells.

The advanced algorithm for tracking objects (AALTO)

ABSTRACTThe advanced algorithm for the tracking of objects (AALTO) constructs tracks from objects, such as thunderstorms or mesocyclones, detected by multiple weather radars at irregular time intervals. It is important to have high accuracy in tracking thunderstorms to generate skilful forecasts and high‐quality climatologies and, fundamentally, to ensure that any derived product from tracks captures only that particular storm, and in its entirety. AALTO incorporates many of the best practices of existing tracking algorithms and techniques employed by meteorologists in constructing tracks. AALTO differs from existing algorithms designed to track meteorological phenomena that manifest in radar data in the following ways: (1) AALTO is designed to track objects from multiple radars, enabling analysis over a larger domain than if a single radar was used; (2) improved tracking is realized through improved initial motion estimates and directional thresholding and (3) AALTO looks both at the track history and at the subsequent possible positions along a track when constructing the best possible tracks, mimicking the approach that would be taken by a human meteorologist. Verification was done using metrics that were objectively determined to distinguish between good and degraded tracks; a description of the approach to determine the appropriate metrics is presented. An overview of the AALTO tracking procedure and an example case are presented in this study.

Fast and accurate surface alignment through an isometry-enforcing game

Surface registration is often performed as a two step process. A feature matching scheme is first adopted to find a coarse initial alignment between two meshes. Subsequently, a refinement step, which usually operates in the space of rigid motions, is applied to reach an optimal registration with respect to pointwise distances between overlapping areas. In this paper we propose a novel technique that allows to obtain an accurate surface registration in a single step, without the need for an initial motion estimation. The main idea of our approach is to cast the selection of correspondences between points on the surfaces in a game-theoretic framework, where a natural selection process allows matching points that satisfy a mutual rigidity constraint to thrive, eliminating all the other correspondences. This process yields a very robust inlier selection scheme that does not depend on any particular technique for selecting the initial strategies as it relies only on the global geometric compatibility between correspondences. The practical effectiveness of the approach is confirmed by an extensive set of experiments and comparisons with state-of-the-art techniques.

Inertial sensor-based smoother for gait analysis.

An off-line smoother algorithm is proposed to estimate foot motion using an inertial sensor unit (three-axis gyroscopes and accelerometers) attached to a shoe. The smoother gives more accurate foot motion estimation than filter-based algorithms by using all of the sensor data instead of using the current sensor data. The algorithm consists of two parts. In the first part, a Kalman filter is used to obtain initial foot motion estimation. In the second part, the error in the initial estimation is compensated using a smoother, where the problem is formulated in the quadratic optimization problem. An efficient solution of the quadratic optimization problem is given using the sparse structure. Through experiments, it is shown that the proposed algorithm can estimate foot motion more accurately than a filter-based algorithm with reasonable computation time. In particular, there is significant improvement in the foot motion estimation when the foot is moving off the floor: the z-axis position error squared sum (total time: 3.47 s) when the foot is in the air is 0.0807 m2 (Kalman filter) and 0.0020 m2 (the proposed smoother).

Real-Time and Robust Stereo Visual Navigation Localization Algorithm Based on ORB

The biggest challenge of visual navigation localization is feature extraction and association. Currently, the most widely used method is simple corner feature and simple matching strategy based on SAD or NCC. Another option is scale invariant feature and rotation invariant descriptor, typically as SIFT, SURF. Feature extraction and matching methods based on the SIFT or SURF are accurate and robust. However, its computational complexity is too high and not suitable for the real-time navigation localization task. This paper presents a new fast, accurate, robust stereo vision navigation localization method, based on a new developed ORB feature and descriptor. First, we presented our matching method based on ORB. Then, we obtained matching inliers and an initial motion estimation parameters using RANSAC and three points motion estimation method. Finally, nonlinear motion refinement method was used to polish the solution. Experimental results show that our method is robust, accurate and real-time.

An Improved Object-Based Video Segmentation Method

This paper presents an improved video segmentation scheme, which consists of two stages: initial segmentation and motion estimation. In the initial segmentation, the watershed transformation followed by a region adjacency graph guided region merging process is used to partition the first video frame into spatial homogenous regions. Then the motion of changed region is estimated. Based on the highly efficient quadratic motion model, the motion estimation is undertaken using Gauss-Newton Levenberg-Marquardt method to minimize the least-square error function. Experimental results show the proposed scheme provides high performance in terms of segmentation accuracy and video compression ratio.

Closing the loop in appearance-guided omnidirectional visual odometry by using vocabulary trees

In this paper, we present a method that allows us to recover the trajectory of a vehicle purely from monocular omnidirectional images very accurately. The method uses a combination of appearance-guided structure from motion and loop closing. The appearance-guided monocular structure-from-motion scheme is used for initial motion estimation. Appearance information is used to correct the rotation estimates computed from feature points only. A place recognition scheme is employed for loop detection, which works with a visual word based approach. Loop closing is done by bundle adjustment minimizing the reprojection error of feature matches. The proposed method is successfully demonstrated on videos from an automotive platform. The experiments show that the use of appearance information leads to superior motion estimates compared to a purely feature based approach. And we demonstrate a working loop closing method which eliminates the residual drift errors of the motion estimation.

Stereo vision based SLAM using Rao-Blackwellised particle filter

We present an algorithm which can realize 3D stereo vision simultaneous localization and mapping (SLAM) for mobile robot in unknown outdoor environments, which means the 6-DOF motion and a sparse but persistent map of natural landmarks be constructed online only with a stereo camera. In mobile robotics research, we extend FastSLAM 2.0 like stereo vision SLAM with "pure vision" domain to outdoor environments. Unlike popular stochastic motion model used in conventional monocular vision SLAM, we utilize the ideas of structure from motion (SFM) for initial motion estimation, which is more suitable for the robot moving in-large-scale outdoor, and textured environments. SIFT features are used as natural landmarks, and its 3D positions are constructed directly through triangulation. Considering the computational complexity and memory consumption, Bkd-tree and Best-Bin-First (BBF) search strategy are utilized for SIFT feature descriptor matching. Results show high accuracy of our algorithm, even in the circumstance of large translation and large rotation movements.

Block motion estimation using adaptive modified two-bit transform

An adaptive modification to the two-bit transform-based low-complexity block motion estimation approach is proposed. Initially, video frames are converted into two-bit representations using the two-bit transform (2BT), and binary block-based motion estimation is performed using these two bit-planes. Modification to the original 2BT-based motion estimation scheme is introduced by conditional local or full searches using the mean absolute difference criterion to improve the initial motion estimation accuracy. The condition threshold is adaptively varied according to block detail using the block variance already computed for the 2BT process. Experimental results show that the proposed adaptive modified 2BT-based motion estimation technique can significantly improve peak signal-to-noise ratio performance compared with 2BT only motion estimation, and outperforms modified one-bit transform (1BT)-based motion estimation as well as modified 2BT-based motion estimation without adaptation at the same motion estimation cost. The main contribution of this work is to provide an adaptive decision on whether a modified search should be carried out after the low-complexity motion estimation process.

Computing depth maps from descent images

In the exploration of the planets of our solar system, images taken during a lander's descent to the surface of a planet provide a critical link between orbital images and surface images. The descent images not only allow us to locate the landing site in a global coordinate frame, but they also provide progressively higher-resolution maps for mission planning. This paper addresses the generation of depth maps from the descent images. Our approach has two steps, motion refinement and depth recovery. During motion refinement, we use an initial motion estimate in order to avoid the intrinsic motion ambiguity. The objective of the motion-refinement step is to adjust the motion parameters such that the reprojection error is minimized. The depth-recovery step correlates adjacent frames to match pixels for triangulation. Due to the descending motion, the conventional rectification process is replaced by a set of anti-aliasing image warpings corresponding to a set of virtual parallel planes. We demonstrate experimental results on synthetic and real descent images.

Hybrid algorithm for estimating true motion fields

A hybrid algorithm is proposed for true motion field estimation. This algorithm consists of three steps: block-based initial motion estimation, image segmentation, and wrong-motion-vector detection and correction based on objects. The hierarchical block-matching algorithms are improved for the initial motion estimation. The improved algorithm uses an adaptive technique to propagate motion vectors between hierarchical levels. It produces accurate motion field everywhere, except in the areas of motion occlusion. In order to correct wrong motion vectors in the areas of motion occlusion, the current image is segmented into objects and an object-based method is proposed to process the estimated motion fields. With the object-based method, wrong-motion vectors are detected by approximating the estimated motion field in each object with a motion model, and are corrected using an object-adaptive interpolator. Experimental results show that the improved hierarchical block-matching algorithm outperforms the conventional hierarchical block-matching algorithms. The proposed algorithm results in dense motion fields that are smooth within every object, discontinuous between objects of different motion, and very close to the true motion fields.

Structure and Motion Estimation from Dynamic Silhouettes under Perspective Projection

We address the problem of estimating the structure and motion of a smooth curved object from its silhouettes observed over time by a trinocular stereo rig under perspective projection. We first construct a model for the local structure along the silhouette for each frame in the temporal sequence. The local models are then integrated into a global surface description by estimating the motion between successive time instants. The algorithm tracks certain surface features (parabolic points) and image features (silhouette inflections and frontier points) which are used to bootstrap the motion estimation process. The entire silhouettes along with the reconstructed local structure are then used to refine the initial motion estimate. We have implemented the proposed approach and report results on real images.