Monocular Image Data Research Articles

Measuring In-Building Spatial-Temporal Human Distribution through Monocular Image Data Considering Deep Learning–Based Image Depth Estimation

AbstractThis research estimated the spatial-temporal distribution of humans in buildings through image sensing. Inputs were the in-building network, image sequences recording the movement of human,...

Robust Visual Localization Across Seasons

Localization is an integral part of reliable robot navigation, and long-term autonomy requires robustness against perceptional changes in the environment during localization. In the context of vision-based localization, such changes can be caused by illumination variations, occlusion, structural development, different weather conditions, and seasons. In this paper, we present a novel approach for localizing a robot over longer periods of time using only monocular image data. We propose a novel data association approach for matching streams of incoming images to an image sequence stored in a database. Our method exploits network flows to leverage sequential information to improve the localization performance and to maintain several possible trajectories hypotheses in parallel. To compare images, we consider a semidense image description based on histogram of oriented gradients features as well as global descriptors from deep convolutional neural networks trained on ImageNet for robust localization. We perform extensive evaluations on a variety of datasets and show that our approach outperforms existing state-of-the-art approaches.

Small Imaging Depth LIDAR and DCNN-Based Localization for Automated Guided Vehicle.

We present our third prototype sensor and a localization method for Automated Guided Vehicles (AGVs), for which small imaging LIght Detection and Ranging (LIDAR) and fusion-based localization are fundamentally important. Our small imaging LIDAR, named the Single-Photon Avalanche Diode (SPAD) LIDAR, uses a time-of-flight method and SPAD arrays. A SPAD is a highly sensitive photodetector capable of detecting at the single-photon level, and the SPAD LIDAR has two SPAD arrays on the same chip for detection of laser light and environmental light. Therefore, the SPAD LIDAR simultaneously outputs range image data and monocular image data with the same coordinate system and does not require external calibration among outputs. As AGVs travel both indoors and outdoors with vibration, this calibration-less structure is particularly useful for AGV applications. We also introduce a fusion-based localization method, named SPAD DCNN, which uses the SPAD LIDAR and employs a Deep Convolutional Neural Network (DCNN). SPAD DCNN can fuse the outputs of the SPAD LIDAR: range image data, monocular image data and peak intensity image data. The SPAD DCNN has two outputs: the regression result of the position of the SPAD LIDAR and the classification result of the existence of a target to be approached. Our third prototype sensor and the localization method are evaluated in an indoor environment by assuming various AGV trajectories. The results show that the sensor and localization method improve the localization accuracy.

OpenRatSLAM: an open source brain-based SLAM system

RatSLAM is a navigation system based on the neural processes underlying navigation in the rodent brain, capable of operating with low resolution monocular image data. Seminal experiments using RatSLAM include mapping an entire suburb with a web camera and a long term robot delivery trial. This paper describes OpenRatSLAM, an open-source version of RatSLAM with bindings to the Robot Operating System framework to leverage advantages such as robot and sensor abstraction, networking, data playback, and visualization. OpenRatSLAM comprises connected ROS nodes to represent RatSLAM's pose cells, experience map, and local view cells, as well as a fourth node that provides visual odometry estimates. The nodes are described with reference to the RatSLAM model and salient details of the ROS implementation such as topics, messages, parameters, class diagrams, sequence diagrams, and parameter tuning strategies. The performance of the system is demonstrated on three publicly available open-source datasets.

A Monocular Pointing Pose Estimator for Gestural Instruction of a Mobile Robot

We present an important aspect of our human-robot communication interface which is being developed in the context of our long-term research framework PERSES dealing with highly interactive mobile companion robots. Based on a multi-modal people detection and tracking system, we present a hierarchical neural architecture that estimates a target point at the floor indicated by a pointing pose, thus enabling a user to navigate a mobile robot to a specific target position in his local surroundings by means of pointing. In this context, we were especially interested in determining whether it is possible to accomplish such a target point estimator using only monocular images of low-cost cameras. The estimator has been implemented and experimentally investigated on our mobile robotic assistant HOROS. Although only monocular image data of relatively poor quality were utilized, the estimator accomplishes a good estimation performance, achieving an accuracy better than that of a human viewer on the same data. The achieved recognition results demonstrate that it is in fact possible to realize a user-independent pointing direction estimation using monocular images only, but further efforts are necessary to improve the robustness of this approach for everyday application.

Attitude estimation using moment invariants

The problem of estimating the three-dimensional rotational parameters of a rigid body from its monocular image data has been considered using the method of moment invariants. Second- and third-order moment invariants are used to construct the feature vector for the scale and orientation independent identification of the camera view axis direction in the body-fixed reference frame. The camera rotation angle about the view axis is derived from second-order central moments. The relative attitude of the rigid body is then expressed in terms of quaternion parameters to model the outputs of a video sensor in attitude control simulations. Experimental results and simulation outputs are presented using the mathematical model of a spacecraft.

Estimation of quaternion parameters from two dimensional image moments

The problem of estimating the rotational parameters of a rigid planar patch from its monocular image data is considered using the moments of image functions. The quaternion elements describing the object transformations in 3D space are functionally related to the first and second order image moments by a set of nonlinear quadratic equations. The moment equations can then be reduced to linear expressions in quaternion product terms to yield a closed form solution for the magnitudes of the rotational parameters. The mathematical formulations are presented along with experimental results.

Dynamic 3D models with local and global deformations: deformable superquadrics

The authors present a physically based approach to fitting complex three-dimensional shapes using a novel class of dynamic models that can deform both locally and globally. They formulate the deformable superquadrics which incorporate the global shape parameters of a conventional superellipsoid with the local degrees of freedom of a spline. The model's six global deformational degrees of freedom capture gross shape features from visual data and provide salient part descriptors for efficient indexing into a database of stored models. The local deformation parameters reconstruct the details of complex shapes that the global abstraction misses. The equations of motion which govern the behavior of deformable superquadrics make them responsive to externally applied forces. The authors fit models to visual data by transforming the data into forces and simulating the equations of motion through time to adjust the translational, rotational, and deformational degrees of freedom of the models. Model fitting experiments involving 2D monocular image data and 3D range data are presented.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Symmetry-seeking models and 3D object reconstruction

We propose models of 3D shape which may be viewed as deformable bodies composed of simulated elastic material. In contrast to traditional, purely geometric models of shape, deformable models are active—their shapes change in response to externally applied forces. We develop a deformable model for 3D shape which has a preference for axial symmetry. Symmetry is represented even though the model does not belong to a parametric shape family such as (generalized) cylinders. Rather, a symmetry-seeking property is designed into internal forces that constrain the deformations of the model. We develop a framework for 3D object reconstruction based on symmetry-seeking models. Instances of these models are formed from monocular image data through the action of external forces derived from the data. The forces proposed in this paper deform the model in space so that the shape of its projection into the image plane is consistent with the 2D silhouette of an object of interest. The effectiveness of our approach is demonstrated using natural images.