Sensor Fusion Problem Research Articles

Modeling and Calibration of Inertial and Vision Sensors

This paper is concerned with the problem of estimating the relative translation and orientation of an inertial measurement unit and a camera, which are rigidly connected. The key is to realize that this problem is in fact an instance of a standard problem within the area of system identification, referred to as a gray-box problem. We propose a new algorithm for estimating the relative translation and orientation, which does not require any additional hardware, except a piece of paper with a checkerboard pattern on it. The method is based on a physical model which can also be used in solving, for example, sensor fusion problems. The experimental results show that the method works well in practice, both for perspective and spherical cameras.

Particle Filtering for State Estimation in Nonlinear Industrial Systems

State estimation is a major problem in industrial systems, particularly in industrial robotics. To this end, Gaussian and nonparametric filters have been developed. In this paper, the extended Kalman filter, which assumes Gaussian measurement noise, is compared with the particle filter, which does not make any assumption on the measurement noise distribution. As a case study, the estimation of the state vector of an industrial robot is used when measurements are available from an accelerometer that was mounted on the end effector of the robotic manipulator and from the encoders of the joints' motors. It is shown that, in this kind of sensor fusion problem, the particle filter outperforms the extended Kalman filter, at the cost of more demanding computations.

Advanced tracking through efficient image processing and visual–inertial sensor fusion

This article presents a new visual–inertial tracking device for augmented and virtual reality applications and addresses two fundamental issues of such systems. The first one concerns the definition and modelling of the sensor fusion problem. Much work has been conducted in this area and several models for exploiting gyroscopes and linear accelerometers have been proposed. However, the respective advantages of each model and in particular the benefits of the integration of the accelerometer data in the filter are still unclear. A comparison of different models with special investigation of the effects of using accelerometers on the tracking performance is therefore provided. The second contribution is the development of an image processing approach that does not require special landmarks but uses natural features. The solution relies on a 3D model of the scene that is used to predict the appearances of the features by rendering the model based on data from the sensor fusion algorithm. The feature localisation is robust and accurate mainly because local lighting is also estimated. The final system is evaluated with help of ground-truth and real data. High stability and accuracy are demonstrated also for large environments.

Particle filtering for state estimation in industrial robotic systems

State estimation is a major problem in industrial systems. To this end, Gaussian and non-parametric filters have been developed. In this paper the extended Kalman filter which assumes Gaussian measurement noise is compared to the particle filter which does not make any assumption on the measurement noise distribution. As a case study the estimation of the state vector of an industrial robot is used, when measurements are available from an accelerometer mounted on the end-effector of the robotic manipulator and from the encoders of the joint motors. It is shown that in this kind of sensor fusion problem the particle filter outperforms the extended Kalman filter, at the cost of more demanding computations.

Decentralized rate control for tracking and surveillance networks

We present a duality-based method for decentralized communication resource management in the context of networked tracking and surveillance systems. This method shares communication resources according to the joint utility provided by information of particular types. By formulating the problem as an optimization of the joint utility of information flow rates, the dual of the problem can be understood to provide prices for particular routes. Decentralized rate control is accomplished using primal–dual iterations in combination with communication of these route prices and marginal utility values. We build on previous work on the subject in a few important ways. First, we consider utility functions that are jointly dependent on flow rates, to properly account for geometric synergy that can occur in sensor fusion problems. Second, we do not require that the rate-update algorithms have explicit knowledge of utility functions. Instead, our update algorithms only require transmission of marginal utility values. We provide a proof of the convergence of the algorithm, and present simulation results to demonstrate its effectiveness.

Évolution artificielle, optimisation et analyse d'images

Parametric models inherited from image synthesis are giving an opportunity to using artificial evolution as a parameter optimisation paradigm in image analysis applications. In this paper, we show how artificial evolution can actually widen the scope of the generalised Hough transform and how some newer evolutionary approaches can efficiently solve real-time computer vision, sensor fusion and robotics problems with little reference to more traditional methods.

A control-theoretic approach to distributed discrete-valued decision-making in networks of sensing agents

We address the problem of global sensor fusion for the purpose of distributed decision-making, from a control-theoretic perspective. In particular, we introduce a quasi-linear stochastic distributed protocol, using which a network of sensing agents can reach agreement in order to take a collective action. Using control-theoretic methods, we design the parameters of our protocol - which include weights in the local update rules used by the agents and a finite stopping time - to achieve agreement in a fair and rapid manner. We show analytically that the developed protocol achieves fair agreement with certainty in the noise-free case and achieves fair agreement with high probability even in the presence of communication noise and assuming very little information storage capability for the agents. Our development is illustrated throughout with a canonical example motivated by autonomous vehicle control

A nonmonotonic observation logic

A variant of Reiter's default logic is proposed as a logic for reasoning with (defeasible) observations. Traditionally, default rules are assumed to represent generic information and the facts are assumed to represent specific information about the situation, but in this paper, the specific information derives from defeasible observations represented by (normal free) default rules, and the facts represent (hard) background knowledge. Whenever the evidence underlying some observation is more refined than the evidence underlying another observation, this is modelled by means of a priority between the default rules representing the observations. We thus arrive at an interpretation of prioritized normal free default logic as an observation logic, and we propose a semantics for this observation logic. Finally, we discuss how the proposed observation logic relates to the multiple extension problem and the problem of sensor fusion.

Simultaneous Registration and Fusion of Multiple Dissimilar Sensors for Cooperative Driving

The fusion of multiple sensory information plays a key role in cooperative driving for flexible platooning of automated vehicles over a couple of lanes within a short intervehicle distance. In this paper, the problem of online sensor fusion with spatially and temporally misaligned dissimilar sensors is considered. A spatial-temporal registration model for the popular intelligent vehicular sensors including radar, global positioning system, inertial navigation system, and camera is first developed for sensor alignment. An unscented Kalman filter (UKF) is proposed here to fuse and register these sensors that are installed on a platoon of vehicles simultaneously. When the road geometry information is available from a digital map database, a constrained UKF is further developed to improve the fusion accuracy. The effect of spatial-temporal sensor misalignment upon the vehicle-state estimation is also analyzed theoretically. Simulations show that the proposed UKF method not only can align the dissimilar vehicular sensors properly with both spatial and temporal biases, but can also obtain accurate fused tracks of vehicles in a platoon.

Robust Visual Tracking by Integrating Multiple Cues Based on Co-Inference Learning

Visual tracking can be treated as a parameter estimation problem that infers target states based on image observations from video sequences. A richer target representation may incur better chances of successful tracking in cluttered and dynamic environments, and thus enhance the robustness. Richer representations can be constructed by either specifying a detailed model of a single cue or combining a set of rough models of multiple cues. Both approaches increase the dimensionality of the state space, which results in a dramatic increase of computation. To investigate the integration of rough models from multiple cues and to explore computationally efficient algorithms, this paper formulates the problem of multiple cue integration and tracking in a probabilistic framework based on a factorized graphical model. Structured variational analysis of such a graphical model factorizes different modalities and suggests a co-inference process among these modalities. Based on the importance sampling technique, a sequential Monte Carlo algorithm is proposed to provide an efficient simulation and approximation of the co-inferencing of multiple cues. This algorithm runs in real-time at around 30 Hz. Our extensive experiments show that the proposed algorithm performs robustly in a large variety of tracking scenarios. The approach presented in this paper has the potential to solve other problems including sensor fusion problems.

An outdoor navigation system using GPS and inertial platform

The use of global positioning system (GPS) in outdoor localization is quite a common solution in large environments where no other reference is available and there are not so demanding positioning requirements. Of course, fine motion without the use of an expensive differential device is not an easy task, even now that available precision has been greatly improved as the military encoding has been removed. In this paper we present a localization algorithm based on Kalman filtering that tries to fuse information coming from an inexpensive single GPS with inertial data and map-based data. The algorithm is able to produce an estimated configuration for the robot that can be successfully fed back in a navigation system. Some experiments show difficulties and possible solutions of this sensor fusion problem.

Vector space methods for sensor fusion problems

In a multiple sensor system, the sensor Sj , j=1,2, . . . ,N, outputs Y( j )?R in response to input X?[0,1], according to an unknown probability distribution PY(j )|X . The problem is to estimate a fusion function f:RN?[0,1], based on a training sample, such that the expected square error is minimized over a family of functions F that constitutes a finite-dimensional vector space. The function f* that exactly minimizes the expected error cannot be computed since the underlying distributions are unknown, and only an approximation f to f* is feasible. We estimate the sample size sufficiently to ensure that an estimator f that minimizes the empirical square error provides a close approximation to f* with a high probability. The advantages of vector space methods are twofold: (1) the sample size estimate is a simple function of the dimensionality of F and (2) the estimate f can be easily computed by the well-known least square methods in polynomial time. The results are applicable to the classical potential function method as well as to a recently proposed class of sigmoidal feedforward neural networks.

Object location based on sensor fusion of visual and tactual sensing

The paper describes a solution of object orientation findings based upon the sensor fusion problems between visual sensing and tactual sensing. We focus on the induction of the tactual images used for identifying the orientation of an object in a sensor fusion system made up on the geodesic dome. As an object is set up at the center of the geodesic dome, its image is taken by the camera positioned at the center of the facet on the geodesic dome. Also, robot fingers take tactual images on the finger behavior. If an object's image taken by the camera is identified with the image model of the facet on the geodesic dome, the position and orientation of the camera which takes its image are determined from the facet on the geodesic dome. As the tactual image corresponds to the image induced from the facet on the geodesic dome, the orientation of object is confirmed from the tactual image. Computer simulations show the effectiveness of the proposed method.

Real-time distributed sensor fusion for time-critical sensor readings

The problem of dynamic distributed sensor fusion, where time critical sensor readings are fused concurrently by physically isolated modules, is considered. A large number of critical applications including target tracking, remote sensing, and autonomous vehicle navigation can depend on interpreting sensor readings in real time. Independent sensor modules receive partially contradictory data, but their answers must agree within computed accuracy bounds. Real-time constraints cause additional uncertainty factors. The dynamic nature of this problem requires that the problem be viewed in four dimensions and processed efficiently. We present a new method that robustly tolerates a limited number of component failures. This method has complexity O(f 4N log N), where N is the total number of modules and f is the number of faulty modules tolerated. Experimental results are given showing the merits of the proposed method.

A multiresolution methodology for signal-level fusion and data assimilation with applications to remote sensing

This paper covers the design of multiscale stochastic models that can be used to fuse measurements of a random field or random process provided at multiple resolutions. Such sensor fusion problems arise in a variety of contexts, including many problems in remote sensing and geophysics. An example, which is used in this paper as a vehicle to illustrate our methodology, is the estimation of variations in hydraulic conductivity as required for the characterization of groundwater flow. Such a problem is typical in that the phenomenon to be estimated cannot be measured at fine scales throughout the region of interest, but instead must be inferred from a combination of measurements of very different types, including point measurements of hydraulic conductivity at irregular collections of points and indirect measurements that provide only coarse and nonlocal information about the conductivity field. Fusion of such disparate and irregular measurement sets is a challenging problem, especially when one includes the objective of producing, in addition to estimates, statistics characterizing the errors in those estimates. In this paper, we show how modeling a random field at multiple resolutions allows for the natural fusion (or assimilation) of measurements that provide information of different types and at different resolutions. The key to our approach is to take advantage of the fast multiscale estimation algorithms that efficiently produce both estimates and error variances even for very large problems. The major innovation required in our case, however, is to extend the modeling of random fields within this framework to accommodate multiresolution measurements. In particular to take advantage of the fast algorithms that the models admit, we must be able to model each nonlocal measurement as the measurement of a single variable of the multiresolution model at some appropriate resolution and scale. We describe how this can be done and illustrate its effectiveness for an ill-posed inverse problem in groundwater hydrology.

Lebesgue Logic for Probabilistic Reasoning and Some Applications to Perception

Reasoning with probabilities is essential to many sciences, such as decision theory, expert systems, neural networks, pattern recognition, and perception in general. In this paper we explore a new logic of probabilities, the Lebesgue logic, in which are defined the logical relations ENTAILS, AND, OR, and NOT on collections of probability measures. In particular, given any two probability measures μ and v, the Lebesgue logic answers questions such as the following: Does μ entail v? What is the conjunction of μ and v, i.e., what is μ AND v? What is the disjunction of μ and v, i.e., what is μ OR v? What is the negation of μ? Several properties of the Lebesgue logic emerge. Among them are (1) the Lebesgue logic is not boolean, in general, but is "locally boolean," (2) the AND of the Lebesgue logic is a generalization of Bayes' rule; (3) one can define probability measures on the Lebesgue logic itself, thereby permitting the representation of probabilistic knowledge without requiring any commitment to a particular probability measure; and (4) many probabilistic inferences can be described as morphisms of the Lebesgue logic, i.e., as maps from one collection of probability measures to another, respecting the Lebesgue logics on both. We close by discussing a concrete problem to which the Lebesgue logic may find application; the problem of sensor fusion in vision and other perceptual modalities.

An evaluation of machine guarding techniques for robot guarding

An increasing number of problems caused by the use of industrial robots should be expected due to the dramatically increasing population of robots today. In this paper, machine guarding functions and techniques are reviewed. The possibilities of transferring the concepts of machine guarding to robotic systems are then analyzed. The analysis shows that most of the guarding techniques are still applicable to industrial robots with modifications to accommodate differences. A set of guidelines for installing safeguarding devices is also presented. Research is need in the areas of guarding techniques for robotic cells, new sensor development, and sensor reliability, fusion, and coordination problems.

Tackling the problem of sensor fusion

Sensor Review's regular analysis of the international sensor scene unearths some major challenges. Stephen McClelland reports.

Sensor data fusion

This paper reviews some knowledge representation approaches devoted to the sensor fusion problem, as encountered whenever images, signals, text must be combined to provide the input to a controller or to an inference procedure. The basic steps involved in the derivation of the knowledge representation scheme, are: (A) locate a representation, based on exogeneous context information (B) compare two representations to find out if they refer to the same object/entity (C) merge sensor-based features from the various representations of the same object into a new set of features or attributes, (D) aggregate the representations into a joint fused representation, usually more abstract than each of the sensor-related representations.

Knowledge Representation Approaches in Sensor Fusion

Abstract This paper reviews some knowledge representation approaches devoted to the sensor fusion problem, as encountered whenever images, signals, text must be combined to provide the input to a controller or to an inference procedure. The basic steps involved in the derivation of the knowledge representation scheme, are: A. locate a representation based on exogeneous context information B. compare two representations to find out if they refer to the same object/entity C. merging sensor based features from the various representations of the same object into a new set of features or attributes D. aggregating the representations into a joint fused representation, usually more abstract than each of the sensor related representations The importance of sensor fusion stems first from the fact that it is generally correct to assume that improvements in control law simplicity and robustness, as well as better classification results, can be achieved by combining diverse information sources. The second element, is that e.g. spatially distributed sensing, or otherwise diverse sensing, does indeed require fusion as well.