Vision-Based Kalman Filtering for Aircraft State Estimation and Structure from Motion

Abstract

In this paper, we provide a critical, albeit brief, overview of feature point tracking, SFM algorithms and Kalman Filtering as they apply specifically to the state estimation problem for aircraft. We naturally focus on those attributesof the methods that are well-suitedto aircraft estimation problems. We also note features of the algorithms that require further work to develop a stable, consistent vision-based state estimation algorithm for aircraft. We specifically discuss the direct, and quite natural, extension of the work of 9 and 1 that incorporates aircraft dynamical system models in the state propagation step of the filter. Withthis framework, we effectively augment the baseline vision-based motion estimation with the benefits of a dynamical model. He structure from motion (SFM) problem, in which the objective is to estimate camera motion and the threedimensional structure of the scene using features extracted from two-dimensional images, has been the subject of an enormous body of literature in the computer vision community. It has been widely recognized for some time that the ability to accurately and robustly solve the structure from motion problem has important implications for the guidance, navigation, and control of autonomous vehicles. One such application is the autonomous control of unmanned and micro air vehicles in complex urban environments using vision as the primary sensor. Clearly, accurate estimation of aircraft states is valuable from a controls standpoint while knowledge of the scene is critical for obstacle avoidance. It should be remembered that much of the research in SFM algorithms has been Carried out for slow moving ground vehicles or robotic systems. In some Instances, SFM algorithms have been derived for post-processing large collections of video data that can be processed simultaneously, as opposed to causally or incrementally in time. This point is discussed more completely shortly below. The implications of these observations is, however, that the SFM problemcan be especially challengingfor aircraft applicationssince the images taken fromaircraft can be subject to noise, occlusions, frame dropout due to telemetry problems, and loss of feature tracking resulting from fast vehicle dynamics. Therefore, any SFM algorithm that is to be applied to aircraft, and particularly micro air vehicles, must be robust with respect to noise in the two-dimensional image plane. Structure from motion is by nature a very general problem and many specific different problem statements can be defined based on the nature of the two-dimensional image data used and the form of the three-dimensional structure of the scene. For example, SFM algorithms have been employed that make use of tracked feature points, lines, and line segments. In addition, homography-basedapproaches can be used in cases where there are known planar features in the images. Likewise, the form of the three-dimensional scene that is computed from SFM can vary in complexity from the locations of discrete feature points in inertial space to complete geometrical descriptions of the environment. Another important distinction in SFM techniques is the choice between batch implementations that make use of all the images obtained from a video sequence and causal methods that only use information from images up to the present time. This choice is usually application dependent since, as noted in Ref. 1, given the luxury of being able to compute SFM estimates in an off-line batch procedure, one can almost certainly obtain better results than from a causal, online estimation process. It is clear that the application of SFM to the real-time guidance, navigation, and control of