Computation of time-varying motion and structure parameters from real image sequences

Abstract

We address the problem of robust estimation of motion and structure parameters, which describe an observer’s translation, rotation, and environmental layout (i.e. the relative depth of visible 3-d points) from noisy time-varying optical flow. Allowable observer motions include a moving vehicle and a broad class of robot arm motions. We assume the observer is a camera rigidly attached to the moving vehicle or robot arm, which moves along a smooth trajectory in a stationary environment. As the camera moves it acquires images at some reasonable sampling rate (say 30 images per second). Given a sequence of such images we analyze them to recover the camera’s motion and depth information for various surfaces in the environment. As the camera moves, with respect to some 3-d environmental point, the relative 3-d velocity that occurs is mapped (under perspective projection) onto the camera’s image plane as 2-d image motion. Optical flow or image velocity is an infinitesimal approximation to this image motion. Since the camera moves relative to a scene we can compute image velocity fields at each time. Given the observer’s translation, \(\vec U\), and rotation, \(\vec \omega \), and the coordinates of a 3-d point, \(\vec P\), a non-linear equation that relates these parameters to the 2-d image velocity, \(\vec \upsilon \), at image point \(\vec Y\), where \(\vec Y\) is the perspective projection of \(\vec P\), is as follows [10]: $$\vec \upsilon \left( {\vec Y,t} \right) = {\vec \upsilon _T}\left( {\vec Y,t} \right) + {\vec \upsilon _R}\left( {\vec Y,t} \right)$$ (1) where \({\vec \upsilon _T} \) and \({\vec \upsilon _R}\) are the translational and rotational components of image velocity: $${\vec \upsilon _T}\left( {\vec Y,t} \right) = {A_1}\left( {\vec Y} \right)\vec u\left( {\vec Y,t} \right){\left\| {\vec Y} \right\|_2} and {\vec v_R}\left( {\vec Y,t} \right) = {A_2}\left( {\vec Y} \right)\vec \omega \left( t \right)$$ (2) and $${A_1} = \left( {\begin{array}{*{20}{c}}{ - 1}&0&{{y_1}} \\ 0&{ - 1}&{{y_2}} \end{array}} \right) and {A_2}\left( {\begin{array}{*{20}{c}} {{y_1}{y_2}}&{ - \left( {1 + y_1^2} \right)}&{{y_2}} \\ {\left( {1 + y_2^2} \right)}&{ - {y_1}{y_2}}&{{y_1}} \end{array}} \right)$$ (3)