Fast-rolling shutter compensation based on piecewise quadratic approximation of a camera trajectory

Abstract

Rolling shutter effect commonly exists in a video camera or a mobile phone equipped with a complementary metal-oxide semiconductor sensor, caused by a row-by-row exposure mechanism. As video resolution in both spatial and temporal domains increases dramatically, removing rolling shutter effect fast and effectively becomes a challenging problem, especially for devices with limited hardware resources. We propose a fast method to compensate rolling shutter effect, which uses a piecewise quadratic function to approximate a camera trajectory. The duration of a quadratic function in each segment is equal to one frame (or half-frame), and each quadratic function is described by an initial velocity and a constant acceleration. The velocity and acceleration of each segment are estimated using only a few global (or semiglobal) motion vectors, which can be simply predicted from fast motion estimation algorithms. Then geometric image distortion at each scanline is inferred from the predicted camera trajectory for compensation. Experimental results on mobile phones with full-HD video demonstrate that our method can not only be implemented in real time, but also achieve satisfactory visual quality.