Are reflectance field renderings appropriate for optical flow evaluation?

Abstract

Ground truth generation for optical flow is hard and costly. Real images can be annotated with ground truth optical flow, but two problems persist: first, no flow measurement techniques for dynamic, large-scale data exist. Second, real-world parameters such as weather or surface wetness cannot be systematically varied. On the other hand, rendered images can have perfect ground truth while light, geometry and materials can be systematically varied. But do they resemble reality well enough to serve as a basis for performance evaluation? In this paper, we take computer graphic realism to an extreme: we compare three optical flow results on reflectance field renderings with supposedly identical real images. A systematic variation of reflectance realism reveals that computer graphics are a valid way to create complex synthetic sequences for optical flow evaluation — but only if modeled carefully.