Static/dynamic filter with nonlocal regularizer

Abstract

Guided (joint) image filters play an important role in many computer vision and image processing applications. The main principle behind these filters is transferring the structural information from a guidance image to an input one. However, in practice, the structures between the two images are not always consistent. As a result, the filtering outputs become sensitive to outliers, which easily leads to texture-copying artifacts. Most recently, by relaxing the dependence on the guidance, static/dynamic (SD) filter overcomes the drawback effectively. With the SD strategy, this filter can jointly leverage structural information from the guidance and input. However, due to the locality of its regularizer, SD is prone to another deficiency, i.e., edge blurring. To tackle this problem, in our work, we extend SD filter to a nonlocal version [nonlocal static/dynamic (NSD)]. Particularly, a nonlocal regularizer is first established in a subspace transformed by partial least squares, which can better respect the unequal roles of the images. Then, to efficiently formulate the structural consistency between the two images (guidance and input), a novel joint term is plugged into the regularizer. Finally, an acceleration approach is designed to reduce the computational complexity induced by the nonlocal extension, which makes NSD achieve a comparable running time in practice. Thorough experimental results demonstrate that the proposed filter not only can avoid texture copy effectively but also can preserve edges powerfully.