Learning Factorized Cross-View Fusion for Multi-View Crowd Counting

Abstract

Crowd counting has been a long-standing task in surveillance video analysis. Most of existing methods focus on a single-view setting. Crowd counting with multiple views can provide richer and complementary information across views. However, the task is still inadequately explored in the literature. Previous works have attempted either to project each camera view onto a common geometric 2-D ground-plane and estimate crowd density map through aggregation [1], or set up connections among all pixel pairs [2]. However, registering a local view to the global ground-plane is error-prone and fails to explicitly model the critical inter-view correlation. Full inter-pixel connections inevitably lead to explosion of parameters. To solve these problems, in this paper, we propose an efficient module that effectively does the job of cross-view fusion by directly modeling the correlation between each pair of views. More specifically, to distill and transfer all useful information from multiple sources views to a target camera view, we factorize the full transformation into a generic-fusion component that encodes all geometric / semantic information of this target view, and a view-specific affine-transform component that encodes the scene geometry / semantics cues of specific source view. This factorization significantly reduces the parameter redundancy and enables plug-and-play of new cameras. Extensive experiments on three multi-view counting datasets (PETS2009, DukeMTMC, and CityStreet) clearly and consistently demonstrate the superiority of the proposed method.