BosonSampling is a restricted model of quantum computation proposed recently, where a non-adaptive linear-optical network is used to solve a sampling problem that seems to be hard for classical computers. Here we show that, even if the linear-optical network has a constant number (greater than four) of beam splitter layers, the exact version of the BosonSampling problem is still classically hard, unless the polynomial hierarchy collapses to its third level. This is based on similar result known for constant-depth quantum circuits and circuits of 2-local commuting gates (IQP).