We introduce a method of solving inverse boundary value problems for wave equations on Lorentzian manifolds, and show that zeroth order coefficients can be recovered under certain curvature bounds. The set of Lorentzian metrics satisfying the curvature bounds has a non-empty interior in the sense of arbitrary, smooth perturbations of the metric, whereas all previous results on this problem impose conditions on the metric that force it to be real analytic with respect to a suitably defined time variable. The analogous problem on Riemannian manifolds is called the Calderón problem, and in this case the known results require the metric to be independent of one of the variables. Our approach is based on a new unique continuation result in the exterior region of double null cones. The approach shares features with the classical Boundary Control method, and can be viewed as a generalization of this method to cases where no real analyticity is assumed.