We analyze a variational time discretization of geodesic calculus on finite- and certain classes of infinite-dimensional Riemannian manifolds. We investigate the fundamental properties of discrete geodesics, the associated discrete logarithm, discrete exponential maps, and discrete parallel transport, and we prove convergence to their continuous counterparts. The presented analysis is based on the direct methods in the calculus of variation, on $\Gamma$-convergence, and on weighted finite element error estimation. The convergence results of the discrete geodesic calculus are experimentally confirmed for a basic model on a two-dimensional Riemannian manifold. This provides a theoretical basis for applications in computer vision: In particular, discrete geodesics offer an effective tool for shape morphing and the computation of a distance between shapes, the discrete logarithm allows a linear representation of strongly nonlinear shape variability, the discrete exponential map provides a robust tool for shape extrapolation, and the discrete parallel transport can be used to transfer geometric details from one shape to another. The basic operations are exemplarily illustrated for two different shape spaces, the space of viscous volumetric objects and the space of discrete viscous shells.