We discuss the construction of relational observables in time-reparametrization invariant quantum mechanics and we argue that their physical interpretation can be understood in terms of conditional probabilities, which are defined from the solutions of the quantum constraint equation in a generalization of the Page-Wootters formalism. In this regard, we show how conditional expectation values of worldline tensor fields are related to quantum averages of suitably defined relational observables. We also comment on how the dynamics of these observables can be related to a notion of quantum reference frames. After presenting the general formalism, we analyze a recollapsing cosmological model, for which we construct unitarily evolving quantum relational observables. We conclude with some remarks about the relevance of these results for the construction and interpretation of diffeomorphism-invariant operators in quantum gravity.