The dynamical behavior of open quantum systems plays a key role in many applications of quantum mechanics, examples ranging from fundamental problems, such as the environment-induced decay of quantum coherence and relaxation in many-body systems, to applications in condensed matter theory, quantum transport, quantum chemistry and quantum information. In close analogy to a classical Markov process, the interaction of an open quantum system with a noisy environment is often modelled by a dynamical semigroup with a generator in Lindblad form, which describes a memoryless dynamics leading to an irreversible loss of characteristic quantum features. However, in many applications open systems exhibit pronounced memory effects and a revival of genuine quantum properties such as quantum coherence and correlations. Here, recent results on the rich non-Markovian quantum dynamics of open systems are discussed, paying particular attention to the rigorous mathematical definition, to the physical interpretation and classification, as well as to the quantification of memory effects. The general theory is illustrated by a series of examples. The analysis reveals that memory effects of the open system dynamics reflect characteristic features of the environment which opens a new perspective for applications, namely to exploit a small open system as a quantum probe signifying nontrivial features of the environment it is interacting with. This article further explores the various physical sources of non-Markovian quantum dynamics, such as structured spectral densities, nonlocal correlations between environmental degrees of freedom and correlations in the initial system-environment state, in addition to developing schemes for their local detection. Recent experiments on the detection, quantification and control of non-Markovian quantum dynamics are also discussed.