This paper studies geometric and spectral properties of $S$-adic shifts and their relation to continued fraction algorithms. These shifts are symbolic dynamical systems obtained by iterating infinitely many substitutions. Pure discrete spectrum for $S$-adic shifts and tiling properties of associated Rauzy fractals are established under a generalized Pisot assumption together with a geometric coincidence condition. These general results extend the scope of the Pisot substitution conjecture to the $S$-adic framework. They are applied to families of $S$-adic shifts generated by Arnoux-Rauzy as well as Brun substitutions. It is shown that almost all of these shifts have pure discrete spectrum. Using $S$-adic words related to Brun's continued fraction algorithm, we exhibit bounded remainder sets and natural codings for almost all translations on the two-dimensional torus. Due to the lack of self-similarity properties present for substitutive systems we have to develop new proofs to obtain our results in the $S$-adic setting.