All-inorganic lead-free halide double perovskites offer a promising avenue toward non-toxic, stable optoelectronic materials, properties that are missing in their prominent lead-containing counterparts. Their large thermopowers and high carrier mobilities also make them promising for thermoelectric applications. Here, we present a first-principles study of the lattice vibrations and thermal transport behaviors of Cs2SnI6 and γ-CsSnI3, two prototypical compounds in this materials class. We show that conventional static zero temperature density functional theory (DFT) calculations severely underestimate the lattice thermal conductivities (κl) of these compounds, indicating the importance of dynamical effects. By calculating anharmonic renormalized phonon dispersions, we show that some optic phonons significantly harden with increasing temperature (T), which reduces the scattering of heat carrying phonons and enhances calculated κl values when compared with standard zero temperature DFT. Furthermore, we demonstrate that coherence contributions to κl, arising from wave like phonon tunneling, are important in both compounds. Overall, calculated κl with temperature-dependent interatomic force constants, built from particle and coherence contributions, are in good agreement with available measured data, for both magnitude and temperature dependence. Large anharmonicity combined with low phonon group velocities yield ultralow κl values, with room temperature values of 0.26 W/m-K and 0.72 W/m-K predicted for Cs2SnI6 and γ-CsSnI3, respectively. We further show that the lattice dynamics of these compounds are highly anharmonic, largely mediated by rotation of the SnI6 octahedra and localized modes originating from Cs rattling motion. These thermal characteristics combined with their previously computed excellent electronic properties make these perovskites promising candidates for optoelectronic and room temperature thermoelectric applications.