Combining the effect of layer mixing, mass mismatch, and intrinsic defects, we have investigated the origin of very low phonon conductivity ${k}_{p}$ in thermoelectric (TE) $\text{BiCuO}Q$ $(Q$: S, Se, Te) compounds. Based on the first-principles anharmonic, lattice dynamics calculations, we use the single-mode relaxation time approximation of the linearized phonon Boltzmann equation, which shows good agreement with experiments. Here, we found that the most important parameter for low ${k}_{p}$ is the interlayer interaction between the BiO and $\text{Cu}Q$ layers. By analyzing the phonon linewidth distribution, which indicates the phonon scattering rate, we propose that the interlayer interactions play a critical role on suppressing ${k}_{p}$, i.e., the heterolayered crystal controls these interlayer interactions, achieving low ${k}_{p}$ and optimal TE properties.