In quasicrystals lacking translational symmetry but having highly ordered structures, understanding how phonons propagate in their aperiodic lattices remains an unsolved issue. We present an inelastic neutron scattering study on acoustic phonon modes of icosahedral quasicrystal AlPdMn, revealing hierarchical pseudo-gap structure in low-energy acoustic modes. Additionally, phonon intensities are asymmetric in energy and wave vectors with respect to the Bragg peak, indicating characteristic nonreciprocal phonon propagation in quasicrystals.