The elucidation of chemical bonding, coupled with an exploration of the correlated dynamics of constituent atoms, is essential for unravelling the underlying mechanism responsible for low lattice thermal conductivity (κL) exhibited by a crystalline solid, which is essential for thermoelectrics and thermal barrier coatings. In this regard, Pauling's third empirical rule, which deals with the cationic repulsion due to proximity in the face or edge shared polyhedra in a crystal structure, can bring about the lattice instability required to suppress the κL. Here, we demonstrate the presence of such instability in a ternary selenide, TlAgSe, leading to a ultra-low κL of 0.17 W/m.K at 573 K. Our study reveals the instability arising from Ag-Ag repulsion within edge-shared AgSe4 tetrahedra through investigation of the local structure using synchrotron X-ray pair distribution function (PDF) analysis and supported by first-principles density functional theory calculations. We observe correlation between weakening in the Ag and the Tl-sublattice, providing direct experimental evidence of Pauling's third empirical rule. The correlated rattling of Ag and Tl induces a highly anharmonic lattice and low energy optical phonons, resulting in suppressed sound velocity and ultralow κL in TlAgSe. The electronic origin of soft and anharmonic lattice is the presence of filled antibonding states in the valence band near the Fermi level constructed by Ag(4d)-Se(4p) and Tl(6s)-Se(4p) interactions. This work demonstrates that the evidence of dynamic distortion in a crystal lattice is governed by the third empirical rule given by Pauling, which can act as a potential new strategy for diminishing κL in crystalline solids.