Exploring the relationship between thermal expansion and structural complexity is a challenging topic in the study of modern materials where volume stability is required. This work reports a new family of negative thermal expansion (NTE) materials, AM(CN)4 with A = Li and Na and M = B, Al, Ga, and In. Here, the compounds of LiB(CN)4 and NaB(CN)4 were only synthesized; others were purely computationally studied. A critical role of nonrigid vibrational modes and spiral acoustical modes has been identified in NaB(CN)4. This understanding has been exploited to design the colossal NTE materials of NaM(CN)4 (M = Al, Ga, In). A joint study involving synchrotron X-ray diffraction, Raman spectroscopy, and first-principles calculations has been conducted to investigate the thermal expansion mechanism. It has been found that the A atoms can either increase the symmetry of the crystal structure, inducing stronger NTE, or lower the crystal symmetry, thus resulting in positive thermal expansion. Conversely, the M-site atoms do not affect the crystal structure. However, as the radius of the M atoms increases, the ionic nature of the C-M bonds strengthens and the CN vibrations become more flexible, thereby enhancing the NTE behavior. This study provides new insights to aid in the discovery and design of novel NTE materials and the control of thermal expansion.