We report structure-terahertz (THz) property relationship for various non-oxide chalcogenide glasses including unary (vitreous selenium (Se)), binary (arsenic sulfide (As-S), arsenic selenide (As-Se), and germanium selenide (Ge-Se)), and ternary (germanium arsenic selenide (Ge-As-Se)), systems along with commercially available AMTIR-1, IRG 22, and IRG 24 Ge-As-Se glasses. This comprehensive study is the first of its kind to combine Raman spectroscopy to examine structural units, connectivity, and glass network and terahertz time-domain spectroscopy (THz-TDS) to record the THz refractive index, n(THz), across a broad THz bandwidth. THz-TDS was carried out at Alfred University (AU) and National Institute of Standards and Technology (NIST), ultimately providing confidence in n(THz) values measured at AU. Vitreous Se, <r> = 2.0, record the minimum THz refractive index value of all Se-containing glasses. As-S and As-Se binary glasses have the highest measurable THz refractive index value at <r> = 2.4. Ge-Se binary glasses measure increased THz refractive index as <r> increases, with the maximum at <r> = 2.8. Ternary Ge-As-Se glasses record the maximum THz refractive index value at <r> = 2.5 for Ge10As30Se60. Low-repetition rate femtosecond laser irradiation (≈1 KHz, ≈40 fs, and ≈70 mW) was used to modify As-S and As-Se glass systems, where Raman and THz-TDS were used to observe minimal structural and THz refractive index values changes, respectively. Long-wave infrared (LWIR) (e.g., 10 μm)-THz (e.g., 1.0 THz) refractive index correlation is presented for all binary and ternary studied chalcogenide glasses. Such a correlation is valuable for predicting and designing chalcogenide glasses for integrated optical applications across THz and IR regions.