The last decade has witnessed explosive growth in terahertz radiation fields, ranging from fundamental research to applications. This progress has been marked by the introduction of various terahertz radiation sources and detectors. Here, we review the progress of nonlinear pnictide and chalcogenide ternary crystals for use in the terahertz frequency regime. At the center of this topical review are the various physical and optical properties of numerous emerging pnictide and chalcogenide nonlinear crystals, which are anchored to state-of-the-art coherent terahertz radiation sources based on optical rectification and difference frequency generation, as well as terahertz radiation phase-resolved detectors based on electro-optic sampling. This review aims to provide the foundation necessary to continue advancing pnictide and chalcogenide ternary crystals for the generation, detection, and manipulation of terahertz radiation.