Coordination complexes that possess large magnetic anisotropy (otherwise known as zero-field splitting, ZFS) have possible applications in the field of magnetic materials, including single molecule magnets (SMMs). Previous studies have explored the role of coordination number and geometry in controlling the magnetic anisotropy and SMM behavior of high-spin (S = 3/2) Co(II) complexes. Building upon these efforts, the present work examines the impact of ligand oxidation state and structural distortions on the spin states and ZFS parameters of pentacoordinate Co(II) complexes. The five complexes included in this study (1-5) have the general formula, [Co(TpPh2)(LX,Y)]n+ (X = O, S; Y = N, O; n = 0 or 1), where TpPh2 is the scorpionate ligand hydrotris(3,5-diphenyl-pyrazolyl)borate(1-) and LX,Y are bidentate dioxolene-type ligands that can access multiple oxidation states. The specific LX,Y ligands used herein are 4,6-di-tert-butyl substituted o-aminophenolate and o-aminothiophenolate (1 and 2, respectively), o-iminosemiquinonate and o-semiquinonate radicals (3 and 4, respectively), and o-iminobenzoquinone (5). Each complex exhibits a distorted trigonal bipyramidal geometry, as revealed by single-crystal X-ray diffraction. Direct current (dc) magnetic susceptibility experiments confirmed that the complexes with closed-shell ligands (1, 2, and 5) possess S = 3/2 ground states with negative D-values (easy-axis anisotropy) of -41, -78, and -30 cm-1, respectively. For 3 and 4, antiferromagnetic coupling between the Co(II) center and o-(imino)semiquinonate radical ligand results in S = 1 ground states that likewise exhibit very large and negative anisotropy (-100 > D > -140 cm-1). Notably, ZFS was measured directly for each complex using far-infrared magnetic spectroscopy (FIRMS). In combination with high-frequency and -field electron paramagnetic resonance (HFEPR) studies, these techniques provided precise spin-Hamiltonian parameters for complexes 1, 2, and 5. Multireference ab initio calculations, using the CASSCF/NEVPT2 approach, indicate that the strongly negative anisotropies of these Co(II) complexes arise primarily from distortions in the equatorial plane due to constrictions imposed by the TpPh2 ligand. This effect is further amplified by cobalt(II)-radical exchange interactions in 3 and 4.