The carbon vacancy in $4H$-SiC is a powerful minority carrier recombination center and a major cause of degradation of SiC-based devices. Despite the extensiveness of the literature regarding the characterization and modeling of the defect, many fundamental questions persist. Among them we have the shaky connection of the EPR data to the electrical measurements, the physical origin of the pseudo-Jahn-Teller (pJT) effect, the reasoning for the observed sub-lattice dependence of the paramagnetic states, and the severe temperature-dependence of some hyperfine signals which cannot be accounted for by a thermally-activated dynamic averaging between equivalent JT-distorted structures. In this work we address these problems by means of semi-local and hybrid density functional calculations. We start by inventorying a total of four vacancy structures. Diamagnetic states have well defined low-energy structures, whereas paramagnetic states display metastability. This rich structural variety is traced back to the filling of electronic states which are shaped by a crystal-field-dependent pJT effect. From calculated minimum energy paths for defect rotation and transformation mechanisms, combined with the calculated formation energies and electrical levels, we arrived at a configuration-coordinate diagram of the defect. The diagram provides us with a detailed first-principles picture of the defect when subject to optical and thermal excitations. The calculated acceptor and donor transitions agree well with Z$_{1/2}$ and EH$_{6/7}$ trap energies, respectively. From comparison of calculated and measured $U$-values, and correlating the site-dependent formation energies with the relative intensity of the DLTS peaks in as-grown material, we assign Z$_{1}$ (EH$_{6}$) and Z$_{2}$ (EH$_{7}$) signals to acceptor (donor) transitions of carbon vacancies located on the $h$ and $k$ sub-lattice sites, respectively.