Context. CO rovibrational lines are efficient probes of warm molecular gas and can give unique insights into the inner 10 AU of proto-planetary disks, effectively complementing ALMA observations. Recent studies find a relation between the ratio of lines originating from the second and first vibrationally excited state, denoted as v2∕v1, and the Keplerian velocity or emitting radius of CO. Counterintuitively, in disks around Herbig Ae stars the vibrational excitation is low when CO lines come from close to the star, and high when lines only probe gas at large radii (more than 5 AU). The v2∕v1 ratio is also counterintuitively anti-correlated with the near-infrared (NIR) excess, which probes hot and warm dust in the inner disk. Aims. We aim to find explanations for the observed trends between CO vibrational ratio, emitting radii and NIR excess, and to identify their implications in terms of the physical and chemical structure of inner disks around Herbig stars. Methods. First, slab model explorations in local thermal equilibrium (LTE) and non-LTE are used to identify the essential parameter space regions that can produce the observed CO emission. Second, we explore a grid of thermo-chemical models using the DALI code, varying gas-to-dust ratio and inner disk radius. Line flux, line ratios, and emitting radii are extracted from the simulated lines in the same way as the observations and directly compared to the data. Results. Broad CO lines with low vibrational ratios are best explained by a warm (400–1300 K) inner disk surface with gas-to-dust ratios below 1000 (NCO < 1018 cm−2); no CO is detected within or at the inner dust rim, due to dissociation at high temperatures. In contrast, explaining the narrow lines with high vibrational ratios requires an inner cavity of a least 5 AU in both dust and gas, followed by a cool (100–300 K) molecular gas reservoir with gas-to-dust ratios greater than 10 000 (NCO > 1018 cm−2) at the cavity wall. In all cases, the CO gas must be close to thermalization with the dust (Tgas ~ Tdust). Conclusions. The high gas-to-dust ratios needed to explain high v2∕v1 in narrow CO lines for a subset of group I disks can be naturally interpreted as due to the dust traps that are proposed to explain millimeter dust cavities. The dust trap and the low gas surface density inside the cavity are consistent with the presence of one or more massive planets. The difference between group I disks with low and high NIR excess can be explained by gap opening mechanisms that do or do not create an efficient dust trap, respectively. The broad lines seen in most group II objects indicate a very flat disk in addition to inner disk substructures within 10 AU that can be related to the substructures recently observed with ALMA. We provide simulated ELT-METIS images to directly test these scenarios in the future.