Aims. The stability of the International Celestial Reference Frame (ICRF), realized through geodetic very long baseline interferometry (VLBI) positions of thousands of extragalactic objects, is dependent on the individual positional stability of these objects. It has been recently shown that the prevalence of offsets between the VLBI positions of ICRF objects and their Gaia optical positions, which limit the optical-radio reference frame tie, is inversely correlated with optical photometric variability, suggesting that photometrically variable objects may be more positionally stable. In this work, we determine the relationship between VLBI position stability of ICRF objects and optical-radio position offsets as well as optical photometric variability. Methods. We created multi-epoch geodetic VLBI solutions for a sample of 520 ICRF sources that have sufficient data to determine the variability in their VLBI positions over time. We compared this position variability with the fractional photometric variability provided by the Gaia extragalactic source catalog, the Gaia-ICRF optical-radio position offsets, the uncertainty-normalized position offsets, and optical BP-RP color as well as with possible confounders such as optical magnitude, VLBI/Gaia position error, and redshift. We determined the relationship between VLBI position stability and γ-ray detection by the Fermi Large Area Telescope (LAT), and we determined how the VLBI position and optical flux variabilities correlate with the spectral classification of our sample, considering flat spectrum radio quasars (FSRQs), quasi-stellar objects, BL Lacs, Seyfert, and gigahertz-peaked spectrum radio sources or compact-steep-spectrum radio sources. Results. We found that VLBI astrometric variability is (i) negatively correlated with optical flux variability, (ii) positively correlated with optical-radio offsets, (iii) negatively correlated with optical color index BP-RP, and (iv) negatively correlated with γ-ray detection. We also found that the most positionally stable sources are among the FSRQ and BL Lac classes. In other words, redder, photometrically variable sources have the most stable VLBI positions, the smallest optical-radio position offsets, and the highest rate of γ-ray detection, and these sources tend to be spectrally classified as blazars. Conclusions. Our results are consistent with the most positionally stable sources being blazars, a class of object in which the jet is oriented close to the line of sight and where relativistic beaming increases photometric variability and minimizes the projected offset between the optical and radio positions. Our study should therefore orient future geodetic VLBI observing programs preferentially toward sources with high photometric variability because these sources are predicted to have better VLBI position stabilities and smaller optical-radio position offsets, improving the stability of the celestial reference frame axes.