We study the electronic structure and the magnetic properties of epitaxial (Ni1-xCox)Fe2O4(111) layers (x = 0 - 1) with thicknesses d = 1.7 - 5.2 nm grown on Al2O3(111)/Si(111) structures, to achieve a high value of inversion parameter y, which is the inverse-to-normal spinel-structure ratio, and hence to obtain good magnetic properties even when the thickness is thin enough for electron tunneling as a spin filter. We revealed the crystallographic (octahedral Oh or tetrahedral Td) sites and the valences of the Fe, Co, and Ni cations using experimental soft X-ray absorption spectroscopy and X-ray magnetic circular dichroism spectra and configuration-interaction cluster-model calculation. In all the (Ni1-xCox)Fe2O4 layers with d = about 4 nm, all Ni cations occupy the Ni2+ (Oh) site, whereas Co cations occupy the three different Co2+ (Oh), Co2+ (Td), and Co3+ (Oh) sites with constant occupancies. According to these features, the occupancy of the Fe3+ (Oh) cations decreases and that of the Fe3+ (Td) cations increases with decreasing x. Consequently, we obtained a systematic increase of y with decreasing x and achieved the highest y value of 0.91 for the NiFe2O4 layer with d = 3.5 nm. From the d dependences of y and magnetization in the d range of 1.7 - 5.2 nm, a magnetically dead layer is present near the NiFe2O4/Al2O3 interface, but its influence on the magnetization was significantly suppressed compared with the case of CoFe2O4 layers reported previously [Y. K. Wakabayasi et al., Phys. Rev. B 96, 104410 (2017)], due to the high site selectivity of the Ni cations. Since our epitaxial NiFe2O4 layer with d = 3.5 nm has a high y values (0.91) and a reasonably large magnetization (180 emu/cc), it is expected to exhibit a strong spin filter effect, which can be used for efficient spin injection into Si.