The relationship between the chemical elements of high-entropy boride (HEB) ceramics and their hardness is important for the prediction of high-hardness HEB ceramics. In this work, by designing four HEB ceramics with different chemical elements, the effect of lattice parameter difference factor (δ, represents the difference-degree in lattice parameters among the five individual constitute diborides) on phase composition and lattice-distortion, and the effect of rule of mixture (ROM) average hardness and lattice-distortion on HEB ceramics hardness were studied. The results indicated that, as δ value increases, more severe lattice-distortion occurs inside the HEB ceramics, and a single solid-solution is difficult to be formed. Furthermore, lattice-distortion and the ROM average hardness codetermine the HEB ceramics hardness. The greater lattice-distortion brings about significantly higher hardness for HEB ceramics than their ROM average hardness. Among the four HEBs, (Hf0.2Zr0.2Ta0.2V0.2Nb0.2)B2 exhibits the high hardness of 26 GPa measured using a load of 9.8 N.