The relationship between microstructure and properties of hydroxylated hexagonal boron nitride nanoplatelets (denoted as OH-h-BN) reinforced epoxy composite coatings was still not clear. In this work, hexagonal boron nitride nanoplatelets (h-BN) were successfully hydroxylated to increase their dispersion and compatibility with the EP matrix, which was confirmed by the results obtained from FTIR, XPS, XRD and TEM. Subsequently, OH-h-BN reinforced epoxy resin (EP) composite coatings were prepared by means of a spraying process, and their curing characteristics, cross-linking density and mechanical and tribological properties were discussed. As a result, the EP/OH-h-BN composite coating took an excellent curing label in terms of the ‘Cure Index’ (CI) and had a cure enthalpy 1.5 times higher than that of the pure EP coating. Moreover, with the presence of OH-h-BN, the storage modulus (at 40 °C) and cross-linking density of the epoxy composite coating were enhanced by 56% and 43%, respectively. The tensile test outcomes presented approximately 16% and 138% improvements in the maximum tensile strength and elongation at break of the EP/OH-h-BN composite coating compared with pure EP coating. In addition, the friction coefficient and the wear rate were reduced by 27% and 58% because of the enhancement of mechanical strength of the EP/OH-h-BN composite coating and the formation of a continuous, dense and high load-carrying lubricating tribofilm on the dual steel ball.