Studying the tribological behavior between diamond materials and ceramics is crucial for understanding the grinding mechanism of diamond wheels on ceramics. To comprehend the tribological characteristics between diamond materials and various ceramics, this paper presents a comparative study of the tribological behaviors of Al2O3, AlN, Si3N4, and ZrO2 ceramics sliding against polycrystalline diamond (PCD) balls. The dependence of the coefficient of friction (COF), wear characteristics as well as the surface morphology of different ceramics and SiC-bonded PCD balls on load, sliding stage were examined. The effect of nanofluids on the COF and wear of different ceramics was also investigated. The results showed that the tribological characteristics of ceramic materials are strongly dependent on mechanical properties. Al2O3 and AlN ceramics with lower fracture toughness exhibit a significantly higher COF and wear volume than that of Si3N4 and ZrO2 ceramics. Al2O3 presents the highest COF (0.38) and wear volume, and Si3N4 owns the lowest COF (0.08) and wear volume under the load of 9.8 N. By examining the wear grooves of different ceramics, it is found that Al2O3 and AlN ceramics own larger debris than that of Si3N4 and ZrO2, which also causes more smearing on the ceramic and adhesion of wear debris to the SiC-bonded PCD ball for Al2O3 and AlN. Under lubricants, the COFs and the adhesion of wear debris are effectively reduced, and the wear rate of the ceramic is increased. There is little difference between deionized (DI) water and nanofluids on the effect of lubricants on COF for Al2O3 and AlN. However, for ceramics with higher fracture toughness, the nanofluids can reduce the COFs of Si3N4 and ZrO2 in comparison with DI water.