Kernel hardness (KH) is one of the primary quality parameters for common wheat (Triticum aestivum L.) and has a major impact on milling, flour quality, and end-product properties. In addition to Puroindoline (Pin) mutations and differences in Pin expression, other factors, such as kernel size and protein-related traits, play noticeable roles in determining hardness, but at the quantitative trait locus (QTL) level, the influence of these factors remains unclear. In this study, genetic relationships between KH and kernel size traits and between KH and protein-related traits were demonstrated by unconditional and conditional mapping using a wheat 90K genotyping assay with a segregating population of 173 recombinant inbred lines in four environments. Eight additive QTL for KH were detected using unconditional QTL mapping analysis; these QTL were primarily distributed on chromosomes 4B, 5A, 5B, and 6D, with phenotypic variation that ranged from 0.2 to 17.7%. In addition, one pair of epistatic QTL (QKH3B.4-65/QKH4B.6-2) was identified by unconditional mapping, and this pair accounted for 1.6% of the phenotypic variation. Through conditional mapping, after excluding the influences of kernel size and protein-related traits, 14 QTL were discovered and accounted for 0.6–18.5% of the phenotypic variation. Of them, the stable QTL QKH4B.4-17 made the largest contribution, which was partially contributed by the kernel length (KL), kernel thickness (KT), and dry gluten content (DGC). Furthermore, QKH4B.4-17 was crucially contributed by the kernel width (KW), kernel diameter (KD), kernel protein content (KPC), and wet gluten content (WGC) and was independent of the sedimentation volume (SV) and gluten index (GI). Another major QTL, QKH5B.10-63, was independent of the KW and KT; partly due to the variations in KL, KD, DGC, and WGC; and conclusively contributed by the KPC, SV, and GI. Seven additional QTL were only detected in the conditional analysis and were crucially contributed by kernel size or protein-related traits. These results demonstrated that kernel size and protein-related traits play significant roles in determining KH. The present study increases the understanding of the relationships between KH and kernel size and between KH and protein-related traits at the QTL level.