Stellite alloys are widely utilized in the aerospace industry due to their excellent hardness and high wear resistance. Optimal properties are predominantly achieved through engineering desired microstructures in terms of type, size, shape, and spatial distribution of carbides within the Co-Cr matrix through alloying. However, a quantitative linkage among composition, carbide, and hardness (CCH) is still lacking. Herein, we attempt to tailor the essential reinforcement elements, Mo and C, to obtain different Stellite alloys using powder metallurgy (PM). With the help of image recognition technology, microstructures of alloys (including the type and content of carbides, as well as the content of defects.) were quantitatively analyzed. Besides, mathematical algorithms based on Analysis of Variance (ANOVA) and Desirability Functional Analysis (DFA) were developed to establish the models for the quantification of CCH. Specifically, the regression equations provide the quantitative relationship between elements (Mo and C), two primary carbides (M7C3 (M=Metal) and M23C6), and the hardness. We believe this quantitative work assisted by image recognition would be beneficial for the development of Stellite alloys and could shed light on the CCH relationship of other cemented carbides alloys.