(1) Objective: This study aimed to assess the effects of dietary casein phosphopeptide (CPP) supplementation on the egg production performance of late laying hens and the resulting egg quality and eggshell ultrastructure. (2) Methods: A total of 800 laying hens aged 58 weeks were randomly assigned into 5 groups with 8 replicates of 20 hens each. The hens were fed a basal diet supplemented with 0 (control, T1), 0.5 (T2), 1.0 (T3), 1.5 (T4), and 2.0 (T5) g/kg CPP for 9 weeks. (3) Results: Dietary CPP supplementation was found to be beneficial for improving eggshell quality. The spoiled egg rate of the experimental groups was lower than that of the control group (linear and quadratic effect, p < 0.05). The yolk color in the T2, T3, and T4 groups was higher than that in the T1 group (quadratic effect, p < 0.05). The shell thickness in the T4 group was higher than that in the T1 and T2 groups (linear effect, p < 0.05). The shell color in the experimental groups was higher than that in the control group (linear and quadratic effect, p < 0.05). The effective thickness in the T3-T5 groups (linear and quadratic, p < 0.05) and the number of papillary nodes in the T2 and T3 groups were higher than those in the T1 group (quadratic, p < 0.05). The calcium content in the T2 and T3 groups was higher than that in the T1 group (quadratic effect, p < 0.05). The iron content in the T2 and T3 groups was higher than that in the T1 group (p < 0.05). (4) Conclusion: In summary, 0.5-1.0 g/kg CPP supplementation reduced the spoiled egg rate, enhanced the yolk and eggshell colors, increased the thickness of the effective layer, and the calcium and iron contents in the eggshell.