Accurately measuring the coefficient of friction (COF) is the fundamental prerequisite of superlubricity research. This study aimed to reduce the COF measurement resolution Δμ of atomic force microscopy (AFM). Based on the theoretical model, a distinctive strategy was adopted to reduce Δμ by optimizing the cantilever’s cross-section of the AFM probe, inspired by civil engineering. Δμ can be reduced by decreasing the width of the horizontal side wR and the wall thickness t and increasing the width of the vertical side wH. Moreover, the I-shape demonstrates the highest reduction in Δμ, followed by the U-shape. Considering the processability, the AFM probe with the U-shaped cross-sectional cantilever was investigated further, and the dimensions are 35 µm wR, 3.5 µm wH, 0.5 µm t, 50 µm l (cantilever length), and 23 µm htip (tip height). The finite element analysis results confirm its reliability. After being fabricated and calibrated, the AFM probe achieves the minimal Δμ of 1.9×10−6 under the maximum normal force so far. Additionally, the friction detection capability of the fabricated AFM probe improves by 78 times compared to the commercial tipless-force modulation mode (TL-FM) AFM probe with the conventional solid rectangular cross-sectional cantilever. This study provides a powerful tool for measuring 10−6 COF.