Micro-milling, widely employed across various fields, faces significant challenges due to the small diameter and limited stiffness of its tools, making the process highly susceptible to cutting chatter and premature tool breakage. Ensuring stable and safe cutting processes necessitates the prediction of chatter by considering the tool breakage. Crucially, the modal parameters of the spindle–holder–tool system are important prerequisites for such stability prediction. In this paper, the FRFs of the micro-milling tool are calculated by direct frequency response functions (FRFs) of the micro-milling cutter and cross-FRFs between a point on the shank and one on the tool tip. Additionally, by utilizing a cutting force model specific to micro-milling, the bending stress experienced by the tool is computed, and the tool breakage curve is subsequently determined based on the material’s permissible maximum allowable stress. The FRFs of the micro-milling tool, alongside the tool breakage curve, are then integrated to generate the final stability lobe diagrams (SLDs). The effectiveness and reliability of the proposed methodology are confirmed through a comprehensive series of numerical and experimental validations.