In this study, a multi-scale modeling framework that spans from molecular chains to macroscopic structure was proposed for a typical particle-filled polymer composite (HTPB propellants). The cohesive zone model (CZM) is utilized in the RVE model of HTPB propellants to capture the debonding phenomena at the AP/HTPB interface. For the HTPB matrix, a free energy function based on the Gaussian chain network is employed. To depict the chain scission behavior, the phase fracture (PF) method along with gradient-damage theory is introduced. Subsequently, microscale fracture behavior under uniaxial tensile load was investigated based on the constructed RVE model, and a phenomenological macroscopic damage model was developed correspondingly. In this developed model, two damage factors related to the debonding of AP/HTPB interface and the growth of voids in matrix are introduced respectively. Thus, it can not only predict the macroscopic stress–strain response, but also can give the microscopic damage evolution information. Overall, this multi-scale modeling framework can offer us a deeper insight into the microstructural changes and the resulting macroscopic mechanical behavior of HTPB propellants.