Carbon fiber reinforced polymer (CFRP) composites are extensively utilized as primary load-bearing components in various engineering applications due to their superior strength-to-weight ratio and excellent mechanical properties. However, their intricate microstructural interactions within composite present a significant challenge for failure analysis of CFRP. Although finite element (FE) simulations have been proven feasible to conduct the failure analysis, the classical FE models are developed based on homogeneous fiber characteristics, ignoring the influence of internal structures on the damage evolution process. This paper presents a multiscale image-based modeling approach to predict the tensile failure procedure of chopped carbon fiber sheet molding compound (SMC) composite. To accurately reconstruct the representative volume element (RVE) model of the SMC composite, synchrotron micro-X-ray computed tomography (μXCT) was adapted to explore the SMC internal microstructure. Then microscale RVE models with different fiber volume fractions were constructed to predict the corresponding microcosmic mechanical properties, which were used as the inputs for mesoscale RVE models to determine the constitutive parameters of fiber chips having varied fiber volume and orientations. Finally, the YOLOv5_Seg algorithm was employed to extract the geometric feature parameters of the fiber chips for mesoscale RVE modeling and then the failure location and sequence under uniaxial tension were predicted. It is found that the final simulated failure behaviors were consistent with the experimental observations, confirming the feasibility of this approach for understanding the failure mechanisms of CFRP composites. Thus, once the internal microstructure is determined using experimental techniques or predicted by simulating the composite manufacturing process, this approach can also be utilized for design optimization and performance evaluation for CFRP composites.