Longitudinal tensile failure prediction has always been the focus of research on unidirectional (UD) Carbon Fiber Reinforced Polymer (CFRP). In this paper, the longitudinal tensile tests of unidirectional T800 grade carbon fiber reinforced epoxy resin composite were carried out first. The experimental stress–strain curves show that there are obvious progressive damages before the final failures of the specimens. However, this phenomenon is usually neglected in most of the existing investigations and tests to only obtain elastic modulus and ultimate strength. Thus, a Representative Volume Element (RVE) model based on micromechanics was established using finite element method, considering the randomness of fiber spatial distribution and fiber strength. The tensile strength of fiber monofilaments was assumed to follow a statistical two-parameter Weibull distribution. The failure process of UD composite under longitudinal tensile load was simulated by explicit finite element method. The stress–strain curves obtained by simulation are in good agreement with the experimental results. Finally, the influence of different boundary conditions on the calculation of the RVE was studied. It is found that the final failure strengths of the RVE with uniform displacement boundary conditions are similar to the case of periodic boundary conditions, while the former has higher computational efficiency.