The interlaminar fracture toughness of unidirectional fiber reinforced polymer composite material is temperature and crack growth history dependence. The non-constant interlaminar fracture toughness challenges the wide application of the linear elastic fracture mechanics-based approach for analyzing the interlaminar crack growth in composite material and structures. In addition, a quantitative relationship between the ductility of the matrix and interlaminar fracture toughness of the fiber reinforced composite is still highly desirable. In this paper, both pure resin and composite double cantilever beam (DCB) are tested at different temperatures. A finite crack growth energy release rate is used to analyze the interlaminar crack growth behavior of composite material at different temperatures. The plasticity of the composite DCB is modeled by Hill’s anisotropic plasticity, with the properties determined from multiscale analyses. It is found that a single constant finite growth energy release rate can well predict the interlaminar mode I crack growth of composite material at different temperatures. The roles of the ductility of matrix, thickness of the DCB, and crack growth history on the interlaminar mode I fracture toughness of composite material are quantitatively determined by using the present method.