A constitutive model to describe the rate-dependent behavior of unidirectional fiber-reinforced composites is developed from phenomenological and continuum mechanics points of view. A particular emphasis is placed on a unified modeling of elastic, viscoelastic and viscoplastic deformation. Composites are assumed to be homogeneous transversely isotropic media. The constitutive equations are systematically derived from thermodynamic potentials, i.e., free energy and dissipation energy functions. A capability to describe the viscoelastic nature is incorporated into the present model through the Ziegler's thermodynamic reconstruction of the classical rheology models. The present model enables us to describe the rate-dependence of the initial tangent modulus and of the flow stress in the subsequent nonlinear regime. The present model may be interpreted as an extension of the Gates-Sun model for the elastic and viscoplastic behavior of unidirectional composites.