Heavy crude oil exhibits very complex viscoelastic behaviors due to its complex composition of resins, asphaltenes, saturates, and aromatics. It has a great influence on oil production and transportation. In this work, the viscoelastic behaviors of three different heavy crude oils were measured using a rotational rheometer. In conclusion, all of these heavy crude oils display linear viscoelastic behaviors in the experimental range. The loss modulus (E″) of the three crude oils decreased as the experimental temperature increased, and the variation trends of the three crude oils were basically the same. However, the experimental temperature has almost no effect on the storage modulus (E′), which always retained a constant value of 0.4 Pa. Furthermore, the storage modulus (E′) and loss modulus (E″) increase as the angular frequency increases. To describe the physical deformation characteristics of viscoelastic materials, the generalized Maxwell model and the fractional derivative Maxwell model are used to establish the constitutive relation of heavy crude oil. In conclusion, the generalized Maxwell model and the fractional derivative Maxwell model can predict the experimental results very well. All of the square of the correlation coefficient (R2) values are greater than 0.95. However, the number of fitting parameters for the fractional derivative Maxwell model is less than that for the fourth-order generalized Maxwell model which can save the calculating time. Therefore, the fractional derivative Maxwell model is suggested to describe the viscoelastic behavior of heavy crude oil in industrial applications.