We derive the stress tensor of a rigid dumbbell by using the virtual work method. In the virtual work method, we virtually apply a small deformation to the system, and relate the change of the energy to the work done by the stress tensor. A rigid dumbbell consists of two particles connected by a rigid bond of which length is constant (the rigid constraint). The energy of the rigid dumbbell consists only on the kinetic energy. Also, only the deformations which do not violate the rigid constraint are allowed. Thus we need the dynamic equations which are consistent with the rigid constraint to apply the virtual deformation. We rewrite the dynamic equations for the underdamped SLLOD-type dynamic equations into the forms which are consistent with the rigid constraint. Then we apply the virtual deformation to a rigid dumbbell based on the obtained dynamic equations. We derive the stress tensor for the rigid dumbbell model from the change of the kinetic energy. Finally, we take the overdamped limit and derive the stress tensor and the dynamic equation for the overdamped rigid dumbbell. We show that the Green-Kubo type linear response formula can be reproduced by combining the stress tensor and the dynamic equation at the overdamped limit.