We apply the dynamical generator coordinate method (DGCM) with a conjugate momentum to a nuclear collective excitation. To this end, we first discuss how to construct a numerically workable scheme of the DGCM for a general one-body operator. We then apply the DGCM to the quadrupole vibration of $^{16}$O using the Gogny D1S interaction. We show that both the ground state energy and the excitation energies are lowered as compared to the conventional GCM with the same number of basis functions. We also compute the sum rule values for the quadrupole and monopole operators, and show that the DGCM yields more consistent results than the conventional GCM to the values from the double commutator. These results imply that the conjugate momentum is an important and relevant degree of freedom in collective motions.