We study a family of mutually commutative difference operators introduced by Ruijsenaars. The conjugations of these operators with an appropriate function give the Hamiltonians of some relativistic quantum systems. These operators can be regarded as elliptic analogs of the Macdonald operators and their coefficients consist of the Jacobi theta functions. We show that these operators act on the space of meromorphic functions on the Cartan subalgebra of affine Lie algebras and that the space spanned by characters of a fixed positive level is invariant under the action of these operators.