Koopmans's theorem is generalized for excited-state ensemble density-functional theory. Formal expressions are derived that relate orbital energy differences to exchange-only excitation energies. These expressions provide stringent requirements for approximating the universal exchange component of the exact exchange-correlation functional for excited-state ensembles, and the expressions lead to a relation, containing a correlation potential, that gives exact excited-state ionization energies.