The interplay between quantum Hall ordering and spontaneously broken "internal" symmetries in two-dimensional electron systems with spin or pseudospin degrees of freedom gives rise to a variety of interesting phenomena, including novel phases, phase transitions, and topological excitations. Here we develop a theory of broken-symmetry quantum Hall states, applicable to a class of multi-valley systems, where the symmetry at issue is a point group element that combines a spatial rotation with a permutation of valley indices. The anisotropy of the dispersion relation, generally present in such systems, favors states where all electrons reside in one of the valleys. In a clean system, the valley "pseudo-spin" ordering, or spatial nematic ordering, occurs via a finite temperature transition. In weakly disordered systems, domains of pseudo-spin polarization are formed, which prevents macroscopic valley and nematic ordering; however, the resulting state still asymptotically exhibits the QHE. We discuss the transport properties in the ordered and disordered regimes, and the relation of our results to recent experiments in AlAs.