The cosmology in the Hubble expansion era of the Horava-Witten M theory compactified on a Calabi-Yau threefold is studied in the reduction to five-dimensions where the effects of the Calabi-Yau manifold are summarized by the volume modulus, and all perturbative potentials are included. Matter on the branes are treated as first order perturbations of the static vacuum solution, and all equations in the bulk and all boundary conditions on both end branes are imposed. It is found that for a static volume modulus and a static fifth dimension, $y$, one can recover the four-dimensional Robertson Friedmann Walker cosmology for relativistic matter on the branes, but not for nonrelativistic matter. In this case, the Hubble parameter $H$ becomes independent of $y$ to first order in matter density. This result holds also when an arbitrary number of 5-branes are included in the bulk. The five-dimensional Horava-Witten model is compared with the Randall-Sundrum phenomenology with a scalar field in the bulk where a bulk and brane potential are used so that the vacuum solutions can be rigorously obtained. (In the Appendix, the difficulty of obtaining approximate vacuum solutions for other potentials is discussed.) In this case nonrelativistic matter is accommodated by allowing the distance between the branes to vary. It is suggested that nonperturbative potentials for the vacuum solution of Horava-Witten theory are needed to remove the inconsistency that nonrelativistic matter creates.