With the introduction of the modern rammed earth technique, a large number of modern rammed earth buildings were constructed in China Mainland. China has a vast territory, which faces the Circum-Pacific seismic belt on the east and the Eurasian seismic belt on the south; earthquake has constantly threatened the safety of people’s lives and property. Consequently, it is necessary to probe in the seismic performance of rammed earth buildings. Two un-stabilized rammed earth specimens, one un-stabilized rammed earth reinforced with geogrid sheets’ specimens, and four stabilized rammed earth specimens were built for obtaining a better insight on the behavior of un-stabilized rammed earth/stabilized rammed earth walls under cyclic in-plane loads. Testing results are discussed in terms of failure mode, shear capacity, hysteretic curve, stiffness degradation, and total energy dissipation to provide comparisons of the seismic performance between un-stabilized rammed earth and stabilized rammed earth specimens. Different failure modes indicated that the cohesion between particles and the bond strength between layers are the two key parameters for the shear capacity of rammed earth buildings. It is also demonstrated that stabilized rammed earth specimens have higher shear and energy dissipation capacity but weaker deformation capacity than un-stabilized rammed earth.