Loess is widely distributed in China, with the Loess Plateau having the largest global bulk accumulation. The micro-scale loess landforms are featured by loess walls and loess columns, which can stand vertically for many years. The main failure modes of loess to form these vertical features are toppling and falling along a rupture surface of high angles. These failure modes are attributed to loess strength behavior, which in turn are dominated by the inner structure of loess soil. Bearing this in mind, we sought to conduct a series of mechanical tests on both vertical and horizontal loess samples. These included tension as well as uniaxial and triaxial compression tests. It is found that Malan loess had strong anisotropy in terms of failure pattern and strength. The vertical specimens possess strengths 1.3 times greater than those of horizontal specimens, and tend to fail at high rupture angles of approximately 85° under uniaxial and triaxial compression conditions. These findings were in good agreement with the vertiloess structure, where the vertically aligned strong units resulting from inter-particle bonding and close compaction make the loess behave strongly in the vertical direction, while the vertically aligned weak segments are responsible for the high rupture angles. These findings explain the formation of deep vertical cracks which mobilize loess toppling and falling and shape the loess mass into columns and walls.