In quantum mechanical many-body systems, long-range and anisotropic interactions promote rich spatial structure and can lead to quantum frustration, giving rise to a wealth of complex, strongly correlated quantum phases1. Long-range interactions play an important role in nature; however, quantum simulations of lattice systems have largely not been able to realize such interactions. A wide range of efforts are underway to explore long-range interacting lattice systems using polar molecules2-5, Rydberg atoms2,6-8, optical cavities9-11 or magnetic atoms12-15. Here we realize novel quantum phases in a strongly correlated lattice system with long-range dipolar interactions using ultracold magnetic erbium atoms. As we tune the dipolar interaction to be the dominant energy scale in our system, we observe quantum phase transitions from a superfluid into dipolar quantum solids, which we directly detect using quantum gas microscopy with accordion lattices. Controlling the interaction anisotropy by orienting the dipoles enables us to realize a variety of stripe-ordered states. Furthermore, by transitioning non-adiabatically through the strongly correlated regime, we observe the emergence of a range of metastable stripe-ordered states. This work demonstrates that novel strongly correlated quantum phases can be realized using long-range dipolar interactions in optical lattices, opening the door to quantum simulations of a wide range of lattice models with long-range and anisotropic interactions.