Over the last decade, there has been progress in understanding one-dimensional non-Markovian processes via analytic, sometimes exact, solutions. The extension of these ideas and methods to two and higher dimensions is challenging. We report the first exactly solvable two-dimensional (2D) non-Markovian random walk model belonging to the family of the elephant random walk model. In contrast to Lévy walks or fractional Brownian motion, such models incorporate memory effects by keeping an explicit history of the random walk trajectory. We study a memory driven 2D random walk with correlated memory and stops, i.e. pauses in motion. The model has an inherent anisotropy with consequences for its diffusive properties, thereby mixing the dominant regime along one dimension with a subdiffusive walk along a perpendicular dimension. The anomalous diffusion regimes are fully characterized by an exact determination of the Hurst exponent. We discuss the remarkably rich phase diagram, as well as several possible combinations of the independent walks in both directions. The relationship between the exponents of the first and second moments is also unveiled.