Over the last decade, substantial efforts have been devoted to understanding the stability properties, transport phenomena, and long-term evolution of weakly-collisional, magnetized plasmas which are stratified in temperature. These studies have improved our understanding of the physics governing the intra-cluster medium (ICM), but assumed that ICM is a homogeneous. This, however, might not be a good approximation if heavy elements sediment in the inner region of the galaxy cluster. In this paper, we analyze the stability of a weakly-collisional, magnetized plane-parallel atmosphere which is stratified in both temperature and composition. This allows us to discuss for the first time the dynamics of weakly-collisional environments where heat conduction, momentum transport, and ion-diffusion are anisotropic with respect to the direction of the magnetic field. We show that, depending on the relative signs and magnitudes of the gradients in the temperature and the mean molecular weight, the plasma can be subject to a wide variety of unstable modes which include modifications to the magnetothermal instability (MTI), the heat-flux-driven buoyancy instability (HBI), and overstable gravity modes previously studied in homogeneous media. We discuss the astrophysical implications of our findings for a representative galaxy cluster where helium has sedimented. Our findings suggest that the core insulation that results from the magnetic field configurations that arise as a natural consequence of the HBI, which would be MTI stable in a homogeneous medium, could be alleviated if the mean molecular weight gradient is steep enough, i.e., $(\nabla \mu)/\mu > (\nabla T)/T$. This study constitutes a first step toward understanding the interaction between magnetic turbulence and the diffusion of heavy elements in the ICM. (abridged)