Magnetic buoyancy instability, which is of astrophysical importance, results from the influence of magnetic pressure variations on the density of a fluid in a gravitational field. It is inherently a compressible phenomenon and is, as such, fully described by the equations of compressible magnetohydrodynamics (MHD). For analytical and computational reasons, it is often convenient to study compressible MHD within simpler, asymptotically consistent reduced systems; the two most widely used result from the Boussinesq and anelastic approximations. Within the standard Boussinesq approximation of MHD, leading to the equations of Boussinesq magnetoconvection, magnetic buoyancy is excluded. It can, however, be included by a rescaling of the basic-state variables and by making further assumptions about the perturbation length scales. Within the anelastic approximation, no special measures are taken to incorporate magnetic buoyancy. It is, however, a priori unclear as to whether this neglect is justified, particularly in the light of the Boussinesq results. Our aims here are thus twofold. The first is to formulate the relationship between descriptions of magnetic buoyancy in the compressible, anelastic and Boussinesq systems. In so doing, we show that, under both the anelastic and Boussinesq approximations, magnetic buoyancy can be included either through a combination of a weak field and strong gradient, or, conversely, a strong field and weak gradient. Each has its own asymptotically consistent reduction, with dedicated governing equations. Our second aim is to address, through a linear stability analysis, under which conditions the standard anelastic system provides a faithful representation of magnetic buoyancy instability. For completeness, we also formulate the energy principle of ideal MHD within the anelastic framework, and demonstrate the relation with its fully compressible counterpart.