Aims.Clouds are expected to form in a broad range of conditions in the atmosphere of exoplanets given the variety of possible condensible species. This diversity, however, might lead to very different small-scale dynamics depending on radiative transfer in various thermal conditions. Here, we aim to provide some insight into these dynamical regimes.Methods.We performed an analytical linear stability analysis of a compositional discontinuity with a heating source term that depends on a given composition. We also performed idealized two-dimensional simulations of an opacity discontinuity in a stratified medium, using theARKcode. We used a two-stream gray model for radiative transfer and explored the brown-dwarf and Earth-like regimes.Results.We revealed the existence of a radiative Rayleigh-Taylor instability (RRTI, hereafter, which is a particular case of diabatic Rayleigh-Taylor instability) when an opacity discontinuity is present in a stratified medium. This instability is similar in nature to diabatic convection and relies only on buoyancy with radiative transfer heating and cooling. When the temperature is decreasing with height in the atmosphere, a lower-opacity medium on top of a higher-opacity medium is shown to be dynamically unstable, whereas a higher-opacity medium on top of a lower-opacity medium is stable. This stability-instability behavior is reversed if the temperature is increasing with height.Conclusions.The existence of a RRTI could have important implications for the stability of the cloud cover with regard to a wide range of planetary atmospheres. In our Solar System, it could help explain the formation of mammatus cloud in Earth atmospheres and the existence of the Venus cloud deck. Likewise, it suggests that stable and large-scale cloud covers could be ubiquitous in strongly irradiated exoplanets, but might be more patchy in low-irradiated or isolated objects such as brown dwarfs and directly imaged exoplanets.