Radiotherapy is widely used for the treatment of breast cancer. However, we have shown that ionizing radiation can provoke premature senescence in breast stromal cells. In particular, breast stromal fibroblasts can become senescent after irradiation both in vitro and in vivo and they express an inflammatory phenotype and an altered profile of extracellular matrix components, thus facilitating tumor progression. Adipose-derived stem cells (ASCs) represent another major component of the breast tissue stroma. They are multipotent cells and due to their ability to differentiate in multiple cell lineages they play an important role in tissue maintenance and repair in normal and pathologic conditions. Here, we investigated the characteristics of human breast ASCs that became senescent prematurely after their exposure to ionizing radiation. We found decreased expression levels of the specific mesenchymal cell surface markers CD105, CD73, CD44, and CD90. In parallel, we demonstrated a significantly reduced expression of transcription factors regulating osteogenic (i.e., RUNX2), adipogenic (i.e., PPARγ), and chondrogenic (i.e., SOX9) differentiation; this was followed by an analogous reduction in their differentiation capacity. Furthermore, they overexpress inflammatory markers, that is, IL-6, IL-8, and ICAM-1, and a catabolic phenotype, marked by the reduction of collagen type I and the increase of MMP-1 and MMP-13 expression. Finally, we detected changes in proteoglycan expression, for example, the upregulation of syndecan 1 and syndecan 4 and the downregulation of decorin. Notably, all these alterations, when observed in the breast stroma, represent poor prognostic factors for tumor development. In conclusion, we showed that ionizing radiation-mediated prematurely senescent human breast ASCs have a decreased differentiation potential and express specific changes adding to the formation of a permissive environment for tumor growth.