Cellular biomechanics plays critical roles in cancer metastasis and tumor progression. Existing studies on cancer cell biomechanics are mostly conducted in flat 2D conditions, where cells' behavior can differ considerably from those in 3D physiological environments. Despite great advances in developing 3D in vitro models, probing cellular elasticity in 3D conditions remains a major challenge for existing technologies. In this work, we utilize optical Brillouin microscopy to longitudinally acquire mechanical images of growing cancerous spheroids over the period of eight days. The dense mechanical mapping from Brillouin microscopy enables us to extract spatially resolved and temporally evolving mechanical features that were previously inaccessible. Using an established machine learning algorithm, we demonstrate that incorporating these extracted mechanical features significantly improves the classification accuracy of cancer cells, from 74% to 95%. Building on this finding, we have developed a deep learning pipeline capable of accurately differentiating cancerous spheroids from normal ones solely using Brillouin images, suggesting the mechanical features of cancer cells could potentially serve as a new biomarker in cancer classification and detection.