Cytotoxicity quantification of nanoparticles is commonly performed by biochemical assays to evaluate their biocompatibility and safety. We explored quantitative phase imaging (QPI) with digital holographic microscopy (DHM) as a time-resolved in vitro assay to quantify effects caused by three different types of organic nanoparticles in development for medical use. Label-free proliferation quantification of native cell populations facilitates cytotoxicity testing in biomedical nanotechnology. Therefore, DHM quantitative phase images from measurements on nanomaterial and control agent incubated cells were acquired over 24 h, from which the temporal course of the cellular dry mass was calculated within the observed field of view. The impact of LipImage™ 815 lipidots® nanoparticles, as well as empty and cabazitaxel-loaded poly(alkyl cyanoacrylate) nanoparticles on the dry mass development of four different cell lines (RAW 264.7, NIH-3T3, NRK-52E, and RLE-6TN), was observed vs. digitonin as cytotoxicity control and cells in culture medium. The acquired QPI data were compared to a colorimetric cell viability assay (WST-8) to explore the use of the DHM assay with standard biochemical analysis methods downstream. Our results show that QPI with DHM is highly suitable to identify harmful or low-toxic nanomaterials. The presented DHM assay can be implemented with commercial microscopes. The capability for imaging of native cells and the compatibility with common 96-well plates allows high-throughput systems and future embedding into existing experimental routines for in vitro cytotoxicity assessment.