Diamond operated as a cryogenic calorimeter is an excellent target for direct detection of low-mass dark matter candidates. Following the realization of the first low-threshold cryogenic detector that uses diamond as absorber for astroparticle physics applications, we now present the resulting exclusion limits on the elastic spin-independent interaction cross-section of dark matter with diamond. We measured two 0.175 g CVD (Chemical Vapor Deposition) diamond samples, each instrumented with a Transition Edge Sensor made of Tungsten (W-TES). Thanks to the energy threshold of just 16.8 eV of one of the two detectors, we set exclusion limits on the elastic spin-independent interaction of dark matter particles with carbon nuclei down to dark matter masses as low as 0.122 GeV/c2\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$^2$$\\end{document}. This work shows the scientific potential of cryogenic detectors made from diamond and lays the foundation for the use of this material as target for direct detection dark matter experiments.