In this study, 2-amino-terephthalic acid was designed for construction of MOFs, and carbon quantum dots (CQDs) were designed to activate the Metal Organic Frameworks (MOFs) to be used as fluorescent sensor for selectively detection for Fe3+ and Cu2+, enhancing the fluorescence to an “on” state by the hybrid with CQDs in the MOFs as activating agent; quenching the fluorescence to an “off” state by removal of CQDs through the metal competitive coordination substitution strategy. Using this strategy, the prepared CQDs/MOFs material can be used as a fluorescence “on-off” probe for highly sensitive detection of Fe3+ and Cu2+ in aqueous solution. It was found that the LOD of CQDs/MOFs for sensing Fe3+ and Cu2+ were 2.3 ppb and 1.3 ppb, respectively. It was proved that the CQDs/MOFs has little cytotoxicity to 4T-1 cells, indicating the potential application in the detection for Fe3+ and Cu2+ ions in cells.