The presence of nitrite in food products has generated significant public concern. A simple and rapid dual-mode surface-enhanced Raman spectroscopy (SERS)/colorimetric detection of nitrite is proposed based on a diazo reaction and multifunctional gold nanoparticle-doped covalent organic framework (Au@COF) composite. Under acidic conditions, the reaction between toluidine blue and nitrite yielded a colorless diazo salt, simultaneously attenuating its characteristic absorption peak and Raman signal. The multifunctional Au@COF materials enhanced the Raman signal and ensured good reproducibility. Additionally, the reaction rates improved, and the sensitivity was enhanced due to the excellent adsorption capacity of the COF. The proposed method demonstrated high sensitivity and excellent recovery rates for nitrite detection in food samples. This approach shows potential for precisely detecting nitrite content in real-world food samples by integrating the simplicity of colorimetric analysis with the enhanced sensitivity of SERS.