A method for surface-enhanced Raman spectroscopy (SERS) sensing of polycyclic aromatic hydrocarbons (PAHs) is reported. Fe3O4@PDA@Ag@GO is developed as the SERS substrate prepared by classical electrostatic attraction method based on the enrichment of organic compounds by graphene oxide (GO) and polydopamine (PDA) and the good separation and enrichment function of Fe3O4. The morphology and structure of the SERS substrate were represented by transmission electron microscopy (TEM), energy-dispersive spectroscopy (EDS), X-ray diffraction (XRD) and the UV–visible absorption spectrum (UV–vis spectra). The effect of different temperatures on SERS during synthesis was investigated, and it was found that the best effect was achieved when the synthesis temperature was 90 °C. The effect of each component of Fe3O4@PDA@Ag@GO nanocomposites on SERS was explored, and it was found that Ag NPs are of great significance to enhance the Raman signal based on the electromagnetic enhancement mechanism; apart from enriching the polycyclic aromatic hydrocarbons (PAHs) through π–π interaction, GO also generates strong chemical enhancement to the Raman signal, and PDA can prevent Ag from shedding and agglomeration. The existence of Fe3O4 is favored for the fast separation of substrate from the solutions, which greatly simplifies the detection procedure and facilitates the cycle use of the substrate. The experimental procedure is simplified, and the substrate is reused easily. Three kinds of PAHs (phenanthrene, pyrene and benzanthene) are employed as probe molecules to verify the performance of the composite SERS substrate. The results show that the limit of detection (LOD) of phenanthrene pyrene and benzanthene detected by Fe3O4@PDA@Ag@GO composite substrate are 10−8 g/L (5.6 × 10−11 mol/L), 10−7 g/L (4.9 × 10−10 mol/L) and 10−7 g/L (4.4 × 10−10 mol/L), respectively, which is much lower than that of ordinary Raman, and it is promising for its application in the enrichment detection of trace PAHs in the environment.