We successfully developed an atomic layer deposition (ALD) method for making Ag noble nanoparticles on cheap, commercial filter paper consisting of three-dimensional porous glass fibers and investigated the evolution of Ag nanostructures with some key process parameters. By tuning Ag particle sizes and controlling the cycle numbers of ALD deposited Ag films, we were able to obtain high-density isolated Ag nanoparticles with average sizes in 3–9 nm without the formation of agglomerates and continuous Ag films. We proved the presence of strong localized surface plasmon resonance peaks near a target wavelength of 632 nm. We further proved the presence of surface enhanced Raman scattering (SERS) signals on the Ag coated filter paper substrates using pyridine as the test analyte. Our results demonstrate that ALD is a very promising technique for a rational design of SERS substrates and, thus, has great potential for the fabrication of large-area, low-cost SERS substrates for future commercial applications, as compared to other existing techniques.