Bacterial pathogens such as Yersinia pestis, Francisella tularensis, and Bacillus anthracis are classified into the highest rank of potential bioterrorism agents. Colorimetric lateral flow assay (LFA) strips are commercially available but these conventional strips have drawbacks in terms of low sensitivity and limit of quantitative analysis. Therefore, there is an urgent need for a new sensing platform to detect these pathogens in the early contamination stage. In this study, a novel surface-enhanced Raman scattering (SERS)-based LFA strip was developed for sensitive detection of bacterial pathogens. Target-specific SERS nanotags (Raman reporter-labeled gold nanoparticles) were used as an alternative to the gold nanoparticles in conventional LFA strips. Using these SERS nanotags the presence of bacteria could be identified through a simple color change in the test line. Additionally, highly sensitive and accurate quantitative analysis could be performed by monitoring the characteristic Raman peak intensity of SERS nanotags that were captured in the test line. This highly sensitive method required a short assay time (15 min) and a tiny volume of pathogen sample (40 μL). We believe that the proposed SERS-based LFA technique has great potential as a valuable tool in the early detection of specific bacterial pathogens in the field due to its excellent analytical sensitivity.