This paper presents an optimization scheme to better design phononic crystals. A locally resonant phononic crystal (LRPC) structure called square spiral with circle inside is employed to verify the performance of the present scheme. Four geometric parameters, i.e., the side length of square scatterer, the length of each elastic beam, the thickness of elastic beams, and the radii of inner circles, are considered to obtain the corresponding influences on band gaps (BGs) using finite element method (FEM). According to the significant influences of the late two key parameters, a 2-factor (the radii of inner circles, and the thickness of elastic beams) and 7-level experiment is designed to obtain optimal BGs with better low-frequency broadband properties. By 29 times calculations using FEM for the different combination of levels, three relationships between the 2-factor and the first BG’s starting frequency, the first BG’s bandwidth, and the second BG’s bandwidth, are obtained and severed as inputs to the software of response surface methodology (RSM). The closed-form expressions of the three relationships are finally obtained to construct optimization models and result in the optimal band gaps (BGs) between 190–300 Hz and 500–600 Hz . It is expected that the present optimization scheme can be extended to material design of phononic/photonic structures in a reasonable way.