Fourier mode coupling theory was first employed in the spectral analysis of several nonuniform fiber Bragg grating (FBG)-based acousto-optic modulators (NU-FBG-AOMs) with the effects of Gaussian-apodization (GA), phase shift (PS), and linear chirp (LC). Because of the accuracy and simplicity of the algorithm applied in this model, the modulation performances of these modulators can be acquired effectively and efficiently. Based on the model, the reflected spectra of these modulators were simulated under various acoustic frequencies and acoustically induced strains. The simulation results of the GA-FBG-AOM and PS-FBG-AOM showed that the wavelength spacing between the primary reflection peak and the secondary reflection peak is proportional to the acoustic frequency, and the reflectivity of reflection peaks depends on the acoustically induced strains. But for the LC-FBG-AOM, the wavelength spacing between the neighboring reflection peaks increased linearly and inversely with the acoustic frequency, and the extinction ratio of each peak relates to the acoustically induced strain. These numerical analysis results, which were effectively used in the designs and fabrications of these NU-FBG-AOMs, can broaden the AOM-based application scope and shed light on the performance optimization of optical wavelength-division multiplex system.