γ-Aminobutyric acid (GABA) is an inhibitory neurotransmitter with important physiological functions such as sleep assistance and anti-depression. In this study, we developed a fermentation process for the high-efficiency production of GABA by Lactobacillus brevis (Lb. brevis) CE701. First, xylose was found as the optimal carbon source that could improve the GABA production and OD600 in shake flasks to 40.35 g/L and 8.64, respectively, which were 1.78-fold and 1.67-fold of the glucose. Subsequently, the analysis of the carbon source metabolic pathway indicated that xylose activated the expression of the xyl operon, and xylose metabolism produced more ATP and organic acids than glucose, which significantly promoted the growth and GABA production of Lb. brevis CE701. Then, an efficient GABA fermentation process was developed by optimizing the medium components using response surface methodology. Finally, the production of GABA reached 176.04 g/L in a 5 L fermenter, which was 336% higher than that in a shake flask. This work enables the efficient synthesis of GABA using xylose, which will provide guidance for the industrial production of GABA.