The aim of this paper is to use an analytic method and the properties of the classical Gauss sums to research the computational problem of one kind of character sum of polynomials modulo an odd prime p and obtain several meaningful third- and fourth-order linear recurrence formulae for them.