We have found two sets of Feynman rules for non-Abelian gauge theories in which ghosts do not appear. These Feynman rules are derived from the canonical formalism which has the advantage (over the path-integral formalism) of the propagators having explicit boundary conditions. These boundary conditions are not necessarily those of Feynman. The new Feynman rules are, however, equivalent to the conventional ones in the Feynman gauge. This demonstrates that ghosts can be eliminated at the cost of Feynman's boundary conditions.