This work is devoted to the study of a higher order numerical scheme for the critical generalized Korteweg-de Vries equation (GKdV with p=4) in a bounded domain. The KdV equation and some of its generalizations as the GKdV type equations appear in Physics, for example in the study of waves on shallow water. Based on the analysis of stability of the first order scheme introduced by Pazoto et al. (Numer. Math. 116:317–356, 2010), we add a vanishing numerical viscosity term to a semi-discrete scheme so as to preserve similar properties of stability, and thus able to prove the convergence in L4-strong. The semi-discretization of the spatial structure via central finite difference method yields a stiff system of ODE. Hence, for the temporal discretization, we resort to the two-stage implicit Runge-Kutta scheme of the Gauss-Legendre type. The resulting system is unconditionally stable and possesses favorable nonlinear properties. On the other hand, despite the formation of blow up for the critical case of GKdV, it is known that a localized damping term added to the GKdV-4 equation leads to the exponential decay of the energy for small enough initial conditions, which is interesting from the standpoint of the Control Theory. Then, combining the result of convergence in L4-strong with discrete multipliers and a contradiction argument, we show that the presence of the vanishing numerical viscosity term allows the uniform (with respect to the mesh size) exponential decay of the total energy associated to the semi-discrete scheme of higher-order in space with the localized damping term. Numerical experiments are provided to illustrate the performance of the method and to confirm the theoretical results.