This article focuses on the stability of error bounds, both local and global, for semi-infinite convex constraint systems in Banach spaces. We present primal characterizations of the stability of these error bounds under small perturbations. These characterizations are expressed through the directional derivatives of the functions that define the systems. It is shown that ensuring stability of the error bounds is closely tied to verifying that the optimal values of several minimax problems, formulated using the directional derivatives, remain outside a certain neighbourhood of zero. Furthermore, this stability condition only requires that all component functions of the system share the same linear perturbation. When applied to the sensitivity analysis of Hoffman's constants for semi-infinite linear systems, these stability results yield primal criteria that guarantee the uniform boundedness of Hoffman's constants under perturbations in the problem data.