Harmonic Balance is one of the most popular methods for computing periodic solutions of nonlinear dynamical systems. In this work, we address two of its major shortcomings: First, we investigate to what extent the computational burden of stability analysis can be reduced by consistent use of Chebyshev polynomials. Second, we address the problem of a rigorous error bound, which, to the authors’ knowledge, has been ignored in all engineering applications so far. Here, we rely on Urabe’s error bound and, again, use Chebyshev polynomials for the computationally involved operations. We use the error estimate to automatically adjust the harmonic truncation order during numerical continuation, and confront the algorithm with a state-of-the-art adaptive Harmonic Balance implementation. Further, we rigorously prove, for the first time, the existence of some isolated periodic solutions of the forced-damped Duffing oscillator with softening characteristic. We find that the effort for obtaining a rigorous error bound, in its present form, may be too high to be useful for many engineering problems. Based on the results obtained for a sequence of numerical examples, we conclude that Chebyshev-based stability analysis indeed permits a substantial speedup. Like Harmonic Balance itself, however, this method becomes inefficient when an extremely high truncation order is needed as, e.g., in the presence of (sharply regularized) discontinuities.