The $\varepsilon$-pseudospectral abscissa and radius of an $n\times n$ matrix are, respectively, the maximal real part and the maximal modulus of points in its $\varepsilon$-pseudospectrum, defined using the spectral norm. Existing techniques compute these quantities accurately, but the cost is multiple singular value decompositions and eigenvalue decompositions of order n, making them impractical when n is large. We present new algorithms based on computing only the spectral abscissa or radius of a sequence of matrices, generating a sequence of lower bounds for the pseudospectral abscissa or radius. We characterize fixed points of the iterations, and we discuss conditions under which the sequence of lower bounds converges to local maximizers of the real part or modulus over the pseudospectrum, proving a locally linear rate of convergence for $\varepsilon$ sufficiently small. The convergence results depend on a perturbation theorem for the normalized eigenprojection of a matrix as well as a characterization of the group inverse (reduced resolvent) of a singular matrix defined by a rank-one perturbation. The total cost of the algorithms is typically only a constant times the cost of computing the spectral abscissa or radius, where the value of this constant usually increases with $\varepsilon$, and may be less than 10 in many practical cases of interest.