In order to investigate the systematics of the loop expansion in high temperature gauge theories beyond the leading order hard thermal loop (HTL) approximation, we calculate the two-loop electron proper self-energy in high temperature QED. The two-loop bubble diagram contains a linear infrared divergence. Even if regulated with a non-zero photon mass M of order of the Debye mass, this infrared sensitivity implies that the two-loop self-energy contributes terms to the fermion dispersion relation that are comparable to or even larger than the next-to-leading-order (NLO) contributions at one-loop. Additional evidence for the necessity of a systematic restructuring of the loop expansion comes from the explicit gauge parameter dependence of the fermion damping rate at both one and two-loops. The leading terms in the high temperature expansion of the two-loop self-energy for all topologies arise from an explicit hard-soft factorization pattern, in which one of the loop integrals is hard, nested inside a second loop integral which is soft. There are no hard-hard contributions to the two-loop Sigma at leading order at high T. Provided the same factorization pattern holds for arbitrary ell loops, the NLO high temperature contributions to the electron self-energy come from ell-1 hard loops factorized with one soft loop integral. This hard-soft pattern is both a necessary condition for the resummation over ell to coincide with the one-loop self-energy calculated with HTL dressed propagators and vertices, and to yield the complete NLO correction to the self-energy at scales ~eT, which is both infrared finite and gauge invariant. We employ spectral representations and the Gaudin method for evaluating finite temperature Matsubara sums, which facilitates the analysis of multi-loop diagrams at high T.