The theory of energy- and time-resolved two-photon photoemission (2PPE) spectra of metal surfaces is presented using density matrix formulation for a three-level system consisting of an initial occupied, intermediate unoccupied and final photoelectron states. A perturbation expansion method is employed to calculate the energy-resolved 2PPE spectrum for continuous light beams. We have obtained analytical expressions of the 2PPE spectrum corresponding to a step-by-step one-photon process through the intermediate state and a direct two-photon-ionization process via virtual transition. It is demonstrated that the intermediate state can also be populated via the nonresonant virtual process. This indicates an absolute importance of “pure dephasing” associated with the transition between the initial and intermediate states. Evolution of the 2PPE spectrum as a function of the pump photon energy is calculated to demonstrate the conditions under which the intrinsic linewidth (total dephasing time) can be deduced from the lineshape analysis. It is also found that the intensity ratio of the two peaks due to the initial and the intermediate states in 2PPE spectrum can be used to estimate the pure dephasing time. Transient behavior of the excited-state population following pulse excitation is calculated with a focus on how the ultrafast relaxation times of the excited states such as image-potential states of metal surfaces are deduced from the transient 2PPE response observed with a pulse laser with much longer duration. The time-resolved 2PPE spectra are calculated for varying detuning from the resonant excitation from the initial state to the intermediate state. Transient responses of the 2PPE signal due to direct ionization and step-by-step processes are also calculated to demonstrate that the nonresonant former process has an influence on the analysis of the cross-correlation trace of the intermediate state, by which the population relaxation time is estimated. Attempts are also made to apply the present theory to a recent time-resolved 2PPE study of the relaxation dynamics of the image-potential states as well as hot electrons in Cu(100) and Ag(100) surfaces.