The dynamics of the lattice relaxation of holes and electron-hole ($e\ensuremath{-}h$) pairs generated by the band-gap excitation in alkali bromide crystals has been studied in femtosecond time regime at temperature range from 6 to 300 K. In KBr including electron-trapping impurities, where relaxation of holes takes place without interaction with electrons, it is found that the self-trapped hole in the form of a halogen-molecular ion, the ${V}_{K}$ center, is generated via a transient localized-hole center as a precursor. This transient hole center shows broad optical absorption bands in the visible region and has a lifetime of about 3 ps at 273 K. It is found that in pure KBr and RbBr, the relaxation processes of ($e\ensuremath{-}h$) pairs have two distinct stages. In the first stage, which terminates within 6 ps after excitation, the transient hole centers are generated as in doped specimens, and their interaction with electrons results in the fast formation of a Frenkel pair consisting of an $F$ center, a halogen vacancy trapping an electron, and an $H$ center, an interstitial halogen atom. Also, an intermediate state is formed in the first stage, and the state is ascribed to the self-trapped exciton (STE) with the on-center configuration on the basis of its spectroscopic features. The second stage of the relaxation, which lasts over 100 ps at low temperatures, comprises the off-center relaxation of the on-center STE formed in the first stage into Frenkel pairs and STE's with the off-center configuration. Relaxation pathways and characteristics including temperature dependence are discussed based on the experimental results and their analysis in terms of a rate-equation model.