Infrared lasers have an extensive range of applications in sensing, detection, communication, medicine, and other fields. The principle of directly pumping solid-state lasers is simple, and it can easily achieve high-power and high-efficiency laser output, which is one of the important means to obtain infrared lasers. Incorporating rare earth ions into the substrate as the gain medium for directly pumping solid-state lasers can alter their optical performance and further enhance the performance of the laser. Lasers based on rare earth ion doping have a small volume, high conversion efficiency, good beam quality, wide tuning range, and multiple operating modes. Therefore, the proportion of rare earth ions doped as the gain medium for activating ions is currently very large. In this review, Ho3+, Tm3+, and Er3+ are selected as the representative rare earth ions, and their optical properties, such as luminous power and fluorescence lifetime, when doped in different substrates, such as crystals, ceramics, and fibers, are introduced, respectively, to illustrate their feasibility as infrared laser gain media. In addition, we show the different optical properties when doped with two ions, three ions, and four ions, demonstrating their great potential as infrared laser gain media.