This study is aimed at investigating the infrared radiation (IR) response and energy dissipation characteristics of sandstone crushing under different impact velocities. To achieve this aim, first, we conducted IR experiments involving sandstone crushing based on drop-hammer impact. Next, the variations in infrared radiation temperature (IRT), infrared thermal imaging, and energy of sandstone crushing under different impact velocities were studied, and the characteristics of sandstone crushing and particle size distribution were also explored. In addition, the correlation between IR parameters and mean particle size of sandstone crushing was determined, and the correlation between IR parameters and fractal dimension was evaluated. The following results were obtained: the peak increments of the maximum infrared radiation temperature (MIRT) and average infrared radiation temperature (AIRT) increase with the increase of impact velocity, and the time of the maximum infrared radiation peak temperature lags that of average infrared radiation peak temperature (AIRPT). Under the same colorimetric scale, the IRT of sandstone surface is higher when the impact velocity is larger, and the differentiation characteristics of infrared thermal images are greater; on the contrary, the IRT of sandstone surface is lower when the impact velocity is lower, and the differentiation between infrared thermal images is more difficult. The peak increments of MIRT and AIRT increase with impact energy, absorption energy, impact energy density, and absorption energy density in cubic function. The mean particle size of sandstone fragments decreases linearly with the increase of impact velocity, and the fractal dimensions increase with impact energy density and absorption energy density in a negative exponential manner. The peak increments of MIRT, AIRT, and infrared radiation energy (IRE) have a good correlation with the mean particle size and fractal dimension of sandstone fragments, which can characterize the degree of sandstone crushing and particle size distribution under impact load.