Researching the mechanical characteristics of concrete subjected to the freeze–thaw cycle is crucial for building engineering in cold climates. As a result, uniaxial compression tests were performed on concrete samples exposed to various freeze–thaw (F–T) cycles, and the measurements of the pore size distribution, porosity, and P-wave velocity of the saturated concrete samples were obtained, both before and after being exposed to the F–T cycles. Concrete’s F–T damage mechanism and damage evolution model were thoroughly examined. Using rock structure and moisture analysis test equipment to observe the T2 spectrum, the results showed that the F–T cycles can cause the internal structure of the samples to deteriorate. Porosity and F–T cycles have a positive correlation, although P-wave velocity has a negative correlation with the F–T cycles. As the F–T cycles increased, the specimens’ peak strength and elastic modulus steadily declined, while the peak strain clearly exhibited an increasing trend. A microscopic F–T damage model that takes into account the pore size distribution was developed, based on the relative changes in the pore structure distribution (PSD), before and after the F–T cycles. The concrete sample damage evolution law under various F–T cycles was examined using the following metrics: total energy, pore size distribution, static and dynamic elastic moduli, porosity, and P-wave velocity. Uniaxial compressive strength (UCS) and peak strain tests were used to evaluate the accuracy of the pore size distribution damage model, as well as that of five other widely used damage models.