During the long-term waterflooding (LTWF) in oil reservoirs, the formation is subject to permeability reduction as clay release and fine migration. At present, the mechanisms of permeability impairment in both macroscopic and microscopic pore structures in ultralow permeability reservoirs under LTWF are unclear. This statement epitomizes the main objective of this work: to understand how long-term waterflood changes porous structures and thus compromises permeability. The standard core flow experiments in conjunction with a couple of tests consisting of online nuclear magnetic resonance (NMR), high-pressure mercury intrusive penetration (HPMIP), X-ray diffraction (XRD), and scanning electron microscope (SEM) were performed to determine the mineral compositions, macrophysical properties, and micropore structures of two kinds of cores with different natures of pore distribution (i.e., unimodal and bimodal) before and after LTWF in Yan Chang field China. Results showed that the permeability decreased while the porosity increased after the LTWF. With respect to the pore size distribution, the small pores (SPs) decreased and the large pores (LPs) increased for both cores. For the unimodal core, the distribution curve shifted upwards with little change in the radius of the connected pores. For the bimodal core, the curve shifted to the right with an increasing radius of connected pores. With respect to the characteristic parameters, the average pore radius, median pore radius, structural coefficient, and tortuosity increased, while the relative sorting coefficient decreased. The relative changes of the parameters for the unimodal core were much smaller than those for the bimodal core. With respect to the clays, chlorite accounted for a majority proportion of the clays, and its content increased after LTWF. According to these changes, the mechanism of LTWF at different stages was interpreted. At the early stages, the blockage of the released clays occurred in SPs. Some of the middle pores (MPs) and LPs became larger due to the release and some of them became smaller due to the accumulation. At the middle stage, the blockage of SPs weakened. Some flow channels formed by MPs and LPs became dominant flow channels gradually. The effluxes of particles occurred, resulting in a significant increase in porosity. At the late stage, the stable flow channels have formed. The higher response of the bimodal core to LTWF could be attributed to its higher content of chlorite, which was more likely to accumulate. This study clarifies the mechanism of fine-migration-induced formation damage in microscopic pore structures and the migration pattern of clay minerals in ultralow permeability reservoirs. The work provides potential guidance for optimizing waterflood strategies in ultralow permeability reservoirs.