Structural materials for laser-based inertial fusion energy (IFE) reactor concepts are expected to operate under pulsed irradiation conditions, with cycles consisting of microsecond-long neutron bursts followed by inter-pulse periods of up to one second in duration. During each laser shot, irradiation damage is introduced at dose rates that are up to six orders of magnitude higher than those in their magnetic fusion energy (MFE) counterparts. Under certain conditions, the inter-pulse periods may last an amount of time sufficient to anneal much of the damage introduced during each shot. This phenomenon is highly temperature dependent, with pulsed heating directly linked to the pulsed damage, large surface temperature spikes may also occur. As such, this intermittent mode of operation has the potential to lead to fundamental differences in how irradiation damage accumulates in structural reactor materials. However, damage to structural materials under IFE conditions has received comparatively much less attention than in MFE, as pulsed conditions add yet an extra dimension to the already extremely challenging problem of microstructural evolution under fusion neutron irradiation in structural materials. In this work we use the stochastic cluster dynamics (SCD) method to simulate the evolution with time of defect cluster concentrations under IFE conditions. We consider the Laser Inertial Fusion Energy (LIFE) reactor concept as the representative IFE design for our study, for which detailed spectral information is available, including gas transmutant production. We simulate several pulse frequencies and three different temperatures, and compare the results with continuous irradiation cases under identical average dose rates. The simulations are run in Fe-9Cr system as a model alloy for reduced-activation ferritic/martensitic (RAFM) steels, which are the leading structural material candidates for first-wall structures in MFE and IFE devices. We find that, in practically all scenarios, pulsed irradiation restricts the formation of helium-vacancy clusters relative to the levels seen under equivalent steady irradiation conditions. As well, although self-interstitial atom clusters do accumulate under pulsed operation, their number densities remain up to an order of magnitude lower than in continuous irradiation conditions. Based on the SCD results, we provide a temperature-pulse rate map to identify regions where pulsed irradiation may lead to larger defect accumulation than under continuous irradiation.