Context. The evolution of the solar wind electron distribution function with heliocentric distance exhibits different features that are still unexplained, in particular, the fast decrease in the electron heat flux and the increase in the Strahl pitch angle width. Wave-particle interactions between electrons and whistler waves are often proposed to explain these phenomena. Aims. We aim to quantify the effect of whistler waves on suprathermal electrons as a function of heliocentric distance. Methods. We first performed a statistical analysis of whistler waves (occurrence and properties) observed by Solar Orbiter and Parker Solar Probe between 0.2 and 1 AU. The wave characteristics were then used to compute the diffusion coefficients for solar wind suprathermal electrons in the framework of quasi-linear theory. These coefficients were integrated to deduce the overall effect of whistler waves on electrons along their propagation. Results. About 110 000 whistler wave packets were detected and characterized in the plasma frame, including their direction of propagation with respect to the background magnetic field and their radial direction of propagation. Most waves are aligned with the magnetic field and only ∼0.5% of them have a propagation angle greater than 45°. Beyond 0.3 AU, it is almost exclusively quasi-parallel waves propagating anti-sunward (some of them are found sunward but are within switchbacks with a change of sign of the radial component of the background magnetic) that are observed. Thus, these waves are found to be Strahl-aligned and not counter-streaming. At 0.2 AU, we find both Strahl-aligned and counter-streaming quasi-parallel whistler waves. Conclusions. Beyond 0.3 AU, the integrated diffusion coefficients show that the observed waves are sufficient to explain the measured Strahl pitch angle evolution and effective in isotropizing the halo. Strahl diffusion is mainly attributed to whistler waves with a propagation angle of θ ∈ [15.45]°, although their origin has not yet been fully determined. Near 0.2 AU, counter-streaming whistler waves are able to diffuse the Strahl electrons more efficiently than the Strahl-aligned waves by two orders of magnitude.