The strategy of the iterative PIV algorithm is discussed with the aim of investigating transitional wall-bounded flows. It is proposed to start the algorithm with a coarse prediction of the large-scale tangential velocity gradient to eliminate its negative effect. A window correlation with image deformation technique is then iteratively conducted to capture the fluctuating flow structures. Synthetic particles are created based on the DNS result of a typical hypersonic transitional boundary layers to evaluate the performance of the algorithm. Effects of several parameters, including the mean tangential displacement gradient, particle diameter, particle density and background noise on the level on the evaluation, are systematically investigated. Four methods to initially predict the near-wall flow are compared, including the directly-used window correlation with image deformation (WC) (Scarano 2002 Meas. Sci. Technol. 13 R1–19), the WC method with static particles added in the solid region (Zhu et al 2013 Meas. Sci. Technol. 24 125302), the image gradiometry method (Nguyen et al 2009 Proc. 6th Int. Symp. on Turbulence and Shear Flow Phenomena (Seoul, Korea, 2009)) and the newly-proposed joint-translation-shear correlation method method. The results show that, compared with the other three approaches, the performance of the new method is greatly enhanced for evaluating the dynamical flow properties, even if the displacement gradient is very high. Finally, experimental data on artificially disturbed boundary layers from a low-speed and a high-Mach-number wind tunnel are used to evaluate the new method, which is able to evaluate the mean and instantaneous velocity fields in both cases.