Among the large number of synthetic aperture radar (SAR) image despeckling approaches existing in literature, nonlocal (NL) filters have received a desirable boost. However, often NL approaches define the similarity criterion based on model assumptions, such as a fully developed speckle model. This assumption may not be verified in high-resolution images of urban environments. To address this issue, a standalone model-free despeckling framework is proposed in this article. The presented approach provides a generic framework for denoising a variety of SAR products, from a single-intensity/amplitude image to polarimetric and interferometric SAR data. In particular, the method is based on the empirical distributional similarity between the patch containing the pixel to be recovered and the patch containing a similar candidate pixel. To decide whether the patches follow a similar distribution, the Kolmogorov–Smirnov test is adapted. Finally, the restoration process aggregates the selected similar pixels based on their relative importance derived from their distribution similarities. To mitigate the blurring effect and preserve the resolution, the inhomogeneity of the ratio image is used to perform the bias reduction step. The designed generic despeckling filter was tested on different products of SAR data. The results show that the method proves to be an unbiased restoration approach and is able to preserve structures and textures. It works fully automatically and efficiently with single and multilook (and multichannel) images.