For the purpose of ultrasonic nondestructive testing of materials, holography in connection with digital reconstruction algorithms has been proposed as a modern tool to extract crack sizes from ultrasonic scattering data. Defining the typical holographic reconstruction algorithm as the application of the scalar Kirchhoff diffraction theory to backward wave propagation, we demonstrate its general incapability of reconstructing equivalent sources, and hence, geometries of scattering bodies. Only the special case of a planar measurement recording surface, that is to say, a hologram plane, and a planar crack with perfectly rigid boundary conditions parallel to the hologram plane and perpendicular to the incident field yields a nearly perfect correlation between crack size and reconstructed image; the reconstruction algorithm is then referred to as the Rayleigh-Sommerfeld formula; it therefore represents the optimal case matched to that special geometrical situation and, hence, may be interpreted as a quasi-matched spatial filter. Using integral equation theory and physical optics, we compute synthetic holographic data for a linear cracklike scatterer for both plane and spherical wave incidence, the latter case simulating a synthetic aperture impulse echo situation, thus illustrating how the Rayleigh-Sommerfeld algorithm or its Fresnel approximation increasingly fail for cracks inclined to the hologram plane and excited nonperpendicularly. Furthermore, we point out how the physical data recording process may additionally influence the reconstruction accuracy, and, finally, guidelines for a careful and serious application of these holographic reconstruction algorithms are given. The theoretical results are supported by measurements.