Conventional light field-based computer-generated holograms, such as holographic stereograms, can provide nature parallax and accurate occlusion relation but are restricted by the inherent tradeoff concerning the angular and spatial resolution of the reconstructed optical field. In contrast, Wigner distribution function based light field generated holograms, also called non-hogel-based holograms, release this limitation by using basic ray-to-wave transformation frameworks. However, at present, this transformation framework is accurate only for generating holograms from orthogonal light field images of 3D scenes with a limited depth range and size. Thus, the radiance field holography method is proposed in this paper to generate high-quality 3D holograms for arbitrary scenes that can achieve high-resolution reconstruction within a deep depth range. In our method, a proposed radiance field is constructed from the perspective light field, which consists of multiple reference planes to resample and redistribute the light rays to represent the diffracted optical field of the 3D objects according to our developed sampling criterion. The complex optical field of the 3D object can be synthesized from these reference planes based on inverse Wigner transform and Fresnel propagation. As a result, both the numerical simulations and the optical experiments are implemented, and it is demonstrated that the high-quality 3D reconstruction of the synthesized hologram is achieved, which reflects the highest reconstruction resolution among the existing methods, the large depth range with accurate refocus and defocus, and the natural view-dependent effects.