Photorefractive materials are capable of reversibly changing their index of refraction upon illumination. That property allows them to dynamically record holograms, which is a key function for developing an updateable holographic 3D display. The transition from inorganic photorefractive crystals to organic polymers meant that large display screens could be made. However, one essential figure of merit that needed to be worked out first was the sensitivity of the material that enables to record bright images in a short amount of time. In this review article, we describe how polymer engineering was able to overcome the problem of the material sensitivity. We highlight the importance of understanding the energy levels of the different species in order to optimize the efficiency and recording speed. We then discuss different photorefractive compounds and the reason for their particular figures of merit. Finally, we consider the technical choices taken to obtain an updateable 3D display using photorefractive polymer. By leveraging the unique properties of this holographic recording material, full color holograms were demonstrated, as well as refreshing rate of 100 hogels/second.