Dear Editor, Infrared imaging, generally, of low quality, plays an important role in security surveillance and target detection. In this letter, we improve the quality of infrared images by combining both hardware and software. To this end, an infrared light field imaging enhancement system is built for the first time, including a 3× 3 infrared light field imaging device, a large-scale infrared light field dataset (IRLF-WHU), and a progressive fusion network for infrared image enhancement (IR-PFNet). The proposed algorithm leverages rich angular views among the infrared light field image to explore and fuse auxiliary information for infrared image enhancement. Given an infrared light field image, multiple views are first divided into four groups according to the angle and each group contains parallax shifts along the same direction. As strong spatial-angular correlations are existing in each group, we customize a progressive pyramid deformable fusion (PPDF) module for intra-group fusion without explicit alignment. In the PPDF module, the deformation and parallax are modeled in a progressive pyramid way. To integrate the supplementary information from all directions, we further propose a recurrent attention fusion (RAF) module, which constructs attention fusion block to learn the residual recurrently and provides several intermediate results for multi-supervision. Experiments on our proposed IRLF-WHU dataset demonstrate that IR-PFNet can achieve state-of-the-art performance on different degradations, yielding satisfying results. The dataset is available at: https://github.com/wxywhu/IRLF-WHU, and the code is available at: https://github.com/wxywhu/IR-PFNet.