Respiratory-gated positron emission tomography (PET)/computed tomography protocols reduce lesion smearing and improve lesion detection through a synchronized acquisition of emission data. However, an objective assessment of image quality of the improvement gained from respiratory-gated PET is mainly limited to a three-dimensional (3D) approach. This work proposes a 4D numerical observer that incorporates both spatial and temporal informations for detection tasks in pulmonary oncology. The authors propose a 4D numerical observer constructed with a 3D channelized Hotelling observer for the spatial domain followed by a Hotelling observer for the temporal domain. Realistic (18)F-fluorodeoxyglucose activity distributions were simulated using a 4D extended cardiac torso anthropomorphic phantom including 12 spherical lesions at different anatomical locations (lower, upper, anterior, and posterior) within the lungs. Simulated data based on Monte Carlo simulation were obtained using geant4 application for tomographic emission (GATE). Fifty noise realizations of six respiratory-gated PET frames were simulated by GATE using a model of the Siemens Biograph mMR scanner geometry. PET sinograms of the thorax background and pulmonary lesions that were simulated separately were merged to generate different conditions of the lesions to the background (e.g., lesion contrast and motion). A conventional ordered subset expectation maximization (OSEM) reconstruction (5 iterations and 6 subsets) was used to obtain: (1) gated, (2) nongated, and (3) motion-corrected image volumes (a total of 3200 subimage volumes: 2400 gated, 400 nongated, and 400 motion-corrected). Lesion-detection signal-to-noise ratios (SNRs) were measured in different lesion-to-background contrast levels (3.5, 8.0, 9.0, and 20.0), lesion diameters (10.0, 13.0, and 16.0 mm), and respiratory motion displacements (17.6-31.3 mm). The proposed 4D numerical observer applied on multiple-gated images was compared to the conventional 3D approach applied on the nongated and motion-corrected images. On average, the proposed 4D numerical observer improved the detection SNR by 48.6% (p < 0.005), whereas the 3D methods on motion-corrected images improved by 31.0% (p < 0.005) as compared to the nongated method. For all different conditions of the lesions, the relative SNR measurement (Gain = SNRObserved/SNRNongated) of the 4D method was significantly higher than one from the motion-corrected 3D method by 13.8% (p < 0.02), where Gain4D was 1.49 ± 0.21 and Gain3D was 1.31 ± 0.15. For the lesion with the highest amplitude of motion, the 4D numerical observer yielded the highest observer-performance improvement (176%). For the lesion undergoing the smallest motion amplitude, the 4D method provided superior lesion detectability compared with the 3D method, which provided a detection SNR close to the nongated method. The investigation on a structure of the 4D numerical observer showed that a Laguerre-Gaussian channel matrix with a volumetric 3D function yielded higher lesion-detection performance than one with a 2D-stack-channelized function, whereas a different kind of channels that have the ability to mimic the human visual system, i.e., difference-of-Gaussian, showed similar performance in detecting uniform and spherical lesions. The investigation of the detection performance when increasing noise levels yielded decreasing detection SNR by 27.6% and 41.5% for the nongated and gated methods, respectively. The investigation of lesion contrast and diameter showed that the proposed 4D observer preserved the linearity property of an optimal-linear observer while the motion was present. Furthermore, the investigation of the iteration and subset numbers of the OSEM algorithm demonstrated that these parameters had impact on the lesion detectability and the selection of the optimal parameters could provide the maximum lesion-detection performance. The proposed 4D numerical observer outperformed the other observers for the lesion-detection task in various lesion conditions and motions. The 4D numerical observer shows substantial improvement in lesion detectability over the 3D observer method. The proposed 4D approach could potentially provide a more reliable objective assessment of the impact of respiratory-gated PET improvement for lesion-detection tasks. On the other hand, the 4D approach may be used as an upper bound to investigate the performance of the motion correction method. In future work, the authors will validate the proposed 4D approach on clinical data for detection tasks in pulmonary oncology.