The fatigue performance of Laser Powder Bed Fusion manufactured 316L stainless steel is critical for cyclic loading applications. Additive manufacturing of 316L produces a characteristic microstructure consisting of grains with a fine cellular structure within. This microstructure shows a comparatively high fatigue strength combined with a high defect tolerance. The purpose of this paper is to numerically analyse the local response of this microstructure to cyclic loading and, in particular, the joint action between lack of fusion defects and the surrounding grain structure. The traditional incremental analysis method for polycrystalline alloys is computationally expensive. Therefore, a direct cyclic analysis combined with a phenomenological crystal plasticity model is used in this work to predict the stable cyclic response of the material, resulting in a much lower computational cost. Three statistically equivalent representative volume elements are generated. The considered crystal plasticity parameters are calibrated using experimental tensile stress–strain curves. The comparison with incremental analysis demonstrates the accuracy and efficiency of the proposed method. The shakedown limit, based on the maximum load that allows the convergent cyclic response, significantly depends on the local grain orientation around lack of fusion defects, revealing a reason for the large scatter in the fatigue strength.