AbstractCloud fraction (CF) significantly affects the short‐ and long‐wave radiation. Its realistic representation in general circulation models (GCMs) still poses great challenges in modeling the atmosphere. Here, we present a neural network‐based (NN‐based) diagnostic scheme that uses the grid‐mean temperature, pressure, liquid and ice water mixing ratios, and relative humidity to simulate the sub‐grid CF. The scheme, trained using CloudSat data with explicit consideration of grid sizes, realistically simulates the observed CF with a correlation coefficient >0.9 for liquid‐, mixed‐, and ice‐phase clouds. The scheme also captures the observed non‐monotonic relationship between CF and relative humidity and is computationally efficient, and robust for GCMs with a variety of horizontal and vertical resolutions. For illustrative purposes, we conducted comparative analyses of the 2006–2019 climatological‐mean cloud fractions among CloudSat, and simulations from the NN‐based scheme and the Xu‐Randall scheme (optimized the same way as the NN‐based scheme). The NN‐based scheme improves not only the spatial distribution of the total CF but also the cloud vertical structure. For example, the biases of too‐many high‐level clouds over the tropics and too‐many low‐level clouds over regions around 60°S and 60°N in the Xu‐Randall scheme are significantly reduced. These improvements are also found to be insensitive to the spatio‐temporal variability of large‐scale meteorology conditions, implying that the scheme can be used in different climate regimes.