ABSTRACT With an X-ray stacking analysis of $\simeq 12\, 000$ Lyman-break galaxies (LBGs) using the Chandra Legacy Survey image, we investigate average supermassive black hole (SMBH) accretion properties of star-forming galaxies (SFGs) at 4 ≲ z ≲ 7. Although no X-ray signal is detected in any stacked image, we obtain strong 3σ upper limits for the average black hole accretion rate (BHAR) as a function of star formation rate (SFR). At z ∼ 4 (5) where the stacked image is deeper, the 3σ BHAR upper limits per SFR are ∼1.5 (1.0) dex lower than the local black hole-to-stellar mass ratio, indicating that the SMBHs of SFGs in the inactive (BHAR $\lesssim 1 \, \mathrm{M}_\odot$ yr−1) phase are growing much more slowly than expected from simultaneous evolution. We obtain a similar result for BHAR per dark halo accretion rate. QSOs from the literature are found to have ∼1 dex higher SFRs and ≳ 2 dex higher BHARs than LBGs with the same dark halo mass. We also make a similar comparison for dusty starburst galaxies and quiescent galaxies from the literature. A duty-cycle corrected analysis shows that for a given dark halo, the SMBH mass increase in the QSO phase dominates over that in the much longer inactive phase. Finally, a comparison with the TNG300, TNG100, SIMBA100, and EAGLE100 simulations finds that they overshoot our BHAR upper limits by ≲ 1.5 dex, possibly implying that simulated SMBHs are too massive.