Hubble Space Telescope imaging shows that most star-forming galaxies at cosmic noon—the peak of cosmic star formation history—appear disk-dominated, leaving the origin of the dense cores in their quiescent descendants unclear. With the James Webb Space Telescope’s high-resolution imaging to 5 μm, we can now map the rest-frame near-infrared emission, a much closer proxy for stellar mass distribution, in these massive galaxies. We selected 70 star-forming galaxies with 10 < log(M) < 12 and 1.5 < z < 3 in the CEERS survey and compare their morphologies in the rest-frame optical to those in the rest-frame near-IR. While the bulk of these galaxies are disk-dominated in 1.5 μm (rest-frame optical) imaging, they appear more bulge-dominated at 4.4 μm (rest-frame near-infrared). Our analysis reveals that in massive star-forming galaxies at z ∼ 2, the radial surface brightness profiles steepen significantly, from a slope of ∼0.3 dex−1 at 1.5 μm to ∼1.4 dex−1 at 4.4 μm within radii <1 kpc. Additionally, we find their total flux contained within the central 1 kpc is approximately 7 times higher in F444W than in F150W. In rest-optical emission, a galaxy’s central surface density appears to be the strongest indicator of whether it is quenched or star-forming. Our most significant finding is that at redder wavelengths, the central surface density ratio between quiescent and star-forming galaxies dramatically decreases from ∼10 to ∼1. This suggests the high central densities associated with galaxy quenching are already in place during the star-forming phase, imposing new constraints on the transition from star formation to quiescence.