No-tillage (NT) and the introduction of cover crops, owing to their positive effects on soil organic carbon (SOC) sequestration and crop yields, are potential agricultural practices that both support food security under the new realities of climate change and alleviate greenhouse gas (GHG) emissions. However, the effects of the combination of long-term NT systems and cover crops on non-carbon dioxide (CO2) emissions and SOC sequestration have not been adequately documented, particularly in East Asia. We conducted a split-plot field experiment involving two tillage systems [NT and moldboard plowing (MP)] and three cover crops, namely, fallow (FA), hairy vetch (HV), and rye (RY). NT had slightly higher soybean yield than MP, although tillage methods and cover crop treatments had no significant effects on soybean yield. Cover crop treatments rather than tillage methods significantly affected methane (CH4) emissions; under FA and RY treatments, we observed CH4 uptakes, whereas under HV, we observed CH4 emissions. In contrast, rather than cover crop treatments, tillage methods affected nitrous oxide (N2O) emissions. Higher WFPS and soil bulk density under NT resulted in significantly higher annual N2O emissions than those under MP. However, under NT, the annual SOC sequestration rate significantly increased compared with that under MP, the global warming potential (GWP) caused by CH4 and N2O emissions was fully offset by net CO2 retention under NT. Additionally, treatment under NT reduced net GWP and yield-scaled GWP to a significantly greater degree than did treatment under MP. Treatments under NT with RY cover crop had the lowest net GWP (−2324 kg CO2 equivalent ha−1 year−1) and yield-scaled GWP (−1037 kg CO2 equivalent Mg−1 soybean yield). These findings suggest that treatments under NT with cover crop systems—especially RY cover crop—in the long-term organic soybean field maintains sustainable crop production and reduces net GWP and yield-scaled GWP, which will be an effective climate-smart agriculture practice in the humid, subtropical regions prevailing in Kanto, Japan.