With growing concerns about global warming, it is crucial to adopt agronomic practices that enhance rice yields from paddy fields while reducing greenhouse gas (GHG) emissions for sustainable agriculture. An optimal nitrogen (N) fertilization rate and planting density are vital to ensure high rice yields, minimize GHG emissions, and understand emission behavior for better field management. We hypothesized that optimizing N application rates and planting density to improve nitrogen use efficiency (NUE) in rice cultivation would reduce resource losses and GHG emissions. To test this hypothesis, we implemented five treatments with a rice straw return cultural system: two planting densities (16 hills m−2 (traditional density, D1) and 20 hills m−2 (25% higher density, D2)) and three N application rates (no N fertilizer (N0), 180 kg N ha−1 (N1), and 144 kg N ha−1 (N2)). The control treatment (CK) was traditional planting density with no N fertilizer. The four new cropping modes were N1D1, N1D2, N2D1, and N2D2. We investigated the effects of N application rates and planting density on rice grain yield, NUE, and GHG emissions in multiple rice-growing seasons. The N1D2 treatment exhibited the highest grain yield over the three years, with a value of 10,452 kg ha−1, representing an increase of 12.2% compared to CK. Moreover, N uptake in N1D2 was the highest, averaging 39.2% (p < 0.05) higher than CK, and 8.5%, 3.5%, and 2.8% (p < 0.05) higher than N1D1, N2D1 and N2D2, respectively. N2D2 exhibited the highest NUE, with a value of 58.99 kg kg−1, surpassing all other treatments over the three years. GHG emissions, global warming potential (GWP), and greenhouse gas intensity (GHGI) in N2D2 were lower than in N1D1, N1D2, and N2D1. Additionally, reducing N application (comparing N1D1 to N2D1) and increasing plant density (comparing N1D1 to N1D2) improved N agronomic efficiency (NAE) and N partial productivity (PFPN). The negative correlation between the NAE and PFPN with GWP and GHG emissions further supports the potential for optimized N management and denser planting density to reduce environmental impact. These findings have important implications for sustainable rice cultivation practices in Southwest China and similar agroecosystems, emphasizing the need for integrated nutrient management strategies to achieve food security and climate change mitigation goals.