Optical remote sensing is one of the most popular methods for estimating leaf nitrogen concentration (LNC). This nondestructive approach based on reflected intensity measurements has been applied to estimate the variation and distribution of nitrogen concentration in leaf and canopy levels in numerous studies. However, both intensity and polarization are necessary to describe the optical properties of light reflected from leaves and to estimate LNC estimation. In this study, based on the Stokes parameters, the total reflectance, the polarized reflectance, and the nonpolarized reflectance factors (NpRFs) were simultaneously obtained through polarimetric hyperspectral measurements under varied source-viewing geometries in both laboratory and field conditions. Several published hyperspectral indices based on the NpRF showed much better LNC estimation accuracy than those using the total reflectance factor. A clear improvement was found in the viewing directions dominated by specular reflection. Thus, using multi-angular polarimetric hyperspectral measurements not only improves the accuracy of hyperspectral indices on LNC estimation using the NpRF, but also enables the hyperspectral indices to be effective for a wide range of viewing angles. Moreover, polarimetric measurements deepen the understanding of the optical properties of light reflected from leaves. These results indicate that the combination of multiangular, polarimetric, and hyperspectral measurements may play a key role in the estimation of LNC.