Polarization-sensitive optoelectronic detection has been achieved by an all-Si detector in the NIR range, based on plasmon hot electron generation/internal photoemission effect. An advanced architecture with a specially designed anisotropic metasurface was developed and structurally optimized for maximizing the internal quantum efficiency (IQE). Assisted by finite difference time domain (FDTD) simulations, the well-designed device exhibits a maximum optical absorption of 80% around 1.45 μm, corresponding to an optical discrimination ratio of 120. Optoelectronic measurements show the peak responsivity and detectivity of 51.2 mA/W and 8.05 × 1010 cm Hz1/2/W, respectively, at 1.45 μm. A high polarization photocurrent ratio of 35 nm is also achieved at 1.55 μm. Moreover, the optoelectronic response can be tuned by a back-gate bias. Last but not least, we built up a model for theoretically estimating the IQE, which provides instructive guidance for further enhancing the optoelectronic performance of hot electron detectors.