Upconversion luminescence is an anti-Stokes' emission process by converting long wavelength near-infrared (NIR, 980 nm) irradiation into shorter wavelength visible light emission, which demonstrates many advantages including no autofluorescence, low damage to samples, no photobleaching, and high sensitivity. Based on the Rhodamine B thiolactone (RBT) functionalized NaYF(4):15%Yb(3+),5%Er(3+) (UCNPs@RBT) nanocomposites, an ultrasensitive, selective, and rapid upconversion luminescence resonance energy transfer (UC-LRET) sensor has been developed for the detection of mercury ions (Hg(2+)) in water. Upconverting luminescence resonance energy transfer from the UCNPs to the RBT derivates occurs after the addition of Hg(2+) ions into the UCNPs@RBT colloidal solution. This UC-LRET recognition of Hg(2+) can be finished within 1 min and other cations have no influence on the detection of mercury ions. This newly developed sensor demonstrates high selectivity toward the mercury ions and enables ultrasensitive and rapid detection of mercury ions in water in the range of 5 nM to 10 μM with a 3σ limit of detection of 3.7 nM. This sensor can be used for a naked-eye detection of Hg(2+) ions via its green upconverting luminescence response under the infrared excitation (980 nm) with the merit of no autofluorescence interference and good photostability. In addition, by dipping the hydrogel of UCNPs@RBT nanocomposites onto the filter paper, a highly selective and convenient luminescent paper sensor for Hg(2+) ions was also developed.