Water pollution is currently a major problem worldwide. Given its detrimental effects on health, Hg(II) is considered an extremely hazardous heavy metal contaminant, even at low doses. Heterocyclic compounds have been thoroughly evaluated as the chemosensor agents for Hg(II) detection. However, they suffer from poor sensitivity. In this study, we prepared two fluorescence chemosensor agents from vanillin via several steps, i.e., etherification, Claisen–Schmidt, and cyclocondensation reactions to yield N-phenyl- and N-pyridine-pyrazoline compounds. Products characterization was accomplished via spectroscopic techniques. Chalcone, N-phenyl-, and N-pyridine-pyrazoline derivatives were successfully obtained at 87.04%, 90.91%, and 91.73% yields, with limits of detection of 156,840, 65.810, and 161.011 nM, respectively. These results show that the conversion of chalcone to pyrazoline structure improved the sensitivity for Hg(II) detection at the nanomolar level, which is 2384 times lower than that for chalcone. Further spectroscopic investigations through Job's plot, Fourier-transform infrared spectroscopy, and proton-nuclear magnetic resonance analyses revealed that Hg(II) ions were chelated with two nitrogen atoms of pyrazoline. Thus, this phenomenon can explain the considerable sensitivity enhancement for Hg(II) detection. N-Phenyl-pyrazoline is the more sensitive chemosensor to Hg(II) compared with N-pyridine-pyrazoline because the more nitrogen groups in the binding site, the less selective and sensitive the compound. This finding is also supported by the higher binding constant value of N-phenyl-pyrazoline (9.416 × 102 mol−1) than N-pyridine-pyrazoline (1.771 × 102 mol−1). Furthermore, N-phenyl-pyrazoline can be applied in the direct quantification of Hg(II) in tap and groundwater samples with a validity parameter in a range of 80.97%–103.54%.