A flexible structure immersed in a flowing fluid may exhibit vortex-induced vibration (VIV) dominated by either a fundamental or a higher-order mode depending on the flow velocity. Multimode coupled VIV may also exist if the structure presents closely spaced modes or significant nonlinear characteristics. While the traditional linear tuned mass damper designed for a single mode is ineffective for VIV control involving multiple modes, the nonlinear energy sink (NES) may work as a promising alternative due to its broadband vibration energy absorption capability. In this study, the governing equation of the coupled fluid-structure-NES system is first established based on a wake-oscillator model of the vortex-induced force. Then, a six-span continuous beam with an NES attachment is used as a case study to investigate the NES-based VIV control of a flexible structure with multiple degrees of freedom. The optimal NES is determined through a systematic parametric analysis considering both control effectiveness and robustness. Numerical results suggest that an NES designed for VIV control of the fundamental mode can also effectively control the VIVs of higher-order modes and that the controlling performance is robust to uncertainties of the stiffness and damping properties of both the primary structure and NES.