The vibration signal of a high arch dam flow discharge structure under the excitation of water flow is difficult to extract because the vibration signals are weak, low-frequency, and easily submerged by noise. To address the problem of modal confusion during the modal identification of dam structures, a combined decomposition method based on the combination of complete ensemble empirical mode decomposition with adaptive noise and a masking signal processing method is proposed based on the analysis of modal confusion mechanism. Based on the measured vibration response signals of the dam body's measuring points, the bandwidth signals that have modal confusion are mixed with the high-frequency part of the original signal by adding a masking signal. After decomposition distinguishes different frequency parts of the signal, better inhibition of modal confusion is obtained. The improved Hilbert–Huang transform and random decrement technique methods are used to identify the modal parameters of the high arch dam under the excitation of flow discharge. The analysis of engineering example shows that this method can effectively inhibit modal confusion, avoid loss of modal information, and accurately identify the modal parameters of the structure. It can be applied to the modal identification process of engineering structures in other related fields.