Multicomplex constraints often need to be considered in the optimal operation of cascade reservoirs with high-dimensional decision variables, making it difficult for traditional optimization methods and modern intelligent algorithms to solve such problems. Therefore, this study proposes a constraint handling method combining a penalty function nested DPSA-POA and an intelligent algorithm, to determine the optimal flood control operation of cascade reservoirs in the middle reaches of the Ganjiang River, a problem with decision variables of up to 2196 dimensions. The results indicate that the constraint handling method proposed in this paper can solve high-dimensional optimization problems in three modes: continuous nesting (Mode 1), optimization with intelligence after obtaining a feasible solution (Mode 2), and optimization with the DPSA-POA after obtaining a feasible solution (Mode 3). Of the three modes, Mode 2 has the highest accuracy, but its calculation time is approximately 10 h. Although the accuracy of Mode 3 is slightly worse, can only achieve 98%–99% of Mode 2, its calculation time is only approximately 1–3 h. The comprehensive performance of Mode 1 is poor, the convergence accuracy can only reach 97% of Mode 2, which corresponds to a calculation time of approximately 4–6 h. The existing superiority of feasibility (SF), stochastic ranking (SR), penalty function (PF), ε-constraint (EC) and adaptive ε-constraint (Adaptive EC) methods cannot converge to a feasible solution stably, and the accuracy of the results of these methods under the condition of obtaining a feasible solution is significantly lower than that of Mode 2 and Mode 3, reaching only 95%.