We proposed a decoupling control algorithm to accomplish the application of woofer–tweeter (W–T) adaptive optics system in generalized irregular pupil region. This algorithm firstly modifies the response matrix of woofer and tweeter by setting corresponding value at some sub-apertures of Shack–Hartmann(S–H) sensor to zero and constructs a slope-based orthogonal basis used to distribute different spatial frequency aberration to dual deformable mirrors (DMs). Then a limited condition is used to calculate the pseudo-inverse matrix of modified response matrix of dual DMs through single value decomposition. Finally, to restrain tweeter generating cross-coupling shape with woofer, a constraint matrix used to reset control vector of tweeter is constructed with the slope-based orthogonal basis and modified response matrix of tweeter. Numeral simulation of W–T adaptive optics system is built to verify the performance of this algorithm and results demonstrate that this algorithm has a good performance to control the W–T adaptive optics system in generalized irregular pupil region, and achieve better performance to suppress the cross-coupling between woofer and tweeter than Zernike decomposition algorithm. At the same time, the experimental results for the W–T AO system demonstrated that this algorithm is practical for the AO system with dual DMs in generalized irregular pupil region.