With the help of a general Nth-order rogue wave solution in a compact determinant form for the AB system, we illuminate that, by suitably choosing the wavenumber and frequency of the background wave of the component A, W-shaped soliton complex containing a fixed number of algebraic solitons merging or separating with each other exists in the component A, and rogue-wave pattern transition between the four-petaled structure and dark structure occurs in the component B. The more complicated rogue-wave pattern transitions due to the nonlinear superposition corresponding to the higher-order four-petaled rogue waves and higher-order dark rogue waves of fundamental, triangular and circular dynamical structures up to third order are demonstrated, respectively. The spectrum properties which are related to the rogue-wave pattern transitions are revealed.