ABSTRACT In this paper, a homotopy-based reinforcement learning optimal control method is developed for Markov switched interconnected systems with unknown system dynamics. By utilising the subsystem decomposition method and parallel learning control method, the solution of the game coupled algebraic Riccati equations with jumping parameters is approximated. To dispense with the requirement of initial stability, a homotopy-based policy iteration is introduced, which can place unstable poles into a stable plane. In this regard, a model-free reinforcement learning method is presented to design the optimal controller for Markov switched interconnected systems. Finally, the effectiveness of the proposed method is verified by a numerical example and a practical example.