A coaxial magnetic gear (CMG) with double-layer permanent magnets (PMs) and Spoke structure is proposed to replace the mechanical gearbox in tidal power generation system. For the inner rotor PMs, the distributions of PMs are the spoke structure with different sizes, and the magnetization direction is Halbach array. The outer rotor PMs have two layers, one is Halbach array, and the other is radial magnetization. According to the sensitivity of parameters, the response surface method and multiobjective genetic algorithm are used for optimization analysis to determine the optimal structural dimension parameters of CMG. A CMG topology with 4 inner poles and 17 outer poles is established. Compared with the conventional CMG, the output torque of the proposed CMG is increased by 33.42%. Finally, a prototype was made and a test platform was built. The experimental results show that the transmission ratio of the proposed model is consistent with the theoretical value, the no-load loss is relatively small, and the efficiency is high.