We investigate the effects of counterrotating terms on geometric phase and its relation to the resonance of the Rabi model. We apply the unitary transformation with a single parameter to the Rabi model and obtain the transformed Hamiltonian involving multiple harmonic terms. By combining the counter-rotating-hybridized rotating-wave method with time-dependent perturbation theory, we solve systematically time evolution operator and then obtain the geometric phase of the two-level system. Our results are beyond adiabatic approximation and rotating-wave approximation (RWA). Higher-order harmonic resonance happens when driving frequency is equal to higher-order subharmonic of the Rabi frequency. In comparison with numerically exact results, our calculated results are accurate over a wide range of parameters space, especially in higher-order harmonic resonance regimes. In these regimes we demonstrate geometric phases change dramatically while those of the RWA are smooth. The RWA is thoroughly invalid even if the driving strength is extremely weak. We find it is the higher-order harmonic terms that play an important role on the cyclic state and demonstrate the characters of geometric phase in higher-order harmonic resonance regime. We also present analytical formalism of the change rate of geometric phase and quasienergies, which agree well with numerically exact ones even in the strong driving case. The developed method can be applied to explore the dynamics of strongly driven qubits and physical properties of higher-order harmonic processes.