In 4G cellular networks: LTE/LTE-Advanced and the mobile WiMAX network, some key impact factors: white Gaussian noise, wireless interference, shadow fading, etc., degrade signal transmission quality and reduce wireless service coverage significantly. The degradations can be improved by deploying more eNBs or BSs, but the expensive eNB/BS extremely increases network cost. The specifications of LTE-Advanced and IEEE 802.16j thus propose a relay-based scheme, namely mobile multihop relay (MMR) networks, to cooperate with the existing cellular network while not obviously increasing the relay station (RS) deployment cost. Various types of RSs: Fixed RS, Nomadic RS and Mobile RS, can be adopted in MMR networks. To efficiently deploy different types of RSs becomes a critical issue. Thus, this paper first analyzes the impact factors: transmission quality, deployment price, service coverage and RS overlap index, and then models the cost-effective issue of the RS deployment as an optimization problem. The paper proposes an adaptive cost-based RS deployment (ACRD) approach to form a cost function in terms of all impact factors, and then solves the optimization problem by determining the RS deployment with the least network cost as the solution. Numerical results demonstrate that ACRD outperforms the compared approaches in network cost, transmission quality, RS deployment price, service coverage, and RS overlap index. Additionally, ACRD deploys more RSs on the areas with high-density populations and thus increases transmission quality and guarantees the quality of service (QoS). Finally, the worst case running time of ACRD is analyzed, which requires $$ T(n) = O(I \cdot P) $$T(n)=O(I·P).