Modular exponentiation is one of the most fundamental operations in lots of encryption and signature systems. Due to the heavy computation cost of modular exponentiation, many schemes have been put forward to securely outsource modular exponentiation to cloud. However, most of the existing approaches need two non-colluded cloud servers to securely complete the modular exponentiation, which will result in private data leakage upon the cloud servers collude. Besides, most existing schemes assume both base and exponent in modular exponentiation are private, which does not conform to many real-world applications. For example, in public key encryption system that uses modular exponentiation, one of base and exponent is the public key, and the other is a message. Usually, only the message should be privately protected. In this paper, we propose two secure outsourcing schemes for fixed-base (public base and private exponent) and fixed-exponent (private base and public exponent), respectively. In the proposed schemes, we employ only one cloud server and can thus avoid collusion attack. Further, we achieve an efficient and secure Paillier encryption outsourcing scheme based on our secure modular exponentiation outsourcing methods. Additionally, we theoretically analyze our overheads and leverage simulation experiments to evaluate our proposed solutions, which show our schemes can achieve high efficiency.