Autonomous exploration is a widely studied fundamental application in the field of quadrotor, which requires them to automatically explore unknown space to obtain complete information about the environment. The frontier-based method, one of the representative works on autonomous exploration, drives autonomous determination by the definition of frontier information so that complete information about the environment is available to the quadrotor. However, existing frontier-based methods are able to accomplish the task but still suffer from inefficient exploration, and how to improve the efficiency of autonomous exploration is the focus of research nowadays. Slow frontier generation affecting real-time viewpoint determination and insufficient determination methods affecting the quality of viewpoints are typical of these problems. Therefore, to overcome the aforementioned problems, this article proposes a two-level viewpoint determination method for frontier-based autonomous exploration. First, a sampling-based frontier detection method is presented for faster frontier generation, improving the immediacy of environmental representation compared to traditional traversal-based methods. Second, the access to environmental information during flight is considered for the first time, and an innovative heuristic evaluation function is designed to decide on high-quality viewpoint as the next local navigation target in each exploration iteration. Extensive benchmark and real-world tests are conducted to validate our method. The results confirm that our method optimizes the frontier search time by 85%, the exploration time by around 20%–30%, and the exploration path by 25%–35%.