Reliability Analysis (RA) is a critical aspect of structural design and performance evaluation aiming to determine the probability of structural failure under given random input parameters. With modern development of modeling techniques, computational models have achieved higher fidelity but at the increased cost of computational time, which poses a significant challenge for RA. Consequently, surrogate model-assisted RA has been explored as a means of improved efficiency and accuracy. This study proposes a novel learning function, Sample-based Expected Uncertainty Reduction (SEUR), for surrogate model-assisted RA. The SEUR function uses statistical information from the metamodeling with fixed hyper-parameters to construct expected failure probability bounds to sequentially update the design of experiment (DoE). The joint probability densities of input variables are accounted for through simulation methods, including Monte Carlo (MC) and subset simulation (SS). Furthermore, the discrete simulated annealing algorithm is used to search for the optimal design point. The performance of proposed AK-SEUR function is systematically evaluated using six examples of different dimensions, failure probability levels and nonlinearities. The AK-SEUR function is demonstrated to be more effective and efficient than other popular active learning methods in dealing with nonlinear performance functions, small probabilities, and complex limit states. The proposed SEUR function has the potential to improve the efficiency and accuracy of RA, particularly in situations where computational models are time-consuming and the search for the optimal solution is challenging.