Full waveform inversion (FWI) has the problem of low computational efficiency. The main reasons are low convergence rate and unstable convergence. We mainly use optimization methods to solve computationally inefficient problems. Among the commonly used optimization methods, the L-BFGS (Limited memory Broyden-Fletcher-Goldfarb-Shanno) method converges fast but not stably, while the HCG (Hybrid Conjugate gradient) method converges stably but slowly. Neither of these methods can maximize the computational efficiency of FWI. By using Gauss-Newton direction to approximate the search direction (QN-HCG) of HCG method, the second-order convergence rate of Gauss-Newton method and the strong convergence stability of HCG method can be used at the same time. However, QN-HCG requires the Hessian matrix to be positive definite, which is complicated to compute and occupies a large memory. The memoryless BFGS method is developed from the L-BFGS method. Since the Hessian matrix is not required to be positive definite, the method is simple to compute and occupies less memory, and is therefore more suitable to approximate the search direction of the HCG method. First, in order to improve the convergence speed of FWI and ensure the stability of convergence, we use BFGS without memory variable to approximate the search direction of HCG method (BFGS-HCG), and introduce this method into FWI. Second, the computational efficiency of the proposed method is verified in model tests of acoustic and elastic wave FWI. Model tests show that this approach can improve the computational efficiency of full waveform inversion.