This paper aims to introduce the numerical methods for solving the optimal control theory to model bufferbloat problem. Mathematical tools are useful to provide insight for system engineers and users to understand better about what we are facing right now while experiment in a large-scale testbed can encourage us to implement in realistic scenario. In this paper, we introduce a survey of the numerical methods for solving the optimal control problem. We propose the dynamic optimization sweeping algorithm for optimal control of the active queue management. Simulation results in network simulator ns2 demonstrate that our proposed algorithm can obtain the stability faster than the others while still maintain a short queue length (≈10 packets) and low delay experience for arriving packets (0.4 seconds).