Abstract As the entrance of the computer systems, the API gateway is an indispensable part of the microservice architecture. To realize the load balancing of API gateway, this paper studies the load balancing algorithm of the API gateway based on the microservice architecture. In doing this, we analyze the microservice architecture level from the data layer, the basic layer, and other levels, take the container cloud as the carrier of the microservice architecture, combine it with the client and API gateway, and design the API gateway based on the microservice architecture. We then judge whether the microservice identifier in the request source of the API gateway client is included in the API gateway routing table and determine the service cluster to which the microservice belongs according to the microservice identifier. After retrieving the qualified backend microservice container list according to the service cluster information, it adopts a load balancing algorithm based on dynamic weight, takes central processing unit (CPU) utilization and memory utilization as parameters to evaluate the resource load of microservers, uses an extreme gradient lifting model to predict CPU utilization and memory utilization, calculates the weight of microservers based on the prediction results, selects the microserver with the highest weight value to make API gateway service requests, and initiates API gateway service calls to specific backend microservice containers, thereby completing the load balancing of the API gateway. The experimental results show that the average load balancing degree of the algorithm is about 95 %, the average network resource utilization rate is as high as 89 %, and the algorithm execution time is short.