Currently, high gain antenna arrays are deployed in cellular radio systems to enhance wireless capacity. These advanced antenna systems increase the peak equivalent isotropic radiated powers (EIRPs), thereby potentially also increasing the size of the exclusion zones for radio frequency (RF) electro magnetic field (EMF) exposure that are applied to limit the presence of workers and the general public in the vicinity of the antennas. The exclusion zones are determined from the RF EMF exposure limits, typically expressed in terms of time-averaged incident power density values as provided by national authorities and regulatory bodies. To mitigate possible deployment challenges in some urban areas due to the overly conservative use of peak EIRP in determining exclusion zones, the possibility of basing the exclusion zone on a time averaged EIRP is discussed. To support this new approach, this paper presents an average power feedback controller that mathematically guarantees that the average power transmitted over a specified time window by a single radio base station stays below a determined value defining the exclusion zone. Such automation may be required by regulators to allow the use of time averaged quantities for RF EMF exposure assessment and exclusion zone determination. The developed average power controller uses a combination of controllers to regulate the rate of change of a momentary fractional frequency resource limit in the scheduler. This restricts the momentary total transmit power since the transmit power is proportional to the scheduled frequency resources. Simulation results, laboratory measurements and on-site measurements for a commercially available 5G MIMO transmitting base station are then reported, all of them verifying that the average power controller performs as intended. The average power control solution is applicable for any 4G or 5G base station, with or without MIMO transmission capability.