Because the position and direction of the human body is not fixed in an actual environment, the incidence direction of the electromagnetic field (EMF) from mobile communication base stations, WiFi access points, broadcasting towers, and other far-field sources is arbitrary. To analyze the overall health effects of radio frequency EMF exposure, the dosimetric assessment for such environmental exposures created from an unspecified number of sources in daily life, along with exposures from specific EMF sources, must be quantified. This study is aimed at numerically evaluating the time-averaged specific absorption rate (SAR) of the human brain for environmental EMF exposure in the frequency range of 50–5800 MHz. Whole-body exposure to EMFs that are evenly incident spatially is considered. By comparing the results of several incidence directions and the number of polarizations, an optimal calculation condition has been derived. Finally, based on the results measured in Seoul at the end of 2021, the SAR and daily specific energy absorption (SA) in the brains of both a child and an adult for downlink exposures from 3G to 5G base stations are reported. Comparison results of the daily brain SA for exposure to DL EMF in all 3G to 5G mobile networks and exposure to a 10-min voice call (uplink EMF) using a mobile phone connected to a 4G network show that the SA from the downlinks is much higher than that from the uplinks.