Due to their limited computation capabilities and battery life, Internet of Things (IoT) networks face significant challenges in executing delay-sensitive and computation-intensive mobile applications and services. Therefore, the Unmanned Aerial Vehicle (UAV) mobile edge computing (MEC) paradigm offers low latency communication, computation, and storage capabilities, which makes it an attractive way to mitigate these limitations by offloading them. Nevertheless, the majority of the offloading schemes let IoT devices send their intensive tasks to the connected edge server, which predictably limits the performance gain due to overload. Therefore, in this paper, besides integrating task offloading and load balancing, we study the resource allocation problem for multi-tier UAV-aided MEC systems. First, an efficient load-balancing algorithm is designed for optimizing the load among ground MEC servers through the handover process as well as hovering UAVs over the crowded areas which are still loaded due to the fixed location of the ground base stations server (GBSs). Moreover, we formulate the joint task offloading, load balancing, and resource allocation as an integer problem to minimize the system cost. Furthermore, an efficient task offloading algorithm based on deep reinforcement learning techniques is proposed to derive the offloading solution. Finally, the experimental results show that the proposed approach not only has a fast convergence performance but also has a significantly lower system cost when compared to the benchmark approaches.