Unmanned aerial vehicles (UAVs) are projected to be utilized in a variety of unexpected applications, including agriculture, firefighting, emergency response, intelligent transportation, and so on. Wireless communication is one of the primary facilitators in bringing UAVs into a new phase in such applications. To realize the vision in fifth-generation (5G) networks, we propose a 5G-integration of the flexible multi-UAV system and the Open Radio Access Network (O-RAN) architecture, named U-ORAN. Although different studies have been proposed to optimize the UAV trajectory and resource allocation in the radio access network (RAN), our work is the first study to investigate the benefits of adopting UAVs in the O-RAN architecture. In U-ORAN, we consider a flying base station system and propose a joint optimization problem of multi-UAV trajectory and offloading tasks (UTOT) in which UTOT can optimize the routes from users to the core network as well as resource allocation to process offloading tasks. We decompose UTOT into two sub-problems and provide learning solutions based on the multi-agent reinforcement learning and online learning methodologies, both of which are well supported by the O-RAN architecture. Our intensive numerical simulations show that the proposed approaches outperform in a variety of settings and validation scenarios.