The cache-enabling unmanned aerial vehicle (UAV) non-orthogonal multiple access (NOMA) networks for mixture of augmented reality (AR) and normal multimedia applications are investigated, which is assisted by UAV base stations. The user association, power allocation of NOMA, deployment of UAVs and caching placement of UAVs are jointly optimized to minimize the content delivery delay. A branch and bound (BaB) based algorithm is proposed to obtain the per-slot optimization. To cope with the dynamic content requests and mobility of users in practical scenarios, the original optimization problem is transformed to a Stackelberg game. Specifically, the game is decomposed into a leader level user association sub-problem and a number of power allocation, UAV deployment and caching placement follower level sub-problems. The long-term minimization was further solved by a deep reinforcement learning (DRL) based algorithm. Simulation result shows that the content delivery delay of the proposed BaB based algorithm is much lower than benchmark algorithms, as the optimal solution in each time slot is achieved. Meanwhile, the proposed DRL based algorithm achieves a relatively low long-term content delivery delay in the dynamic environment with lower computation complexity than BaB based algorithm.