Fog radio access networks (F-RANs) architecture is regarded as a prominent solution to deploy the caching and computing functions at the edge nodes of the network. However, the ever-increasing data traffic demand and the latency requirements of emerging applications pose new challenges to F-RANs. To minimize content fetching latency, we propose a cooperative content caching and power allocation scheme, which proactively caches the content at the network edge and enables users to dynamically obtain the desired content either from Fog-Access points (F-APs) or proximate user equipments (UEs) through Device-to-Device (D2D) communication. Furthermore, the proposed scheme allocates appropriate transmit power to D2D user equipments (DUEs) so that the transmission rate can be maximized. Specifically, the cooperative content caching issue is initially formulated as a probability-triggered combinatorial multi-armed bandit (CMAB) framework. By considering the user preference and content popularity prediction, an enhanced multi-agent reinforcement learning algorithm is proposed to obtain an optimal caching strategy. Besides, to minimize the content fetching latency and guarantee that each UE can retrieve the desired content, the power allocation problem is then modeled as maximizing the sum data rates of users. Finally, a Q-learning based power allocation strategy is derived. The simulation results based on the dataset from MovieLens reveal that compared with baseline methods, our proposed cooperative caching and power allocation scheme can not only reduce the content fetching latency but also increase the cache hit rate.