Recent years have seen an explosion in wireless video communication systems. Optimization in such systems is crucial – but most existing methods intended to optimize the performance of multi-user wireless video transmission are inefficient. Some works (e.g., Network Utility Maximization (NUM)) are myopic: they choose actions to maximize instantaneous video quality while ignoring the future impact of these actions. Such myopic solutions are known to be inferior to foresighted solutions that optimize the long-term video quality. Alternatively, foresighted solutions such as rate-distortion optimized packet scheduling focus on single-user wireless video transmission, while ignoring the resource allocation among the users. In this paper, we propose a general framework of foresighted resource allocation among multiple video users sharing a wireless network. Our framework allows each user to flexibly choose individual cross-layer strategies. Our proposed resource allocation is optimal in terms of the total payoff (e.g., video quality) of the users. A key challenge in developing foresighted solutions for multiple video users is that the users' decisions are coupled. To decouple the users' decisions, we adopt a novel dual decomposition approach, which differs from the conventional optimization solutions such as NUM, and determines foresighted policies. Specifically, we propose an informationally-decentralized algorithm in which the network manager updates state- and user-dependent resource “prices” (i.e., the dual variables associated with the resource constraints), and the users make individual packet scheduling decisions based on these prices. Because a priori knowledge of the system dynamics is almost never available at run-time, the proposed solution can learn online while performing the foresighted optimization. Simulation results show 7 dB and 3 dB improvements in Peak Signal-to-Noise Ratio (PSNR) over myopic solutions and existing foresighted solutions, respectively.