Sufficient Conditions on the Optimality of Myopic Sensing in Opportunistic Channel Access: A Unifying Framework

Abstract

This paper considers a widely studied stochastic control problem arising from opportunistic spectrum access in a multichannel system, with the goal of providing a unifying analytical framework whereby a number of prior results may be viewed as special cases. Specifically, we consider a single wireless transceiver/user with access to N channels, each modeled as an independent identically distributed discrete-time two-state Markov chain. In each time step, the user is allowed to sense k ≤ N channels, and subsequently use up to m ≤ k channels out of those sensed to be available. Channel sensing is assumed to be perfect, and for each channel used in each time step the user gets a unit reward. The user's objective is to maximize its total discounted or average reward over a finite or infinite horizon. This problem has previously been studied in various special cases including k = 1 and m = k ≤ N, often cast as a restless bandit problem, with optimality results derived for a myopic policy that seeks to maximize the immediate one-step reward when the two-state Markov chain model is positively correlated. In this paper, we study the general problem with 1 m ≤ k ≤ N, and derive sufficient conditions under which the myopic policy is optimal for the finite and infinite horizon reward criteria, respectively. It is shown that these results reduce to those derived in prior studies under the corresponding special cases, and thus may be viewed as a set of unifying optimality conditions. Numerical examples are also presented to highlight how and why an optimal policy may deviate from the otherwise-optimal myopic sensing given additional exploration opportunities, i.e., when m ≤ k.