Sleeping Contextual/Non-Contextual Thompson Sampling MAB for mmWave D2D Two-Hop Relay Probing

Abstract

Millimeter wave (mmWave) band, i.e., 30 to 300 GHz, is characterized by short range transmissions and vulnerability to path blockage necessitating the use of relaying. Probing more relays finds out the best relay having the highest spectral efficiency but at the expense of increasing the probing overhead due to excessive beamforming training (BT) causing a decrease in the overall throughput. In this paper, mmWave two-hop relaying will be formulated as a single player multi-armed bandit (MAB) problem enabling one relay probing while maximizing the achievable spectral efficiency. Moreover, the relays could not establish the mmWave link due to blockage for instance will be identified as sleeping relays and eliminated from the rest of the MAB game. Thus, sleeping non contextual MAB (S-MAB) algorithm, namely sleeping Thompson sampling (S-TS) will be proposed to handle the problem. Furthermore, by utilizing the multiband capability of standardized WiGig devices containing both 2.4/5 GHz WiFi and 60 GHz mmWave bands, WiFi information will be used as contexts of the MAB game. Therefore, sleeping contextual MAB (S-CMAB) algorithm, namely S-CTS, will be proposed as well. Numerical and regret analysis ensure the superior performance of the S-CMAB algorithm over the S-MAB counterpart and the existing mmWave relay probing solutions accompanied with high convergence rates.