The increasing demand for new mobile applications puts a heavy demand for more processing power and resources in smart mobile devices (SMD). Offloading is a promising solution for these issues which tries to move data, code, or computation from the SMDs to the remote or nearby resourceful servers. To increase the effectiveness of the offloading process and make better decisions, various stochastic offloading schemes are proposed in the literature which has adapted different stochastic models. Although offloading issues are vastly studied in the literature, there is a lack of comprehensive paper to focus on stochastic offloading solutions. This paper presents a meticulous review and classification of the stochastic offloading frameworks designed for different environments such as mobile cloud computing, mobile edge computing), and Fog computing. Following this, it first presents the required background concepts and key issues regarding the offloading problem and stochastic models. It then puts forward a taxonomy of the stochastic offloading approaches according to their applied stochastic models and highlights their architectures and contributions. In addition, in each category, a comparison of the stochastic offloading schemes is provided to illuminate their features. Finally, the concluding remarks and open research areas.