AbstractWe consider the adaptive stochastic problem for a system described by a controlled Markov chain (CMC) with a finite number of states. The novelty of the approach consists in adaptation technique for optimization of the system with unknown distribution of the cost function. This approach is applicable to the Internet congestion control with active users, motivating our study. In fact, we consider the homogenous finite CMC, introduce the assumptions of processes independence, stationarity of positive random losses and the bounds for their second moments, and the model regularity. The uncertainty is that the mean loss matrix is unknown. The related convex stochastic optimization problem as well as ideas of Mirror Descent Algorithm (MDA) give us the ability to design the control algorithm and to prove the upper bound for the difference between the expectation of time-averaged losses and their theoretical minimum. Asymptotically, the upper bound equals (InTTσ2N In(N K))1/3 and weakly depends on the state number K (standing inside the logarithm); here N and T stand for the number of control actions and the time horizon, respectively, parameter σ2 bounds the conditional second moments of the losses.