In this paper, we consider two players zero-sum dynamical games in linear stochastic systems described by input-output model with unknown parameters, and with an ergodic output-tracking quadratic index. We combine the ideas of game theory and control theory as a modeling method, and investigate the corresponding stochastic adaptive game problems. There is few existing results related to such stochastic adaptive game problems, since uncertainty raises the difficulty of making decisions for both players. We attempt to cope with this problem by the ideas and methods of adaptive control. We will firstly use the standard least squares to estimate the unknown parameters, and then construct the adaptive strategy profile for both players by using the so-called ``certainty equivalence principle. We will prove that, the adaptive strategic profile makes the system globally stable, and that the action profile is an asymptotic Nash equilibrium solution to our game problem in a certain sense.