This paper focuses on solving stochastic linear quadratic nonzero-sum differential games with completely unknown system matrices and long-time average costs. Firstly, for the case that the system matrices are known, we design a model-based value iteration (VI) algorithm and a model-based robust VI algorithm to solve the coupled algebraic Riccati equations (AREs), which are directly related to the construction of the feedback Nash equilibrium solution. Then, we present a model-free VI algorithm, based on which feedback control strategies for players are designed using only the data of states and inputs, without requiring the prior knowledge of the system matrices. Moreover, we show that the designed feedback control strategies make the closed-loop game system stable and reach the Nash equilibrium. Finally, a numerical simulation is given to demonstrate the effectiveness of the proposed method.