In this paper, we consider linear quadratic optimal control with constraint for discrete-time stochastic systems with state and disturbance dependent noise. With the aid of the Lagrange multiplier theorem, we present a necessary condition under which the problem is well posed and a state feedback solution can be derived. Moreover, a sufficient condition is introduced for the case in which the quadratic-term matrices are non-negative. In a way, the previous results on stochastic linear quadratic optimal control without constraint can be regarded as corollaries of the theorems of this paper.