This paper considers linear transceiver design for downlink multiuser multiple-input multiple-output (MIMO) systems. We examine different transceiver design problems. We focus on two groups of design problems. The first group is the weighted sum mean-square-error (WSMSE) (i.e., symbol-wise or user-wise WSMSE) minimization problems and the second group is the minimization of the maximum weighted mean-square-error (WMSE) (symbol-wise or user-wise WMSE) problems. The problems are examined for the practically relevant scenario where the power constraint is a combination of per base station (BS) antenna and per symbol (user), and the noise vector of each mobile station is a zero-mean circularly symmetric complex Gaussian random variable with arbitrary covariance matrix. For each of these problems, we propose a novel downlink-interference duality based iterative solution. Each of these problems is solved as follows. First, we establish a new mean-square-error (MSE) downlink-interference duality. Second, we formulate the power allocation part of the problem in the downlink channel as a Geometric Program (GP). Third, using the duality result and the solution of GP, we utilize alternating optimization technique to solve the original downlink problem. For the first group of problems, we have established symbol-wise and user-wise WSMSE downlink-interference duality. These duality are established by formulating the noise covariance matrices of the interference channels as fixed point functions. On the other hand, for the second group of problems, we have established symbol-wise and user-wise MSE downlink-interference duality. These duality are established by formulating the noise covariance matrices of the interference channels as marginally stable (convergent) discrete-time-switched systems. The proposed duality based iterative solutions can be extended straightforwardly to solve many other linear transceiver design problems. We also show that our MSE downlink-interference duality unify all existing MSE duality. In our simulation results, we have observed that the proposed duality based iterative algorithms utilize less total BS power than that of the existing algorithms.