The purpose of this paper is to introduce, study, and partially develop new iterative methods for parallel processing in estimation of large systems. The methods are imitations of the classical Jacobi and Gauss-Seidel iterative methods for solving linear algebraic equations, but otherwise use entirely different concepts and techniques. One of the main motivations for this new development has been the fact that the resulting algorithms are suitable for implementation on multiple processor systems with all the advantages that such systems offer in off-line and especially on-line computations, such as cost, availability, response time, and program modularity. Equally important is the fact that the sampling rates for different parts of the system can be chosen freely to match the dynamics of each individual subsystem, which provides a basis for building efficient multirate schemes for parallel estimation.