The recently developed iterative Wiener filter using a fourth-order tensorial (FOT) decomposition owns appealing performance in the identification of long length impulse responses. It relies on the nearest Kronecker product representation (with particular intrinsic symmetry features), together with low-rank approximations. Nevertheless, this new iterative filter requires matrix inversion operations when solving the Wiener–Hopf equations associated with the component filters. In this communication, we propose a computationally efficient version that relies on the conjugate gradient (CG) method for solving these sets of equations. The proposed solution involves a specific initialization of the component filters and sequential connections between the CG cycles. Different FOT-based decomposition setups are also analyzed from the point of view of the resulting parameter space. Experimental results obtained in the context of echo cancellation confirm the good behavior of the proposed approach and its superiority in comparison to the conventional Wiener filter and other decomposition-based versions.