The data-driven cluster-based reduced-order model (CROM) methods[47–50] are very effective for predicting the nonlinear effective properties of heterogeneous materials. The Cluster Interaction Matrix (CIM) in these methods can be regarded as a clustered discrete Green's function and is the critical part of CROM methods, which more and more scholars have realized. However, the explicit expression of Green's function is available only for homogeneous material. The CIM for heterogeneous materials needs to be calculated through numerical approaches, which face the challenge of computational efficiency. Therefore, this paper proposes an efficient method to construct the CIM of heterogeneous materials. By deriving the Green's function in Fourier space under the governing equation of FCA, the CIM of homogeneous materials that can provide the self-equilibrium stress field is calculated efficiently. Then, to avoid the effects of stress averaging on clusters, we augment the proposed cluster-element interaction matrix with the self-equilibrium stress basis vector of heterogeneous material without introducing additional computational cost. Based on the augmented cluster-element interaction matrix, the self-equilibrium stress space is improved, and the CIM of highly heterogeneous materials is calculated using the minimum complementary energy principle. The proposed algorithm significantly improves the offline computational accuracy of FCA, enabling efficient prediction of nonlinear effective properties and yield surface under complex load paths. Numerical examples demonstrate the efficiency and accuracy of the proposed method. In addition, we provide the relationship between the CIMs of a homogeneous material of FCA and SCA. Based on the proposed algorithm in this paper, it can be helpful for other CROM methods.