In this paper, a cluster-based nonuniform transformation field analysis (CNTFA) is suggested for the multicale analysis of elastic-viscoplastic composite. The current method estimates the effective behavior of composite directly from the local constitutive relation of phase materials, so that the effective behavior of composite can be predicted only by the integration of local constitutive equation. Furthermore, the number of integration points of the local constitutive relation necessary for the evolution of reduced variables is significantly decreased by dividing the representative volume element (RVE) into a number of material clusters using the data compression technology by clustering. This plays a decisive role in reducing the amount of information necessary for the prediction of effective behavior, and enhancing the computational efficiency. Numerical examples for the aluminium matrix composite with boron particles, the plate-fin structure under high temperature and short fiber reinforced epoxy matrix composite explain that the current method can be applied efficiently for the prediction of effective behavior in the elastic-viscoplastic composite.