Considering stress constraints in multi-material topology optimization is of great importance from both theoretical and application perspectives. In this article, the stress-constrained multi-material topology optimization problem is considered under the framework of an alternating active-phase algorithm. A nodal variable strategy is employed. In addition, a material distribution-based cluster method is employed instead of a global stress constraint to improve control of the local stress level. The von Mises stresses of the elements are aggregated into several clusters using a p-norm function to represent the stress constraints. Numerical examples are presented, and the influences of key parameters are discussed. The effectiveness of the proposed approach is demonstrated through numerical results.