The algebraic structures of zero curvature representations are furnished for multilayer integrable couplings associated with matrix spectral problems, both discrete and continuous. The key elements are a class of matrix loop algebras consisting of block matrices with blocks of the same size. As illustrative examples, isospectral and non-isospectral integrable couplings and the corresponding commutator relations of their Lax operators are computed explicitly in the cases of the Volterra lattice hierarchy and the AKNS hierarchy, along with their τ-symmetry algebras.