AbstractIn this article, an optimal command‐filtered backstepping control approach is proposed for uncertain strict‐feedback nonlinear multi‐agent systems (MASs) including output constraints and unmodeled dynamics. One‐to‐one nonlinear mapping (NM) is utilized to recast constrained systems as corresponding unrestricted systems. A dynamical signal is applied to cope with unmodeled dynamics. Based on dynamic surface control (DSC), the feedforward controller is designed by introducing error compensating signals. The optimal feedback controller is produced applying adaptive dynamic programming (ADP) and integral reinforcement learning (IRL) techniques in which neural networks are utilized to approximate the relevant cost functions online with established weight updating laws. Therefore, the entire controller, including feedforward and feedback controllers, not only ensures that all signals in the closed‐loop systems are cooperative semi‐globally uniformly ultimately bounded (SGUUB) and the outputs maintain in the provided time‐varying constraints, but also makes sure that the cost functions achieve minimization. A simulation example is presented to illustrate the feasibility of the proposed control algorithm.