This paper presents a novel observer-based controller for a class of nonlinear multi-agent robot models using the high order sliding mode consensus protocol. In many applications, demand for autonomous vehicles is growing; omnidirectional wheeled robots are suggested to meet this demand. They are flexible, fast, and autonomous, able to find the best direction and can move on an optional path at any time. Multi-agent omnidirectional wheeled robot (MOWR) systems consist of several similar or different robots and there are multiple different interactions between their agents, thus the MOWR systems have complex dynamics. Hence, designing a robust reliable controller for the nonlinear MOWR operations is considered an important obstacles in the science of the control design. A high order sliding mode is selected in this work that is a suitable technique for implementing a robust controller for nonlinear complex dynamics models. Furthermore, the proposed method ensures all signals involved in the multi-agent system (MAS) are uniformly ultimately bounded and the system is robust against the external disturbances and uncertainties. Theoretical analysis of candidate Lyapunov functions has been presented to depict the stability of the overall MAS, the convergence of observer and tracking error to zero, and the reduction of the chattering phenomena. In order to illustrate the promising performance of the methodology, the observer is applied to two nonlinear dynamic omnidirectional wheeled robots. The results display the meritorious performance of the scheme.