Multi-agent systems (MAS), known for their autonomy, fault tolerance, and flexible collaboration capabilities, are widely used in various fields. Communication security is one of the most critical problems of MAS tasks. Once a malicious attacker invades agents in the system, it brings massive threats to the entire system. It tampers with routing messages, discards packets, and paralyzes the communication network. To implement collaboration tasks of MASs safely and efficiently, this work makes security improvements on the hybrid wireless mesh protocol (HWMP) with an additional authentication system. We enhance the proactive and reactive mode of the HWMP and apply it to an integrated system, i.e., the MAS formation control as a proper instance. We present a HWMP-based secure communication model (HSCM) for MASs. The theoretical analysis and simulation results demonstrate a significant enhancement in network performance under various threats, including packet loss rate, throughput, etc. The results indicate that HSCM plays a significant role in resisting common attacks (such as black hole and flooding attacks) when compared to multi-agent systems without any security protection. There is an approximate 30% increase in packet delivery rate, a substantial enhancement in throughput, and a reduction in routing overhead. The latency also fluctuates within reasonable bounds. Our solution provides secure communication for formation control under malicious attacks.