This paper provides a framework to characterize the stability margins of a linear time-invariant multi-agent system where the interaction topology is described by a graph with a directed spanning tree. The stability analysis of the multi-agent system, which is based on the generalized Nyquist theorem, is converted to finding a minimum gain positive definite Hermitian perturbation and a minimum phase unitary perturbation in the feedback path of the loop transfer function. Further, this paper provides a framework to compute the input delay margin of the multi-agent system based on the phase perturbation of the loop transfer function. Specifically, two constrained minimization problems are solved to calculate the gain, phase, and input delay margins of the multi-agent system. Necessary and sufficient conditions concerning the stability of the multi-agent system independent of gain and phase perturbations and input delay are also stated. The theoretical results are then applied to compute the gain, phase, and input delay margin of a cooperative system wherein each agent is an unmanned F-16 VISTA aircraft with short-period dynamics.