This paper considers the leaderless consensus problem of linear time-invariant multi-agent systems with infinite distributed communication delays. A novel distributed low gain controller is proposed based on the solution to a parametric algebraic Riccati equation. It is shown via the newly developed Lyapunov-like method that not only the consensus of linear time-invariant multi-agent systems can be achieved exponentially under some mild assumptions but also an estimate of the exponential convergence rate of consensus is given in this work. The Lyapunov-like method is also extended to handle a special case of linear time-varying multi-agent systems. In addition, the obtained results include the results on the leaderless consensus of linear multi-agent systems with bounded distributed communication delays as special cases. To the best of our knowledge, this is the first work that develops the Lyapunov-like method for the leaderless consensus problems of both time-invariant and time-varying linear multi-agent systems with infinite distributed communication delays. Finally, a numerical example is presented to illustrate the effectiveness of the proposed controller.