Recent IoT proliferation has undeniably affected the way organizational activities and business procedures take place within several IoT domains such as smart manufacturing, food supply chain, intelligent transportation systems, medical care infrastructures etc. The number of the interconnected edge devices has dramatically increased, creating a huge volume of transferred data susceptible to leakage, modification or disruption, ultimately affecting the security level, robustness and QoS of the attacked IoT ecosystem. In an attempt to prevent or mitigate network abnormalities while accommodating the cohesiveness among the involved entities, modeling their interrelations and incorporating their structural, content and temporal attributes, graph-based anomaly detection solutions have been repeatedly adopted. In this article we propose, a multi-agent system, with each agent implementing a Graph Neural Network, in order to exploit the collaborative and cooperative nature of intelligent agents for anomaly detection. To this end, against the propagating nature of cyber-attacks such as the Distributed Denial-of-Service (DDoS), we propose a distributed detection scheme, which aims to monitor efficiently the entire network infrastructure. To fulfill this task, we consider employing monitors on active network nodes such as IoT devices, SDN forwarders, Fog Nodes, achieving localization of anomaly detection, distribution of allocated resources such as the bandwidth and power consumption and higher accuracy results. In order to facilitate the training, testing and evaluation activities of the Graph Neural Network algorithm, we create simulated datasets of network flows of various normal and abnormal distributions, out of which we extract essential structural and content features to be passed to neighbouring agents.