Double Deep Q-Network Next-Generation Cyber-Physical Systems: A Reinforcement Learning-Enabled Anomaly Detection Framework for Next-Generation Cyber-Physical Systems

Abstract

In this work, we considered the problem of anomaly detection in next-generation cyber-physical systems (NG-CPS). For this, we used a double deep Q-network-enabled framework, where an agent was trained to detect anomalies in the traffic that does not match the behavior of the legitimate traffic at the end side. Furthermore, the proposed paradigm recognizes known and unknown anomalies by directly engaging with a simulation environment. Given that, it progressively develops its interpretation of anomalies to encompass new, previously unrecognized classes of anomalies by proactively exploring probable anomalies in data that have not been labeled. The method achieves this by concurrently optimizing the use of a limited amount of labeled abnormality data for better understanding (exploitation) and the identification of infrequent, unlabeled anomalies (exploration). During analysis, we observed that the proposed model achieves significant results in the context of average and greedy catching of anomalies in the presence of comparative models.