Hidden Semi-Markov models for semantic-graph language modeling

Abstract

Semantic communication is expected to play a critical role in reducing traffic load in future intelligent large-scale sensor networks. With advances in Machine Learning (ML) and Deep Learning (DL) techniques, design of semantically-aware systems has become feasible in recent years. This work focuses on improving the reliability of the semantic information represented in a graph-based language that has been recently proposed. Inaccuracies in the representation of the semantic information can arise due to multiple factors, such as algorithmic shortcomings or sensory errors, decreasing the performance of the semantic extractor. This study aims to model the temporal evolution of semantic information, represented using the graph language, to enhance its reliability. Each unique graph configuration is treated as a distinct state, leading to a Hidden Semi-Markov Model (HSMM) defined over the state space of the graph configurations. The HSMM formulation enables the integration of prior knowledge on the semantic signal into the graph sequences, enhancing the accuracy in identifying semantic innovations. Within the HSMM framework, algorithms designed for graph smoothing, semantic information fusion, and model learning are introduced. The efficacy of these algorithms in improving the reliability of the extracted semantic-graphs is demonstrated through simulations and video streams generated in the CARLA simulation environment.