The ubiquitous nature of Internet of Things (IoT) devices has posited many challenges that need innovative solutions in the 5G era. Software defined networks (SDNs) are becoming indispensable in managing several aspects of next-generation IoT networking that arise from the need to control highly heterogeneous, geographically dispersed, mobile IoT devices. One such aspect is cache management at the edge. Recently, multiple forms of edge resources, including mobile device clouds and micro-edge data centers have emerged to provide scalable cache placement locations that reduce the costs for the mobile network operator (MNO). As all of these service locations are registered with the MNO (or links established after registration with the 5G base station, BS), content should be placed according to the user's demand and the cost the user is willing to pay to receive the desired level of QoS. To this end, it is important to understand the future popularity of the content for its optimal placement considering the highly dynamic user mobility. In this article, we address two key aspects of a mobile IoT network: security and seamless connectivity for data delivery. We rely on the federated learning (FL) architecture, which enables harnessing data and computational capabilities at end-user devices to train machine learning models. We study FL concepts in the domain of edge computing for IoT use cases, such as caching. We draw conclusions from various state-of-the-art models and posit several challenges that can be overcome via a novel proposed control algorithm.