Integrating blockchain (BC) with Federated Learning (FL) shows promise but presents challenges, particularly in the selection of the most appropriate IoT nodes for sensitive tasks. Existing Artificial Intelligence (AI) based approaches are tailored to dynamic environments, but they are complex and resource-intensive. On the other hand, score-based methods are faster to implement but lack flexibility. In this paper, we propose a two-step hybrid solution which uses the reputation score approach to train a DRL model, creating a framework that combines the efficiency of deterministic methods with the adaptability of AI-based solutions. In fact, we designed a score-based method relying on devices attributes and behavior making the system operational from the outset. Also, this allows the gathering of relevant real-time data for training the DRL model. Besides, the variations in the performances of IoT devices pose a challenge in achieving synchronous aggregation. To address this, we designed a multi-level aggregation mechanism, which allows local models to be uploaded to the BC, where an aggregator is in charge of validation. The validated models are then aggregated into intermediate models. This process continues until a global model is formed. To evaluate our approach, we created several simulation scenarios including the number of nodes to assess scalability, the dropout rate to estimate availability, and the percentage of malicious nodes to evaluate the robustness of the system against attacks. These experiments aimed to demonstrate the effectiveness of our approach. The obtained results are promising highlighting its robustness and flexibility showing improved performance, security, and availability.