When implementing Federated Learning (FL) on Edge Intelligence Controllers (EIC) in the Industrial Internet of Things (IIoT), it is important to consider the limitations of the EICs’ computational capabilities and to address potential privacy concerns. For the efficient and secure implementation of FL on EICs, three key issues require attention: (i) efficient deployment on EICs with limited computational capacity, (ii) avoiding privacy issues that arise from offloading strategies when using offloading to accelerate, and (iii) mitigating privacy leaks that may result from disclosed parameters. To address the aforementioned concerns, this paper proposes a task offloading model called FedOffloading. Employing Deep Reinforcement Learning (DRL) techniques, FedOffloading accelerates EIC training by offloading the training tasks of the model to the Edge servers (ES). It utilizes the Laplace distribution to safeguard the privacy of the offloading strategies. Meanwhile, to prevent privacy breaches caused by disclosed parameters, FedOffloading allows EICs to inject different levels of artificial noise before transmitting training data. Experimental studies conducted on a test platform reveal that, compared to classical FL, FedOffloading can reduce training time by 54.70%, and even up to 78.06% when training larger models. The Security Module effectively protects the offloading strategies, meeting privacy requirements while also minimizing training time. In addition, to prevent privacy leakage caused by EICs, we introduce noise in the parameters disclosed during training, and show that the intermediate activation data is more susceptible to noise.