The Internet of Things (IoT) landscape is booming, and a wide range of IoT endpoints has been integrated into various aspects of life, opening opportunities for malicious attacks that compromise IoT security. Consequently, effective malicious traffic identification methods play a crucial role in ensuring IoT security. However, the current lightweight methods face two main challenges. Firstly, they often generate feature sets that are time-consuming to construct and less adaptable to various scenarios. Secondly, classification models built upon inefficient neural networks incur a significant computational overhead. To address these drawbacks, in this work, we propose GateKeeper, which is an UltraLite method for malicious traffic identification on the IoT gateway. We first propose a base model as the cornerstone and target for subsequent optimization. Then we propose dual-aspect optimization strategies for reducing the input dimension and simplifying the model structure, i.e., Key Bytes Selection (KBS) and Attention Module Simplification (AMS) strategy. Finally, we obtain the optimized UltraLite model-GateKeeper, specifically tailored for integration into IoT gateway deployments, where high-speed real-time identification is paramount. Our extensive experiments demonstrate that GateKeeper performs remarkably in identifying malicious IoT traffic, achieving over 97% accuracy in all three IoT malicious traffic classification benchmark tasks, outperforming six state-of-the-art methods. Besides, GateKeeper’s parameters and FLOPs are 65% smaller than those of existing methods. Moreover, in the case of an IoT gateway platform, GateKeeper demonstrates remarkably low time overhead and hardware resource utilization, surpassing state-of-the-art methods.