SummaryLoad balancing among the processing elements (PEs) of distributed stream processing system (DSPS) is a key issue in the presence of data skewness. Existing data partitioning schemes for DSPS suffer from the scalability problem and system in‐efficiency. Non‐key based partitioning strategies raise prohibitively high memory overhead for the stateful operations with a large number of keys and high data parallelism, while the key‐based schemes introduce load imbalance for highly skewed data. Predicting the nature of stream data in advance can help to reduce the load imbalance among the PEs of DSPS. For this purpose, the heavy hitter algorithms approximate the hot items of streaming data. However, existing designs suffer from unsatisfied prediction accuracy. In this work, we propose an efficient algorithm to filter hot items in a stream of incoming data. The proposed scheme dynamically monitors the items of a stream and greatly improves the accuracy of estimation by keeping the actual key‐value pair for the frequent items. On one hand, to ensure better load balancing for the skewed data streams, the detected hot keys are directed to more than two PEs randomly from the limited workers. On the other hand, for less frequent keys, the proposed scheme explores the principle of the power of two choices to distribute load. We conduct extensive experiments on both real‐world and synthetic data sets. The results show that the proposed pre‐filtering approach significantly outperforms existing designs in terms of prediction accuracy. The results also show that our design achieves a more balanced load as compared to the existing designs.