The balance between exploration and exploitation in the particle swarm optimization (PSO) algorithm is often discussed but has not been well solved. To further improve the capabilities of exploration and exploitation in the search space, the aim of this study is to focus on a novel comprehensive learning strategy for the PSO algorithm, which is named the heterogeneous pbest-guided comprehensive learning particle swarm optimizer (HPBPSO). In this algorithm, the population is divided into two subpopulations. In the exploration subpopulation, a pbest-guided mechanism is designed for obtaining better particles’ personal best information. Meanwhile, the non-elite personal best particles learn from elite personal best particles by using dynamic crossover strategy. In the exploitation subpopulation, in order to fully leverage the benefits of learning from high-quality individuals, the learning process is divided into two distinct categories: elite individual learning and intragroup learning. Furthermore, intergroup perturbation is designed to accelerate convergence performance. The experiments on classical functions are firstly used to verify the effectiveness and mutuality of the proposed strategies. Moreover, the performance of the proposed algorithm is evaluated on two well-known benchmarks and medical image segmentation problem. According to the statistical results, the HPBPSO algorithm is not only better than the other existing state-of-the-art PSO variants and metaheuristic evolutionary algorithms, but also applicable to real-world optimization problems.