Abstract Beluga whale optimization (BWO) algorithm is a recently proposed population intelligence algorithm. Inspired by the swimming, foraging, and whale falling behaviors of beluga whale populations, it shows good competitive performance compared to other state-of-the-art algorithms. However, the original BWO faces the challenges of unbalanced exploration and exploitation, premature stagnation of iterations, and low convergence accuracy in high-dimensional complex applications. Aiming at these challenges, a hybrid BWO based on the jellyfish search optimizer (HBWO-JS), which combines the vertical crossover operator and Gaussian variation strategy with a fusion of jellyfish search (JS) optimizer, is developed for solving global optimization in this paper. First, the BWO algorithm is fused with the JS optimizer to improve the problem that BWO tends to fall into the best local solution and low convergence accuracy in the exploitation stage through multi-stage exploration and collaborative exploitation. Then, the introduced vertical cross operator solves the problem of unbalanced exploration and exploitation processes by normalizing the upper and lower bounds of two stochastic dimensions of the search agent, thus further improving the overall optimization capability. In addition, the introduced Gaussian variation strategy forces the agent to explore the minimum neighborhood, extending the entire iterative search process and thus alleviating the problem of premature stagnation of the algorithm. Finally, the superiority of the proposed HBWO-JS is verified in detail by comparing it with basic BWO and eight state-of-the-art algorithms on the CEC2019 and CEC2020 test suites, respectively. Also, the scalability of HBWO-JS is evaluated in three dimensions (10D, 30D, 50D), and the results show the stable performance of the proposed algorithm in terms of dimensional scalability. In addition, three practical engineering designs and two Truss topology optimization problems demonstrate the practicality of HBWO-JS. The optimization results show that HBWO-JS has a strong competitive ability and broad application prospects.