Metaheuristic optimization is considered one of the most efficient and powerful techniques of recent decades as it can deal effectively with complex optimization problems. The performance of the optimization technique relies on two main components: exploration and exploitation. Unfortunately, the performance is limited by a weakness in one of the components. This study aims to tackle the issue with the exploration of the existing jellyfish search optimizer (JSO) by introducing a hybrid jellyfish search and particle swarm optimization (HJSPSO). HJSPSO is mainly based on a JSO structure, but the following ocean current movement operator is replaced with PSO to benefit from its exploration capability. The search process alternates between PSO and JSO operators through a time control mechanism. Furthermore, nonlinear and time-varying inertia weight, cognitive, and social coefficients are added to the PSO and JSO operators to balance between exploration and exploitation. Sixty benchmark test functions, including 10 CEC-C06 2019 large-scale benchmark test functions with various dimensions, are used to showcase the optimization performance. Then, the traveling salesman problem (TSP) is used to validate the performance of HJSPSO for a nonconvex optimization problem. Results demonstrate that compared to existing JSO and PSO techniques, HJSPSO contributes in terms of exploration and exploitation improvements, where it outperforms other well-known metaheuristic optimization techniques that include a hybrid algorithm. In this case, HJSPSO secures the first rank in classical and large-scale benchmark test functions by achieving the highest hit rates of 64% and 30%, respectively. Moreover, HJSPSO demonstrates good applicability in solving an exemplar TSP after attaining the shortest distance with the lowest mean and best fitness at 37.87 and 36.12, respectively. Overall, HJSPSO shows superior performance in solving most benchmark test functions compared to other optimization techniques, including JSO and PSO. As a conclusion, HJSPSO is a robust technique that can be applied to solve most optimization problems with a promising solution.