Wireless mesh networks (WMNs) play a vital role in modern communication systems, and optimizing the placement of wireless mesh routers is crucial for achieving efficient network performance in terms of coverage and connectivity. However, network congestion caused by overlapping routers poses challenges in WMN optimization. To address these issues, researchers have explored metaheuristic algorithms to strike a balance between coverage and connectivity in WMNs. This study introduces a novel hybrid optimization algorithm, namely Transient Trigonometric Harris Hawks Optimizer (TTHHO), specifically designed to tackle the optimization problems in WMNs. The primary objective of TTHHO is to find an optimal placement of routers that maximizes network coverage and ensures full connectivity among mesh routers. Notably, TTHHO's unique advantage lies in its efficient utilization of residual energy, strategically placing the sink node in areas with higher energy levels. The effectiveness of TTHHO is demonstrated through a comprehensive comparison with seven well-known algorithms, including Harris Hawks optimization (HHO), Sine Cosine Algorithm (SCA), Gray Wolf Optimization (GWO), Particle Swarm Optimization (PSO), Moth Flame Optimization (MFO), Equilibrium Optimizer (EO), and Transient Search Optimizer (TSO). The proposed algorithm is rigorously validated using 33 benchmark functions, and statistical analyses and simulation results confirm its superiority over other algorithms in terms of network connectivity, coverage, congestion reduction, and convergence. The simulation outcomes demonstrate the effectiveness and efficacy of the proposed TTHHO algorithm in optimizing WMNs, making it a promising approach for enhancing the performance of wireless communication systems.