The accurate prediction of wave height attenuation due to vegetation is crucial for designing effective and efficient natural and nature-based solutions for flood mitigation, shoreline protection, and coastal ecosystem preservation. Central to these predictions is the estimation of the vegetation drag coefficient (Cd). The present study undertakes a comprehensive evaluation of three distinct methodologies for estimating the drag coefficient: traditional manual calibration, calibration using a novel application of state-of-the-art metaheuristic optimization algorithms, and the integration of an empirical bulk drag coefficient formula (Tanino and Nepf, 2008) into the XBeach non-hydrostatic wave model. These methodologies were tested using a series of existing laboratory experiments involving nearshore vegetation on a sloping beach. A key innovation of the study is the first application of metaheuristic optimization algorithms for calibrating the drag coefficient, which enables efficient automated searches to identify optimal values aligning with measurements. We found that the optimization algorithms rapidly converge to precise drag coefficients, enhancing accuracy and overcoming limitations in manual calibration which can be laborious and inconsistent. While the integrated empirical formula also demonstrates reasonable performance, the optimization approach exemplifies the potential of computational techniques to transform traditional practices of model calibration. Comparing these strategies provides a framework to determine effective methodology based on constraints in determining the vegetation drag coefficient.