Optimizing concrete mix designs, especially for recycled brick and tile (RBT) concrete, is critical in construction materials. The optimization of RBT concrete mix designs was investigated using machine learning (ML) methods and metaheuristic algorithms. Hyperparameter tuning of the XGBoost (XGB) model was conducted using SFO, HGS, and HBA algorithms. Critical hyperparameters, including population size and iterations, were identified to ensure model convergence and performance. The model's predictive accuracy for compressive strength (CS) was evaluated using the coefficient of determination (R2) metric, revealing its superiority in mitigating overfitting. SHAP analysis highlighted testing age, cement content, and total coarse aggregate as influential input variables. A practical approach to RBT concrete mix design optimization was introduced, enabling the achievement of specific CS targets while maximizing RBT usage. Pareto solutions provided actionable insights for engineers to create tailored high-performance concrete mixes. This research contributes significantly to sustainable concrete mix design by offering practical ML tools.