Output-only methods based on machine/deep-learning algorithms are highly practical approaches for timely detecting potential damages in civil structures as they directly employ measured vibration signals but do notrequire exact knowledge of input loading nor the service interruption for manual inspection. However, there isno one-size-fits-all model that is optimal for all problems in different perspectives; hence, it is necessary todiscover the advantages as well as drawbacks of different models, then leverage these understandings to selectthe most appropriate model for specific structures in reality. Therefore, this study develops a framework thatfacilitate the model selection by extensively comparing various machine learning-based methods ranging fromrelatively simple ones such as Na¨ıve Bayes to complex ones such as the extreme boosting tree-based ensemblemodel. The framework can provide comparison results include various aspects such as model complexity, training time, detection accuracy, and inference time.