Use of swarm intelligence has proliferated over previous couple of years for damage assessment in large and complex structures using vibration data. Available literatures shows ‘ant colony optimization’ (ACO) and ‘particle swarm optimization’ (PSO) are predominantly used for solving complex engineering problems including damage identification and quantification problems. The time requirement and accuracy of the vibration based damage identification algorithms depends on early exploration and late exploitation capabilities of soft computing techniques. However, there are not any literature available comparing algorithms on these bases. In the current study, an inverse problem is constructed using the natural frequency changes which is then solved using ACO and PSO algorithms. The algorithm is run for identification of single and multiple damages in simple support and cantilever beam structures. It's found that, both ACO and PSO based algorithms are capable of detecting and quantifying the damage accurately within the limited number of iterations. However, ACO based algorithm by virtue of its good exploration capability is able to identify near optimal region faster than PSO based algorithm, whereas PSO algorithm has good exploitation capability and hence able to provide better damage quantification than ACO algorithm at latter stages of iteration. Further, PSO based algorithm takes less time to reach at required accuracy level. It is also observed that, the time required for these algorithms are independent of numbers of damage and support conditions.