AbstractIn cloud computing, virtual machine (VM) consolidation and migration are significant schemes to enhance the energy efficiency and costs of the cloud environment. The process of VM consolidation is to execute a more number of tasks with less power consumption. However, due to unreliable physical resources, energy consumption is increased. So, to solve this issue, improved beetle swarm optimization (IBSO) algorithm based on energy‐aware VM consolidation is presented in this article. IBSO algorithm joins together beetle swarm optimization (BSO) and particle swarm optimization (PSO). The proposed algorithm is defined with the effective representation of solution and multi‐objective functions such as power consumption, resource wastage, and SLA violation. Results of the work are depicted that the proposed approach outperformed different previous methods in terms of different evaluation measures.