Firefly algorithm (FA) is an easily implementable, robust, simple and flexible technique, but the major drawback associated with this technique is the imbalanced exploration and exploitation during firefly position changing stage. This imbalanced relation degrades the solution quality which ultimately results in either skipping the most optimal solution even present in the vicinities of the current solution or trapping the solution in the local optima. In this paper, this issue is resolved by introducing genetic algorithm (GA) operators namely selection, mutation and crossover operators in the firefly position stage of the standard FA. The performance of the proposed approach has been tested on energy consumption optimization and user comfort management inside smart building and has been compared with the standard FA, GA, artificial bee colony (ABC) and ant colony optimization (ACO) algorithm in terms of temperature, illumination, air quality and total power consumption minimization and user comfort maximization. The minimum, average, maximum and total power consumption and minimum, maximum and average user comfort were the performance evaluation parameters. The least amount of 145.39 kilowatt hour (kWh) of total power consumed for temperature control was observed for the proposed approach followed by ACO, ABC, FA and GA where the power consumed for temperature was observed as 173.68 kWh, 179.27 kWh, 181.93 kWh and 188.95 kWh, respectively. Similarly, for illumination control, the consumed power for the proposed model was 118.30 kWh followed by FA, ABC, ACO and GA where the power consumed was 146.93 kWh, 162.96 kWh, 169.28 kWh and 193.53 kWh, respectively. For air quality control, the minimum power of 186.94 kWh was found for the proposed algorithm followed by FA, ABC, ACO and GA where the power consumed was 229.01 kWh, 234.38 kWh, 240.47 kWh and 244.76 kWh, respectively. Likewise, maximum user comfort was observed for the proposed technique with the value of 0.94004/1 followed by ACO, ABC, FA and GA where the user comfort recorded was 0.939655/1, 0.93878/1, 0.938314/1 and 0.937896/1, respectively. The statistical analysis shows the efficiency of the proposed model for power consumption minimization and user comfort maximization.