This article proposes a finite population Markovian queueing model with discouragement and vacationing server to examine the performance of the fault-tolerant system in which admission of failed machines for repair jobs is controlled according to F-policy. When the number of broken-down machines in such systems reaches to its maximum capacity, it does not allow the failed machines to enter in the system until the counts of broken-down machines drop to a pre-specified threshold level ‘F’. The feature of vacationing server along with set up, reboot and recovery are incorporated to deal with real-time scenarios of fault-tolerant systems. By considering the birth–death process, the governing equations are constructed using appropriate transition rates of the concerned system. The steady-state queue size distribution and various system metrics are evaluated using a recursive approach. To illustrate the usage of analytical findings, sensitivity analysis of the system metrics with regard to specific parameters is presented. The particle swarm optimization and harmony search optimization techniques are employed to determine the optimal decision parameters and optimum total cost.