Abstract This paper deals with an M/G/1 queuing system with server setup and a Bernoulli vacation schedule, in which an unreliable server applies a random N-policy. The server may break down at any time, and the service channel will fail for the duration of the breakdown. When units in the queue are exhaustively served, the server will become idle until the queue size grows to meet a random threshold N(≥1). As soon as the queue size reaches N, the server will immediately begin to serve the waiting units. Upon the service completion of each unit, the server may take a vacation or may remain in the system to serve the next unit, if any. This study derives the queue size distribution during a busy period initiation epoch and a departure epoch, respectively. We also give the analytic expression for the distribution of the delay busy period, which includes breakdown periods and actual busy periods. Furthermore, numerical examples of the long-run cost functions for the random N-policy and the classical N-policy are investigated.