We consider a gauge theory on the 5-d $\kappa$-Minkowski which can be viewed as the noncommutative analog of a $U(1)$ gauge theory. We show that the Hermiticity condition obeyed by the gauge potential $A_\mu$ is necessarily twisted. Performing a BRST gauge-fixing with a Lorentz-type gauge, we carry out a first exploration of the one loop quantum properties of this gauge theory. We find that the gauge-fixed theory gives rise to a non-vanishing tadpole for the time component of the gauge potential, while there is no non-vanishing tadpole 1-point function for the spatial components of $A_\mu$. This signals that the classical vacuum of the theory is not stable against quantum fluctuations. Possible consequences regarding the symmetries of the gauge model and the fate of the tadpole in other gauges of non-covariant type are discussed.