In this paper we prove Hormander-Mihlin multiplier theorems for pseudo-multipliers associated to the harmonic oscillator (also called the Hermite operator). Our approach can be extended to also obtain the $L^p$-boundedness results for multilinear pseudo-multipliers. By using the Littlewood-Paley theorem associated to the harmonic oscillator we also give $L^p$-boundedness and $L^p$-compactness properties for multipliers. $(L^p,L^q)$-estimates for spectral pseudo-multipliers also are investigated.