This paper is a continuation of the work on unbounded Toeplitz-like operators TΩ with rational matrix symbol Ω initiated in Groenewald et al. (2021) [11], where a Wiener-Hopf type factorization of Ω is obtained and used to determine when TΩ is Fredholm and to compute the Fredholm index in case TΩ is Fredholm. Due to the high level of non-uniqueness and complicated form of the Wiener-Hopf type factorization, it does not appear useful in determining when TΩ is invertible. In the present paper we use state space methods to characterize invertibility of TΩ in terms of the existence of a stabilizing solution of an associated nonsymmetric discrete algebraic Riccati equation, which in turn leads to a pseudo-canonical factorization of Ω and concrete formulas of TΩ−1.