We show that integrable systems of the Calogero type and its generalizations can be obtained from gauged one-dimensional matrix models including a fermionic part and a Chern-Simons term. The hermitian matrix model gives rise to the inverse square potential system, while the unitary matrix model gives rise to the inverse sine square potential. For the quantum version of these models to exist, we find that the coefficient of the interaction potential must be quantized to l( l+1), with l an integer. These are exactly the values that, in the absence of an external potential, make the system equivalent to a collection of free fermions or free bosons.