We propose a method to calculate the spectral functions of many-body systems by Chebyshev expansion in the framework of matrix product states coupled with canonical orthogonalization (coCheMPS). The canonical orthogonalization can improve the accuracy and efficiency significantly because the orthogonalized Chebyshev vectors can provide an ideal basis for constructing the effective Hamiltonian in which the exact recurrence relation can be retained. In addition, not only the spectral function but also the excited states and eigenenergies can be directly calculated, which is usually impossible for other MPS-based methods such as time-dependent formalism or correction vector. The remarkable accuracy and efficiency of coCheMPS over other methods are demonstrated by calculating the spectral functions of spin chain and ab initio hydrogen chain. For the first time we demonstrate that Chebyshev MPS can be used to deal with ab initio electronic Hamiltonian effectively. We emphasize the strength of coCheMPS to calculate the low excited states of systems with sparse discrete spectrum. We also caution the application for electron-phonon systems with dense density of states.