This paper puts forward a new factorization, named the semi-rank factorization, for para-unitary poly-phase matrix with linear phase, and reveals the equivalent relationship between the orthogonal projection matrix and the linear phase, and characterizes the orthogonal projection matrix. The semi-rank factorization has the completeness property no matter for even or odd number of channels. In addition, a decomposition technique for the starting block matrix is developed using the reflection matrix. The semi-rank factorization provides an efficient and compact representation for multi-band filter banks with linear phase. In particular, the complete representation for multi-band symmetry filter banks with odd number of channels is brand new. As a result, the programmable algorithms are obtained for designs of optimal filter banks based on the complete representation. The optimal examples are presented to illustrate the excellent performance on image compression.