The mechanical properties of traditional pervious concrete are insufficient, which limits its application. In view of the imperfections of traditional permeable concrete in mechanics, this paper aimed to find a suitable material composition that can be used as a feasible mix design of high-performance pervious concrete, to essentially improve its mechanical properties. Based on the view that concrete is a two-phase material, in order to understand the rheological properties of the matrix, it was subjected to a rheological test, and then the filler aggregate was uniformly incorporated into the aforementioned matrix to further explore the composition and properties of the resulting pervious concrete. For the matrix, the orthogonal array employed was L16(45), which consisted of five factors, each with four levels. Base on the fluidity and compressive strength of the tested matrix, three groups of suitable matrixes mix proportions were selected to serve as the matrix type for pervious concrete mix proportion design. Then, an orthogonal array L9(34), which consisted of four controllable three-level factors, was adopted in the pervious concrete. The parameters investigated included the coarse aggregate size, fine aggregate content, matrix type, and aggregate-to-binder ratio. The test results demonstrate that the key factors affecting the compressive strength of the matrix and the pervious concrete were closely related to the cementitious material. In the matrix, the proportion of the cementitious material was the most important factor, while in the pervious concrete, the type of matrix was the most important factor.