In this paper, the meso-structure of the skeleton of coarse-graded asphalt mixtures was studied numerically in terms of particle coordination numbers and mixture porosity. The results show that coarse aggregates play a double role in the mixture. They are elements of the mixture skeleton and also void filling materials of the skeleton structure. Upon the assumption that the coarse aggregates can be further categorized into 2 classes as larger coarse aggregates and fine-coarse aggregates which divided by the sieve size of 1/2 of the nominal maximum size, some fine-coarse particles will actually act as void fillers rather than the elements of the skeleton structure. With the increase of the number of the fine-coarse aggregates, more fine-coarse aggregates will have a similar spatial location as that of the larger coarse aggregates in the mixture. It is therefore justifiable to reason that if fine aggregates in the mixture exceed certain amounts, the skeleton of coarse aggregate to coarse aggregate contact will no longer predominate. Reasonably good agreement was obtained between the numerical studies and the laboratory work.