Asphalt mixtures are multi-phase composites composed of aggregates, bitumen, mineral powders, and voids, and various structures are intertwined during the compaction process. Most of the traditional research focuses on the macro-scale domain, and it is difficult to obtain the internal structure of asphalt mixture in different compaction processes. With the continuous development of digital image technology, the influence of the meso-structure of the asphalt mixture on the compaction quality of the asphalt mixture has become a new means to evaluate the performance of the asphalt mixture. In this paper, different numbers of compactions are selected to represent different stages in the compaction process, the digital images of specimens in different compaction stages are obtained by industrial CT scanning technology. Then, the images are processed and reconstructed in three dimensions using improved image segmentation methods, and the position characteristics and geometric information of coarse aggregate are obtained by combining the Oriented Bounding Box (OBB). The meso-response characteristics of the skeleton structure of the asphalt mixture during compaction were studied. The influence of the internal structure of the mixture on the compaction quality of the mixture was obtained, which is of great significance for the study of improving the durability of the pavement. The results show that the "effective coordination number" (the number of aggregate particles that can transmit force in the skeleton structure) is greatly related to the aggregate size. With the compaction process, the centroid of coarse aggregate in the upper layer of the specimen reflects the overall downward movement trend. The inclination angle of the aggregate spindle tends to be in the range of 80°~100°; the anisotropic amplitude of the xy plane increases, and the direction of the aggregate spindle becomes more and more consistent. With the increase in the number of rotational compactions, these four parameters showed obvious rules, indicating that this meso-characteristic index could well characterize the compaction quality of the asphalt mixture in the compaction process.