Due to the problem of complex aggregate stacking and adhesion, current construction site aggregate grade detection relies on traditional screening methods and single digital image processing technology, which causes inefficiency and segmentation identification difficulties. This problem has become a technical bottleneck in achieving automatic construction site mixed aggregate grade detection. This study constructs a noncontact testing platform for aggregate gradation based on a self-developed sampling and testing device for mixed aggregate gradation and improved deep learning algorithms, which enables rapid testing of mixed aggregate gradation. Based on the principle of instance segmentation, each aggregate in the stacked mixed aggregates collected based on hardware observation is used as an instance to detect independent aggregate targets in the mixed aggregate images with different moisture contents, sand contents and cohesive stacking. An improved aggregate segmentation convolutional neural network model (AS Mask RCNN: Aggregate Segmentation Mask RCNN) is used to achieve the gradation detection of mixed aggregates. This study employed three different types of experiments, and the results showed that the AS Mask RCNN network model achieved an accuracy of over 89.13% in the three experimental situations, and compared the results with those of the Faster RCNN and Mask R-CNN models, with an accuracy improvement of 8.85% and a reduction of 1.29 s in the processing time of a single image segmentation, which can meet the field near real-time detection requirements. The self-developed noncontact testing platform for aggregate grading can adapt to practical applications in complex environments, enabling digital, automated and intelligent noncontact rapid testing of mixed aggregate grading, further improving the accuracy of aggregate grading testing and serving the high-quality development of reservoir dam construction in China.