Modern and heterogeneous asphalt mixtures are usually produced using various kinds of modifiers such as rubber, polymer, and fiber. These materials are incorporated to improve sustainability and reduce the extent and severity of distresses such as rutting and low-temperature cracking. Currently, there is a lack of a robust real-time method for the identification of these additives in mixtures. In this research, a portable molecular sensing technology is integrated with machine learning (ML) to characterize asphalt binders modified with ground tire rubber (GTR) and asphalt mixtures containing different amounts of recycled materials. A database containing several near-infrared (NIR) spectra for binder and asphalt mixture samples are used to develop the ML-based detection models. The acceptable accuracy reported in this study implies that the proposed integrated NIR and ML approach can be used as a promising tool to differentiate and classify various types of asphalt binders and mixtures. This monitoring and data collection framework can contribute to improved sustainability via accelerating and optimizing the construction material detection and selection process throughout the pavement life. Furthermore, the expensive and cumbersome process of binder extraction and recovery could be avoided using the proposed method.