The application of dual vector millimeter-wave (mm-wave) signals in radio-over-fiber (RoF) systems represents a significant opportunity to enhance spectrum efficiency, transmission capacity, and access flexibility. In addition, facing the increasingly intricate application scenarios, the comprehensive exploitation of high-order quadrature-amplitude-modulation (QAM) signals with hybrid single-carrier (SC) and orthogonal-frequency-division-multiplexing (OFDM) modulation is also vital to rich systematic connotation. Based on bandpass delta-sigma modulation (BP-DSM) and heterodyne detection, we propose what we believe to be a novel scheme for the simultaneous wireless mm-wave transmission of both SC-modulated and OFDM-modulated high-order QAM signals. The innovation lies in the modulation-agnostic nature, accommodating both SC-modulated and OFDM-modulated vector radio-frequency (RF) signals. The BP-DSM is utilized to digitize two independent SC-modulated and OFDM-modulated high-order QAM signals into relatively simple sequences at the transmitter side. With the aid of an optical I/Q modulator, we can integrate both signals after BP-DSM to generate the desired optical quadrature-phase-shift keying (QPSK) signal carrying both information of two original high-order QAM signals. Facilitated by heterodyne detection and a single photodetector (PD), our scheme attains prowess in the detection of both SC-modulated and OFDM-modulated high-order signals. Based on our proposed scheme, we experimentally demonstrate the simultaneous wireless mm-wave transmission of both SC-modulated and OFDM-modulated 512QAM signals at 30-GHz mm-wave band, demonstrating bit-error-rates (BERs) below the hard decision forward error correction (HD-FEC) threshold of 3.8 × 10−3 after transmission over 10-km single-mode fiber (SMF) link and 1-m wireless link. In addition, we further investigate the performance impact between SC-modulated and OFDM-modulated high-order QAM signals, and experiment results indicate that the impact is virtually negligible. Moreover, the performance of the generated QPSK mm-wave signal is transparent to the QAM modulation formats of both SC-modulated and OFDM-modulated signals in our proposed scheme.