Extremely high frequency carrier communications such as millimeter-wave (mmW) is considered a promising solution for the future mobile networks. To compensate for the large path loss of the millimeter-wave links, saturated power amplifiers are inevitably used during amplify-and-forward relaying, which potentially causes system degradation due to power saturation induced clipping. In this paper, the tolerance of OFDM and Nyquist subcarrier multiplexing (SCM) signals against power amplifier saturation induced clipping is studied in the context of fiber-optic and millimeter-wave integrated communications for heterogeneous mobile backhaul networks. In the system, 10 Gbaud OFDM and Nyquist SCM signals are converted to an intermediate frequency passband before converted to the optical domain for a radio-over-fiber transmission to a remote antenna unit (RAU). At RAU, the optical radio signals are converted back to the electrical domain at millimeter-wave range. Those are then amplified by an power amplifier prior to a millimeter-wave communication to a remote radio head (RRH) for detection. Since Nyquist SCM signal exhibits a level of lower peak-to-average power ratio (PAPR) than that of OFDM, Nyquist SCM signals are more tolerant against the clipping caused by the power amplifier saturation. For 16 QAM, at clipping ratio (CR) of 4.5 dB, one-band Nyquist SCM signal shows a signal-to-noise ratio (SNR) performance over 10 times higher than that of OFDM. While the increase in the number of Nyquist SCM bands reduces the tolerance, Nyquist SCM signal maintains a better tolerance than OFDM one with respect to 16 QAM, 32 QAM and 64 QAM formats, which suffer severer impact with larger SNR penalty for higher modulation levels.