A nonlinear pre-distorter (NLPD) at the transmitter, and a maximum-a-posteriori probability (MAP) detector, time-domain Volterra nonlinear equalizer (VNLE), or sparse-VNLE at the receiver are compared for compensation of the pattern-dependent distortion that can occur in high baud rate transmitters and receivers. Experimental results are presented for a 1.206-Tb/s dual-polarization 16-ary quadrature amplitude-modulation (16-QAM) superchannel signal with three subcarriers. The NLPD with iterative calculation of the pre-distortion provides the best performance in back-to-back systems and transmission systems followed by the MAP detector, VNLE, and sparse-VNLE. At the FEC threshold of 1.9 × 10 -2 , the NLPD provides an increase in the transmission distance of 300 km compared with the sparse-VNLE, VNLE, and MAP detector. For transmission over 1500 km of SMF, the VNLE and MAP detector exhibit additional optical signal-to-noise ratio margins of 0.16 and 0.47 dB relative to the sparse-VNLE. Compared with the VNLE, the sparse-VNLE exhibits a 55% reduction in number of kernel coefficients.