Abstract. Transmission in a 40-Gbit/s alternating-phase 0 and duo-binary modulation format with 67%, 50%, and 33% pulse width modu-lation duobinary carrier suppression CS and carrier maximum pulsecarving is demonstrated. A minimum 136-ps/nm dispersion tolerancecan be achieved at this transmission rate.At least 5- and 1.2-dB receiversensitivity improvements are achieved for CS return-to-zero RZ 67%and 50% duobinary modulation, respectively, as compared with the ex-perimental measurement under CS RZ differential phase-shift keyingtransmission over a 320-km three-span optically amplified dispersion-compensated standard single-mode fiber system. A MATLAB Simulinkplatform is described for simulation of the transmission performance. © 2008 Society of Photo-Optical Instrumentation Engineers. DOI: 10.1117/1.2957965 Subject terms: optical communications; modulation formats; duobinary;modulation . Paper 080186 received Mar. 8, 2008; accepted for publication Apr. 28, 2008;published online Jul. 22, 2008.