AbstractThis article presents a rigorous full‐wave modal analysis of a rectangular waveguide with rounded sides. The plane wave series expansion of Bessel function is employed to obtain the closed‐form modal functions for both TE and TM modes. Characteristic equations for calculating the eigenvalues (cutoff wavenumbers) of both TE and TM modes are derived. Numerical results are presented for the cutoff wavenumbers of the dominant mode as well as the higher‐order modes supported in the rounded‐side rectangular waveguide. © 2002 Wiley Periodicals, Inc. Microwave Opt Technol Lett 33: 365–368, 2002; Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/mop.10323