Transmission line‐type electromagnetic (EM) methods for estimating soil volumetric water content (θv) have advanced significantly in recent years, with many sensing systems now available. To estimate θv, EM systems make use of the dependence of soil dielectric permittivity on θv. However, a standard method for characterizing and comparing EM system measurement capability has not been established. Our objective was to evaluate the permittivity measurement ability of seven different EM sensing systems using readily available media. Sensing system outputs were converted to real permittivity (ε′) values and compared with reference ε′ values in lossless and lossy dielectric liquids under four different test conditions: nonrelaxing and nonconducting (NR‐NC), relaxing and nonconducting (R‐NC), nonrelaxing and electrically conducting (NR‐C), and temperature variation in NR‐NC. The higher frequency broadband sensing systems, consisting of two time domain reflectometry (TDR) systems and one time domain transmissometry (TDT) system, deviated from a network analyzer by less than ±2.94 ε′ units across a ε′ range of 12.7 to 78.5 in NR‐NC media. Two lower frequency impedance sensing systems deviated from the network analyzer by less than ±3.94 ε′ units across a ε′ range of 12.7 to 36.5 in the same media. Measurement of ε′ using higher frequency broadband sensing systems was impacted more by bulk electrical conductivity (σb) and temperature (T) than by dielectric relaxation. Imaginary permittivity values (due only to relaxation, ε″rel) of up to 14.5 in R‐NC media resulted in ε′ errors of ±0.511, whereas σb values ranging from 0 to 2 dS m−1 in NR‐C media resulted in ε′ errors of ±2.69 and T values ranging from 5 to 40°C resulted in ε′ errors of ±4.89. Determination of ε′ using lower frequency sensing systems—including one transmission line oscillator, two impedance probes, and one capacitance probe—was impacted more by σb than by T and ε″rel For the lower frequency sensors (and the same ranges of σb, T, and εrel), σb resulted in ε′ errors of ±111, T resulted in ε′ errors of ±6.59, and ε″rel resulted in ε′ errors of ±3.28. The effects of ε″rel, σb, and T on permittivity measurement accuracy is to a large extent dependent on measurement frequency, with higher frequency broadband sensing systems generally yielding better measurements.