In diotic cocktail-party listening tasks, performance decreases dramatically when the number of talkers increases from two (one target and one masker) to three (one target and two maskers). In this experiment, listeners were asked to extract information from a Coordinate Response Measure (CRM) target phrase that was masked by one or two interfering utterances. The maskers were either semantically similar to the target (i.e., CRM phrases with different call signs than the target phrase) or semantically different from the target (time-reversed, contextually dissimilar, or in a different language than the target phrase). The two-talker results showed that performance was much worse when the interfering phrase was semantically similar to the target than when it was semantically different. However, given that one interfering talker was similar to the target, the contextual similarity of the second interfering talker had very little effect on performance; in other words, target intelligibility in the presence of two CRM maskers was no worse than with one CRM masker and one time-reversed or foreign-language masker. These results suggest that the disruption of speech segregation caused by the addition of a second interfering talker cannot be explained solely by contextual similarity between the target and masking speech.