Non-native speakers (NNSs) of English must often communicate in English even if they share an L1. This study investigated how the talker–listener match with respect to language background affects L2 intelligibility. Two non-native speakers of English with an L1 of Chinese and two monolingual English speakers were recorded in English reading sentences and producing spontaneous speech. Listeners were 32 monolingual native speakers of English, 18 NNS of English with an L1 of Chinese and 14 NNS of English with an L1 other than Chinese. Results showed patterns of objective intelligibility and subjective comprehensibility ratings across listener groups that were strongly influenced by the relationship between the language backgrounds of talker and listener. As expected, native English listeners showed a preference for the native English talkers over either of the Chinese talkers. In contrast, the Chinese listeners found one of the Chinese talkers most intelligible, and the non-Chinese non-native listeners found one of the Chinese talkers equally intelligible to one of the native talkers. From these patterns of intelligibility, we conclude that, provided the talker is above a certain threshold of intelligibility, if talker and listener both share an interlanguage or are both native speakers, intelligibility is maximized.