Task-irrelevant speech is known to cause disruption of short-term memory, either through specific interference with encoding processes (e.g., seriation, semantic processing) or by diverting attention from the focal task. Previous studies found that semantically related background speech can induce memory intrusions of words that were not part of the to-be-remembered list. While these findings suggest false memories due to semantic interference, the present study aims to test whether the presence of task-irrelevant speech affects the susceptibility to memory intrusions also in the absence of semantic interference. Therefore, incomprehensible to-be-ignored speech was presented during encoding of semantically related words. It was found across three experiments that incomprehensible changing-state speech increased the rate of false memories of non-presented but semantically related words in a subsequent recognition (Experiments 1 and 2) or recall test (Experiment 3), compared with white noise or steady-state speech. The findings indicate that speech interfered with serial-order processing of the to-be-remembered items, thus urging participants to rely on semantic information to encode and retrieve the presented words. While a focus on semantic information enabled participants to correctly recollect the majority of presented words, it most likely also increased the proportion of false memories of words with semantic associations to the presented words both in recall and recognition tests. In all three experiments, the presence of an auditory deviant in background speech did not increase the rate of false memories, suggesting that attentional capture alone does not necessarily induce source monitoring errors. However, Experiment 3 revealed that an increase in visual task-encoding load attenuated the changing-state effect on the production of false memories. This indicates that the semantic organisation processes initiated as a result of the loss of order information in case of changing-state speech may be sensitive to attentional control.