Phenolic rich extracts from edible plants exhibit antimicrobial ability against various pathogens, however, their effects on probiotic bacteria have been much less investigated. This study, therefore, investigated and compared the effects of phenolic rich extracts from six dietary spices and medicinal herbs (Padang cassia, Chinese cassia, oregano, Japanese knotweed, pomegranate peel and clove) on five common food-borne pathogenic bacteria, including Bacillus cereus, Escherichia coli, Salmonella enterica subsp. enterica serovar typhimurium, Shigella flexneri and Staphylococcus aureus, and five selected lactic acid bacteria (LAB), including Lactobacillus (L.) acidophilus, L. delbrueckii subsp. bulgaricus, L. casei, L. plantarum and L. rhamnosus. Antimicrobial tests, included diameter of inhibition zone (DIZ), minimal inhibitory concentration (MIC) and minimal bactericidal concentration (MBC), were performed. Considering the food-borne pathogenic bacteria, the DIZ values ranged from 9.33 to 26 mm, while the DIZ for LAB were overall less than 8 mm. From the results of MIC, Japanese knotweed, pomegranate peel and clove extracts showed stronger inhibitory effects on Shigella flexneri and Staphylococcus aureus, about 313–2500 μg/mL, while the MIC values of phenolic rich extracts on other bacteria, including LAB, were generally more than 2500 μg/mL. Overall, the phenolic rich extracts possessed evident antibacterial ability against food-borne pathogenic bacteria, but not against LAB, suggesting that these phenolic rich extracts did not inhibit the growth of probiotic bacteria. Furthermore, the main phenolic compounds of selected edible plants, including carvacrol, trans-cinnamaldehyde, p-coumaric acid, eugenol, gallic acid and rosmarinic acid, were confirmed to possess strong antibacterial effect against the selected food-borne pathogenic bacteria, suggesting that they were mainly responsible for the antibacterial activity of phenolic rich extracts against pathogenic bacteria. In conclusion, natural phenolic rich extracts exhibit differential effects on food-borne pathogenic bacteria and probiotic LAB, providing a scientific basis for their application in foods not only to extend the shelf-life by controlling bacterial contamination but also to enhance health benefits of foods.