When distinguishing stop consonants, Korean listeners use the fundamental frequency of the following vowel (F0) as well as the voice onset time (VOT). However, it is unclear whether the F0 and VOT of natural speech sounds play an effective role in hearing nonnative stops. In the present study, we examined whether Korean listeners perceive Japanese voiced stops, rather than Japanese voiceless stops, as being similar to Korean lenis stops depending on F0 and VOT. Native Korean listeners matched a speech sound starting with either a Japanese voiced or voiceless stop to a target. The targets were meaningful visual words starting with a lenis stop in Experiment 1 and meaningless speech sounds starting with a lenis stop in Experiment 2. Native Japanese listeners, who primarily use VOT for stop distinction, also participated in Experiment 2. The results revealed that the extent to which Korean listeners chose voiced stops was influenced consistently by F0, and the pattern of results differed between Korean and Japanese listeners. These findings suggest that even when hearing natural speech sounds, Korean listeners use F0 when judging the similarity between lenis and nonnative stops.