This study investigates the interference of L1 phonology with L2 speech perception, focusing on the identification of the English fricatives /f/ and /h/ by Japanese listeners. In Japanese, the contrast of /f/ vs /h/ is lost only in the /_u/ environment by neutralization (/fu/ and /hu/ > [ɸu]). This makes food vs who’d extremely difficult for Japanese to discern while fee vs he poses no problem. Perceptual experiments demonstrated that while /f/ and /h/ were accurately identifiable when isolated from non-/u/ syllables [e.g. f(a) vs h(a)], those isolated from /_u/ were rather hard [f(u) vs h(u)] and became even harder when presented with the vowel [fu vs hu]. Such Cs alone [f(u)/h(u)] were easier than the whole CVs [fu/hu] from which those Cs were excised, contrary to our general expectations. A cross-splicing experiment further revealed it was not the acoustics of coarticulated fricatives but the presentation of /u/ that made the identification difficult. The basic phonetic process of f/h was debilitated in fu/hu, where the L1 neutralization applied, cancelling the contrast and the need for attunement, which was then transferred to L2. It was argued that the L1-L2 sound correspondence can be affected by the knowledge of L1 phonological processes.