The present study examines implicit phonetic symbolism which posits that arbitrary linguistic sound is related to certain aspects of characteristics of other modalities, such as color, touch, or emotion. In consonant discrimination and lightness discrimination using Garner's speeded classification paradigm, spoken sounds (voiced/voiceless consonants) and lightness of visual stimuli (black/white squares) were systematically varied to assess cross-modal interactions. Congruent audio-visual pairs (voiced consonants and black, and between voiceless consonants and white) facilitated consonant discrimination. In lightness discrimination, no congruent facilitation or congruence effect was observed. These results indicated that cross-modal interactions in implicit phonetic symbolism can be found in correlations between linguistic spoken sounds and visual lightness.