The present study examines the perceptual status of allophones to examine the interaction of the phonological knowledge and speech perception. With the use of Korean bilabial stops, phonemic, allophonic and two types of low-level phonetic pairs were generated, and their perception by Korean listeners was examined. Through a categorial AXB discrimination and similarity rating tasks, it was shown that phonemes are fully discriminated, while other categories are less. However, there were significant differences in similarity ratings between the tokens for an allophone pair and two identical tokens, suggesting that allophones are also discriminated by natives. Furthermore, perceptual discrimination of phonological and phonetic categories was shown to be influenced by lexicality and partly by episodic information such as token frequency. The relevance of the results to exemplar models is discussed.