An approach to deep understanding of speech is proposed. Deep understanding here means detection and analysis of hidden meanings like puns and metaphors. This report focuses mainly on understanding puns that involve words composed of the same number of syllables of different or similar phonemes. The analysis procedure for puns is evoked if some of the phoneme candidates of a high confidence factor in the input phoneme lattice differ from those in the phoneme sequence of legitimate words in the dictionary even if the input phoneme sequence seems understandable using low rank phonemes. The system searches possible words in the dictionary and assigns them scores according to the degree of coincidence between the phoneme sequence in the dictionary and that in the input phoneme lattice evaluating semantic conformity. In case there exist some words with high fitting scores, the system concludes that the input sentence has a hidden meaning assuming the doubly interpreted phoneme sequence to be a pivot word. [Work partly supported by Science and Technology Agency, Japan.]