Subjective congruency between a sound effect and a switching pattern of a visual image

Abstract

The relationship between the transformation of a visual image and the pitch pattern of sound can create formal congruency between sounds and moving pictures. This effect by switching patterns of enlarging and reducing images that were combined with ascending and descending pitch scales was examined. Rating experiments showed two congruent patterns of the combination of switching and scale patterns; one was a combination of an ascending pitch scale and an enlarging image pattern, and the other a combination of a descending pitch scale and a reducing image pattern. These forms of matching might be based on a Doppler illusion. An additional pair of congruent patterns for combinations of switching and scale patterns was also found: one was a combination of an ascending pitch scale and a sliding movement from left to right, and the other a combination of a descending pitch scale and a sliding movement from right to left. These forms of matching might be based on the correspondence of a progressive sensation. Further, the formal congruency between a pitch pattern and the formal transportation can contribute to integrating auditory and visual information and to making audio-visual products more impressive.