Abstract
This article draws a perceptual approach to audio- visual mapping. Clearly perceivable cause and effect relationships can be problematic if one desires the audience to experience the music. Indeed perception would bias those sonic qualities that fit previous concepts of causation, subordinating other sonic qualities, which may form the relations between the sounds themselves. The question is, how can an audio-visual mapping produce a sense of causation, and simultaneously confound the actual cause-effect relationships. We call this a fungible audio-visual mapping. Our aim here is to glean its constitution and aspect. We will report a study, which draws upon methods from experimental psychology to inform audio-visual instrument design and composition. The participants are shown several audio-visual mapping prototypes, after which we pose quantitative and qualitative questions regarding their sense of causation, and their sense of understanding the cause-effect relationships. The study shows that a fungible mapping requires both synchronized and seemingly non-related components - sufficient complexity to be confusing. As the specific cause- effect concepts remain inconclusive, the sense of causation embraces the whole.
Highlights
Perceivable cause-effect relationships are as problematic for the music as disruptive visual changes
When we focus on a goal such as accomplishing a task, perception simplifies the information according to presuppositions; the gestaltist principles are a major example
The high rating for Transparency shows that cause and effect concepts were conclusive, despite the gaps when the sphere was invisible. This is well known in gestaltist psychology and neuroscience: inconsistency does not impede the formation of conclusive cause and effect concepts, or conclusive audiovisual objects – if we extend Kubovy and Schutz’ terminology
Summary
Perceivable cause-effect relationships are as problematic for the music as disruptive visual changes. The question is whether, and how, an audiovisual mapping can foster a sense of causation, and simultaneously confound the cause and effect relationships so that they remain inconclusive. We call this a fungible audio-visual mapping. In our mentioned publication (Sa, 2013) we substantiate the fungible mapping as a principle for instrument design and composition It goes together with two other principles: to threshold control and unpredictability so as to potentiate sonic expression; and to dispense with disruptive visual changes, which would automatically attract attention, subordinating audition. On the other hand, we can permeate the broad membrane of complexity formed of sensory information, when we focus on the experience itself
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
