Multimodal theories of recognition and their relation to Molyneux's question

Abstract

In the seventeenth century, William Molyneux posed his now famous question to John Locke, a question about the contents of human sensory experience, the relation between cross-modal perceptions, and the origins of sensory content more generally. Locke (as cited by Molyneux's Question, n.d.1) paraphrased Molyneux's Question as follows: “Suppose a man born blind, and now adult, and taught by his touch to distinguish between a Cube, and a Sphere of the same metal, and nighly of the same bigness, so as to tell, when he felt one and t'other, which is the Cube, which the Sphere. Suppose then the Cube and Sphere placed on a Table, and the Blind Man to be made to see. Quaere, whether by his sight, before he touched them, he could now distinguish, and tell, which is the Globe, which the Cube (Locke, 1694/1979).” Since then, empiricists and nativists have argued about these issues. Empiricists, who take all sensory content to be derived from experience, have traditionally argued for the negative conclusion—a blind man would not differentiate the cube from the sphere on first sight. Nativists, who believe that we are born with at least some core conceptual resources, have often championed the positive view: A transfer of content from haptics to vision would occur, thus enabling the blind man to differentiate between the two shapes. Today, a burgeoning interest in multimodal processing has again resurrected Molyneux's Question. Sometimes authors have identified and addressed this set of problems by name (Held et al., 2011; Connolly, 2013, 2014). Other researchers raise very similar questions but pose them in a modern guise. They ask: How we should interpret data from multimodal experiments in the behavioral and neural sciences (e.g., Fowler, 2004; Wallace et al., 2006; Connolly, 2014)? I show why the traditional answers, “yes” by Nativist and “no” by the Empiricist, are too simple2. Specifically, I begin by arguing that at least two nativist theories can predict a “no” answer to Molyneux's Question. I then go on to argue that theories promoting a common coding scheme across modalities are the nativist accounts that predict an affirmative answer. This is important as many influential accounts of multisensory perception in the field of speech recognition and elsewhere shares these features and should therefore be explored in light of these issues. A framework in the field of Psychology known as direct perception, for instance, argues that speech and other phenomena are recognized “directly” as dynamic-gestural events. This is true regardless of whether cues are obtained via the auditory, visual, or even tactile modalities (Rosenblum et al., 1997; Fowler, 2004). Crucially, the information accessed by the perceiver is amodal and hence not tied to a specific modality, such as the auditory. I also illustrate how these issues make predictions for theories of speech perception and our interpretation of phenomena such as the McGurk effect (McGurk and MacDonald, 1976). Finally, I describe how ongoing developments in behavioral and reaction-time methods can be used to distinguish the different nativist and empiricist accounts of perception discussed in this paper.