The Language of Vision*

Patrick Cavanagh

doi:10.1177/0301006621991491

Abstract

The descriptions of surfaces, objects, and events computed by visual processes are not solely for consumption in the visual system but are meant to be passed on to other brain centers. Clearly, the description of the visual scene cannot be sent in its entirety, like a picture or movie, to other centers, as that would require that each of them have their own visual system to decode the description. Some very compressed, annotated, or labeled version must be constructed that can be passed on in a format that other centers—memory, language, planning—can understand. If this is a “visual language,” what is its grammar? In a first pass, we see, among other things, differences in processing of visual “nouns,” visual “verbs,” and visual “prepositions.” Then we look at recursion and errors of visual grammar. Finally, the possibility of a visual language also raises the question of the acquisition of its grammar from the visual environment and the chance that this acquisition process was borrowed and adapted for spoken language.

Highlights

The visual system takes up a very large part of the brain—some say that as much as 30% of prime cortical real estate is specialized for visual processing in humans (Orban et al, 2004)
The area where the adaptation had an effect moved when the eyes moved. These results strongly suggested that at least this form of causality was determined in the visual system and was not a cognitive inference
Is there a language of vision? Clearly, even if there is, much of vision has nothing to do with the compressed messages the visual system would exchange with central processes

Summary

Introduction

The visual system takes up a very large part of the brain—some say that as much as 30% of prime cortical real estate is specialized for visual processing in humans (Orban et al, 2004). The messages that vision sends to other modules are not likely to be pictures or movies of what is happening If they were, the receiving area would have to have its own visual system to interpret the pictures. Both Barrs and Dehaene proposed that the content of the billboard is the content of consciousness This is a plausible choice as attention and awareness have in common a severely limited capacity.

Displacement

Findings

Conclusions