Interactional Context in Graphical Communication

Abstract

Interactional Context in Graphical Communication Patrick G. T. Healey Information, Media and Communication Research Group Department of Computer Science, Queen Mary University of London. Simon Garrod, Nicholas Fay HCRC Department of Psychology, University of Glasgow. John Lee and Jon Oberlander HCRC and Department of Architecture, Division of Informatics, University of Edinburgh. Abstract A body of empirical evidence indicates that interac- tional context has a key influence on the form and interpretation of language. This paper reviews a se- ries of experiments which indicate that interactional context also plays a key role in the interpretation of drawings and sketches. Two graphical communi- cation tasks, analogous to definite reference tasks, are described. The findings from these tasks show significant parallels between the mechanisms of co- ordination in graphical dialogue and natural lan- guage dialogue. Specifically; the coordination of graphical representation types by ‘dialogue’ partic- ipants, the contraction of recurrent ‘graphical re- ferring expressions’, e↵ects of direct interaction on the use of abstract drawings, and the development of community-specific graphical conventions. Interactional Context in Dialogue Conversation is a, if not the, key context of under- standing for language. People’s use of language to represent objects, events and situations is sensitive to, amongst other things; who they are speaking to, the mutual availability of referents, the history of their conversation and their (dis)joint membership of cultural and linguistic sub-communities (Hymes, 1972; Clark, 1998). Evidence for the direct influence of interactional context on interpretation and under- standing comes from a variety of sources (see Krauss and Fussell, 1996, for a review). One example is provided by work on the Collaborative Model of di- alogue (Schober and Clark, 1989; Clark and Wilkes- Gibbs, 1986). Wilkes-Gibbs and Clark (1992) have shown that full understanding of referring expres- sions depends on the degree of active participation in conversation by speaker and addressees. Non-active participants in a conversation, such as passive side- participants, overhearers, or bystanders, have more difficulty in interpreting referring expressions than active participants. This is observed even when, in gross informational terms, they are equivalent to ac- tive participants. A second example of the influence of interactional context comes from studies of conceptual and lin- guistic co-ordination in dialogue. Garrod and An- derson (1987) have shown that conversational part- ners tend to match or ‘entrain’ on the form and in- terpretation of utterances during interaction. Where several types of semantically distinct referring ex- pressions are possible for describing a location, peo- ple show a strong preference for matching the type of expression used by their conversational partner. Brannigan, Pickering and Cleland (2000) have ob- served similar entrainment e↵ects with syntax. Gar- rod and Anderson (1987) argue that these dialogue phenomena reflect the operation of a basic dialogue co-ordination mechanism which simplifies the pro- cesses of production and comprehension in interac- tion. Interactional Context in Graphical Dialogue Intuitively, it might be supposed that graphical rep- resentations would be less sensitive to interactional context. One reason for this is that the production and use of drawings and sketches is normally treated, and analysed, as an activity more akin to monologue than dialogue (cf. Scaife and Rogers, 1996). There is evidence, however, that this underestimates both the actual and potential use of drawing activities as a mode of interaction. Anecdotally, drawings are of- ten incrementally produced and modified as part of a conversational exchange. For example, sketch maps and explanatory diagrams form a familiar extension of many routine conversations. van Sommers (1984) provides evidence from a questionnaire study that approximately half of rou- tine, non-work, drawing activities take place with or for an audience. Although van Sommers does not report how often these interactions involve di- rect graphical exchanges, his findings demonstrate the variety of interactional contexts in which draw- ing occurs. The most frequently cited category is the production of sketch maps of a local area, either as part of an explanation or in order to give directions. The second most frequently cited category relates to activities with children including; games and amuse- ments, teaching or helping with homework and help- ing children learn to draw. Additional categories of ‘public’ drawing include; sketching of hair, makeup and clothing, sketching house plans, drawing to ex- press feelings, defacing pictures and drawing people.