Why Are Analogue Graphics and Natural Language Both Needed in HCI?

Abstract

The combined use of language and analogue graphics for the expression of information probably is as old as language itself. The paper addresses the question why we need both the expressions of natural language and analogue graphics for the representation of information. It is argued that analogue graphics and natural language have the complementary expressive virtues of specificity and focus, respectively. Their corresponding lack of focus and specificity, respectively, explain why (a) both have developed a number of mechanisms for coping with these deficiencies and (b) why their combination may often have superior expressive power. Since specificity follows from the analogue character of analogue graphics rather than from their graphic character, analogue sound and touch representations are analysed to explore whether results from the analysis of analogue graphics and their complementarity with natural language can be transferred to other analogue modalities of expression. The paper exemplifies the comparatively new field of Modality Theory.