Cognitive Semiotics Research Articles

Are Interactive Exhibits at a Science Center Cognitive Artifacts?

AbstractThe paper examines the semiotic and cognitive status of interactive exhibits at science centers, taking the Copernicus Science Center in Warsaw (CSC) as an example. Such science centers support bottom-up interactions, encouraging visitors to spontaneously explore the exhibits in various ways. We analyze one distinctive way of interaction, when young visitors ignore an exhibit’s instruction and use it as if it were a kind of a toy or machine to play with (this is particularly common with exhibits that are unfamiliar “open-ended objects”). Drawing on cognitive semiotics we describe this particular way of interacting with exhibits as the reality mode of experience, in which the user ignores an intended exhibit’s representational function. We consider whether such interactive objects can be framed as cognitive artifacts, given that standard conceptualizations of artifacts emphasize their representational function. How can we convincingly describe the process by which the cognitive function of an exhibit experienced in reality mode is constituted? In this paper we apply concept of ecological cognitive artifact and the idea of the enactive signification to these questions. We argue that exhibits experienced in reality mode do indeed perform cognitive functions, even in the absence of a representational relation. Our investigation provides insights into the cognitive functions of exhibits and contributes to the conceptualization of non-representational cognitive artifacts.

Extending the Life World: Phenomenological Triangulation Along Two Planes

Phenomenology is often mistakenly understood as both introspectionist and anthropocentric and thus as incapable of providing us with objective knowledge. While clearly wrong, such critiques force us to spell out how the life world that is given in human experience is in fact not anthropocentric and not incompatible with science. In this article we address this by adapting a recent proposal to extend the key methodological principle of cognitive semiotics, phenomenological triangulation, along two planes. The first is horizontal and concerns the dimensions of Self, Others and Things, as irreducibly interrelated dimensions of the life world. The second is vertical, and deals with the way phenomena are accessed: from a first-person (philosophical), second-person (empirical in a qualitative sense) and third-person (scientific in a quantitative sense) perspective. With each perspective, the life world becomes correspondingly extended beyond direct experience. It is thus neither static nor confining. We exemplify each step with corresponding research, also providing examples of how non-human animals and not only human beings may serve as Others, thus addressing the critique of anthropocentrism. We conclude by pointing out how, despite some theoretical differences, the focus on subjectivity and the explicit or implicit adoption of the principle of phenomenological triangulation can serve as common ground for cognitive semiotics and biosemiotics.

Polymodality of perception and its reflection in fiction text: Сognitive and semiotic perspectives

The issues related to the polymodality of perception and various forms of intermediality are widely discussed in linguistics whereas the aspect of their interrelation is seldom considered. This research is aimed at tracing the participation of various modes of perception in the process of interaction with the world, the reflection of this process in our consciousness and memory in the form of various images up to the manifestation of those images in the intermedial relations in fiction text. The study draws on cognitive semiotics, text linguistics, polymodality and intermediality. The theoretical analysis sugessts that the polymodality of perception results in the formation of synesthetic mental images represented in the phenomenon of intermediality. The empirical material of the research includes text fragments from the works by Virginia Woolf, James Joyce, Joyce Cary and Adam Thorpe containing samples of linguа-artistic and linguа-musical techniques of writing as forms of intermediality. The intermedial analysis of the empirical material shows how the polymodality of perception determines the specificity of the authors’ creative thinking represented in the choice of language means that activate musical and visual images in the readers’ minds. As a result, such fiction texts are constituted as related to a specific art whose forms and devices are used in it. The analysis has also revealed that in the process of verbal representation of musical or artistic images language exploits all forms of its semiotic signs: symbols (for naming colours, shades of light, sounds of music), indices (allusions to works of art or their authors) and icons (the use of forms and devices characteristic of other arts). The study confirms the productivity of exploring intermedial interaction in the cognitive and semiotic perspectives to demonstrate the relations between perception, conceptualization and the embodiment of the results in fiction text.

Psychologism in the study of children’s semiotic development

Abstract This essay discusses the notion of ‘semiotic development in child development’ and highlights potential concerns for ‘psychologism’ when semiotics turns into cognitive semiotics. The notion of ‘semiotic development in child development’ indicates a transdisciplinary approach involving both semiotics, the general study of meaning and signs, and child psychology. This, however, invites the criticism of committing the fallacy of psychologism. Piaget was aware of this dilemma when developing his theory of the semiotic function as a united capacity in children’s cognitive development. Sonesson’s proposal of a general definition of signs in meaning-making is suggested to, at some points, meet the dilemma with psychologism in studies of children’s semiotic development. Starting from a phenomenological point of view in semiotics and integrating Piaget’s theory on cognitive development and meaning-making meet the study of subjectivity in intersubjectivity. On the one hand, the sign as a theoretical object is not reducible to any given psychological process or processes; on the other hand, sign meaning can only exist if there are beings (consciousnesses) capable of grasping and using signs.

The cultural semiotics of Jingshen and cognitive homeostasis

Abstract Sonesson’s vision of the diversity of meaning-generation coincides well with the purview of the cultural semiotics of Jingshen, in which semiotic inquiry is envisioned to improve mental capacity, the expansion of mental/cognitive space, and ultimately the liberation of human thinking. When embracing new epistemologies for cognitive semiotics, semiotic inquiry can work well as a form of art of thinking that has the potential to maintain “cognitive homeostasis”, the tendency towards a relatively stable equilibrium that can be maintained in the holistic flux of mind, vitality, and creativity. A plausible path to this equilibrium is through a balance between the left and right hemispheres, between analytical thinking and synthetic thinking. In this sense, what we are facing is essentially a problem of “ecology of cognition” or “ecological cognition”, that is, viewing cognition from an ecological perspective or considering cognition as a form of ecological activity. The focus of cognitive semiotics nowadays must transcend not only words or other external symbols, but even the traditional sense of “humanity” needs to be reconsidered.

Tempo-spatial construction in human-law-society triangle from the perspective of cognitive semiotics

Time and space, habitually seen as the human psychological background, are increasingly being studied as the cognitive foreground for the development and evolution of matters. Via applying the corpus-based method based on the self-complied corpora of the US, the UK, the EU and Chinese data protection laws, this study examines the basic logic framework of time and space, including human psychological representation, semiotic essence, and function, seeking to present a holistic landscape of tempo-spatial construction in humanity, law and society from the perspective of cognitive semiotics. It finds that the human psychological representations in the tempo-spatial construction process contain cyclical time, linear time, place space and field space; the tempo-spatial essence is to express semiotic meanings through social dialogue and power negotiation among legal subjects in social contexts; and the tempo-spatial functions are manifested in mapping cognitive thinking mode of humanity and constructing law and society. It can be argued that legal discourse is the result of the tempo-spatial dialogue between legal subjects that depends on the interaction between society and its members and the overall evolution of its meaning interpretation; the tempo-spatial construction in human-law-society triangle is thus a typical intersemiotic operation. This study demonstrates the utility of sign of time and sign of space as a cognitive perspective in discourse construction and maps the strong interdependence between humanity, law and society.

Narrative modeling and cultural literacy in the storyworld: a quest for meaning

Abstract Following Per Aage Brandt’s “Towards a cognitive semiotics” (2011), culture as a cognitive-semiotic model allows us to look into the interrelation of cognition and signs through the act of interpretation of culture. Thus, culture characterized as habits of feeling, attention, thought, and action plays a role of modeling in the lifeworld. Regarding cultural evolution, culture also has a feature of practice of habit-taking for transformation through dialogic relations of semiosis from Peirce’s semiotic perspective. This paper argues for narrative modeling which enables habit-taking in feelings by way of analogical reasoning in a form of parable as cognitive process. A story in a form of qualia as a model for thought is embodied in narrative modeling to be enactive by a storytelling agent. Therefore, narrative modeling reveals the process of thought through habit-taking in feelings of empathy and sympathy to a feeling of an idea as a person. This leads to an act of understanding other mind, sharing the meaning and value of a story for enhancing sensibility and cultural literacy for cultural evolution. This journey of story-making and story-telling by way of narrative modeling shows a trajectory for a quest for meaning and value which will be found between “you” and “I”.

Oblique semiotics: the semiotics of the mirror and specular reflections in Lotman and Eco

AbstractNovelty, the creation of new information, has been the hallmark of Juri M. Lotman’s thought. This issue resurfaces in the discussion of his now famous article “On the semiosphere,” in which Lotman, drawing on Vernadsky, identifies the principles of symmetry, asymmetry, and enantiomorphism as pivotal aspects of the semiotic mechanism of the semiosphere. Specular phenomena and mirror reflections have not only found a prominent place in contemporary semiotic theories of different scholarly traditions – from general semiotics (Eco, Volli) to cognitive semiotics (Sonesson) and to the semiotics of culture (Lotman, Levin) – but they also nail down a key element of the inner mechanism of Lotman’s concept of the semiosphere. By using the analogy of thefacereflecting in amirror, Lotman remarks: “It is also like a face, which, wholly reflected in a mirror, is also reflected in any of its fragments, which, in this form, represents the part and yet remains similar to the whole mirror.” By capitalizing on this excerpt, this study unpacks the significance of Lotman’s idea of specular mechanisms as generators of meaning within the semiosphere.

A Cognitive Semiotic Interpretation of Chinese Culture Through Digitalized Reading

Cultural differences and geopolitical disputes severely impede the globalization process. As a method of eliminating cross-cultural misunderstandings and fostering mutual respect, it is crucial to increase cross-cultural awareness through cross-cultural education to support the world's peaceful and healthy development. This article investigates, from the perspective of cognitive semiotics, how the integration of digitization and language learning through reading can provide foreign language learners with a more immersive and stimulating learning experience. Questionnaires and interviews demonstrate that digital reading has exerted more significant learning effects on cross-cultural education than traditional paper reading. Digital reading reflects cognitive communication, cognitive subjectivity, and cognitive diversity more accurately. The integration of digital reading and cross-cultural education makes language learning more accessible and effective and can encourage students to grasp the target language while simultaneously promoting Chinese culture.

Navigating Nollywood Filmmaking in Covid-19 Pandemic through New Technology

Despite the challenges posed to film production by the lockdown during the pandemic and post-pandemic period, some filmmakers still explored new technological media to express their creative ideas about the situation in which the nation and world were enmeshed. With the need for continuity editing in the construction of visual narratives, it appears that the psychological effect of a narrative is dependent on the mode in which it is constructed to represent the ideologies to be construed. This paper examines the communicative potentials of shots in Obi Emelonye's Heart 2 Heart (2020) shot within the Pandemic era with the use of phones, coordinated and collated with the zoom communication application before editing. It highlights the aggressive psychological and emotional impact of the pandemic as x-rayed in the narrative anchoring on Kipendoff's and Urajav, Begum & Pavadai’s cognitive semiotic theories to explain the communicative impact of the film narrative and technical possibilities in this 21st century brought together through film. It is hoped that the paper would provoke film scholars and filmmakers to explore new ideas in film and the need for more study on cognitive semiotics.

Cross-modal iconicity and indexicality in the production of lexical sensory and emotional signs in Finnish Sign Language

Abstract In the present study, cross-modal (i.e., across sensory modalities such as smell and sound) iconicity (i.e., resemblance) and indexicality (i.e., contiguity) in lexical sensory and emotional signs in Finnish Sign Language will be considered from an articulatory perspective (i.e., the production of signs). Such cross-modal iconicity has not been extensively studied previously, so here, with the help of cognitive semiotics, I aim to carefully describe the cross-modal patterns observed across 118 signs, including 60 sensory signs and 58 emotional signs. The analysis is framed within the theoretical model of Semiotic Hierarchy, which entails a non-reductionist view of meaning. In addition, a pheno-methodological triangulation will be applied: phenomenology (first-person method), literature of phenomenological and semiotic descriptions (second-person perspective) and experimental findings (third-person perspective). The results of this analysis show that (a) 71 of the 118 sensory and emotional signs are cross-modally indexical, (b) only 10 of the 71 signs can be regarded as cross-modally iconic, (c) cross-modal iconicity is highly diagrammatic, (d) iconicity and indexicality are highly integrated, and (e) articulatory feedback matters in the formation of semiotic patterns. This study contributes to our understanding of cross-modal iconicity in signed languages, as well as studies in semiotic systems more generally.

Musical Meaning and the Semiotic Hierarchy: Towards a Cognitive Semiotics of Music

Research on the meaning of music has a long tradition, with approaches from several fields, but it lacks a coherent framework for interdisciplinary discussions. As a result, the notion of meaning in music is fragmented among contrasting perspectives. I propose a cognitive-semiotic approach to the analysis of the meaning evoked by music listening, adopting a framework that eludes disciplinary limitations and expands the notion of meaning to the phenomenological concept of intentionality. For this purpose, I apply Zlatev’s Semiotic Hierarchy to the experience of listening to music, analysing the diversity of meaning-making processes involved in music as distributed among several layers of experience. As a result, I propose an updated version of the Semiotic Hierarchy, clarifying its structure as based on possibilities of meaning-making, and allowing for temporality to pervade experience throughout all layers. I highlight the connectedness and simultaneity of different kinds of intentionality, resulting in the addition of the dimension of aesthetic experience – which I analyze as characterizing culture-general music listening. A key claim is that experiencing music aesthetically articulates the listener’s body in their inner sense of space and time, making them feel a sense of movement and vitality. This grounds music as a semiotic system, connecting with and fostering virtually uncountable subject-relative and culture-specific meaning-making acts.

Analyzing polysemiosis: language, gesture, and depiction in two cultural practices with sand drawing

AbstractHuman communication is by defaultpolysemiotic: it involves the spontaneous combination of two or moresemiotic systems, the most important ones beinglanguage,gesture, anddepiction. We formulate an original cognitive-semiotic framework for the analysis of polysemiosis, contrasting this with more familiar systems based on the ambiguous term “multimodality.” To be fully explicit, we developed a coding system for the analysis of polysemiotic utterances containing speech, gesture, and drawing, and implemented this in the ELAN video annotation software. We used this to analyze 23 video-recordings of sand drawing performances on Paama, Vanuatu and 20 sand stories of the Pitjantjatjara culture in Central Australia. Methodologically we used theconceptual-empirical loopof cognitive semiotics: our theoretical framework guided general considerations, such as distinguishing between the “tiers” of gesture and depiction, and the three kinds of semiotic grounds (iconic, indexical, symbolic), but the precise decisions on how to operationalize these were made only after extensive work with the material. We describe the coding system in detail and provide illustrative examples from the Paamese and Pitjantjatjara data, remarking on both similarities and differences in the polysemiosis of the two cultural practices. We conclude by summarizing the contributions of the study and point to some directions for future research.

Making meaning of an artwork: promoting expression of thoughts through artwork for patients with eating disorders

ABSTRACT The purpose of the study was to demonstrate how art could be connected to therapeutic treatment processes. The intervention took place between 2020 and 2021 at an eating disorder (ED) unit in Vaasa Central Hospital, Finland. Eating disorder patients and reference group saw an art video by artist Johanna Ketola depicting nature and mystical features. Participants then discussed thoughts stimulated by the artwork. Participants’ depictions were coded, and content analysis of the written texts was carried out by applying cognitive semiotics blending theory (CSBT). Findings show that the artwork stimulated flexibility of thought, which allowed further possibilities to develop thinking and ways of expression. Art-viewing interventions can help patients find new ways to express themselves. The study shows how CSBT, and art viewing interventions, can be used in the clinical context to analyse meaning-making processes. With such interventions, the patient can develop skills to express one's own personal condition.

Getting Fra Angelico’s splotch out: rehabilitating visual cognitive semiotics

Abstract Most contemporary approaches to meaning presume the limitation of semiotics (Didi-Huberman, Gumbrecht, Belting). The question of what kind of “semiotics” is required has not been asked. However, without some general science of meaning it is impossible to reform theory without committing past errors or ignoring progress. In the interest of reconnecting contemporary interests in “presence” to long-evolving needs, I review the ossification and decline of one theory of semiotics that serves as the tacit model rejected today. I return to problems of the nature of the sign – whether it is “digital” or “analog” and conceived as “communication” or merely “meaning.” I then reconstitute a workable visual cognitive semiotics based on phenomenological premises.

The Cognitive Processus Behind Neolithic Schematic Rock Art. Archaeological Implications and Research Hypothesis

The issue addressed in this article is essentially whether the same cognitive processes are at work for mimetic prehistoric graphic productions and schematic ones. Holocene schematic rock art is one of the main graphic expressions of European prehistory, from the Iberian peninsula to Italy. Despite its wide distribution and the incomparable insight it may provide on the functioning of prehistoric human groups and the cultural geography of the western European Neolithic, this rock art's imprecise chronology and geometric and schematic nature has often led to its exclusion from research on these societies, particularly in France. This paper proposes a study of schematic rock art from the perspective of the pragmatic and cognitive semiotics of visual culture and suggests that the production and purpose of diagrams, which compose so-called schematic rock art and which are common to all human societies, are different to those of figurative images, as is their cognitive origin. This demonstration sheds a new light on schematic rock art and the social practices it involved and invites us to rethink its coexistence with the Levantine imagery from the Spanish Levant.

“Dizziness of Freedom”: Anxiety Disorders and Metaphorical Meaning-making

ABSTRACT Would metaphors used in the context of psychotherapy by people who experience various forms of anxiety disorders differ from those used by people who experience stress? We investigated this question with the help of the Motivation & Sedimentation Model (MSM), a theory of meaning-making developed within the synthetic new discipline of cognitive semiotics. The analysis of a sample of ten transcripts of psychotherapy sessions concerning the topic of anxiety, and a comparable sample concerning stress, showed a significantly stronger proportion of conventionalized metaphors in the stress sample, and a marginally significant difference in the number of innovative metaphors in the anxiety sample. These results suggest that lived experience of an anxiety disorder or another form of maladaptive anxiety affects metaphorical meaning-making, and manifests itself in spontaneous metaphor use. Furthermore, as a result of the conceptual and the empirical investigations of the topic, we propose novel theoretical and operational definitions of the notion of metaphoricity.

Who’s talking? Cognitive semiotics in the (new media) wild

Abstract An axiom of Per Aage Brandt’s approach to conceptual blending, known colloquially as the “Aarhus model”, is that semiosis only makes sense when grounded in communicative interaction. Here we adopt that approach in relation to the reality of current, daily communication which is increasingly mediated by digital audio-visual technology platforms. We pursue this goal via a small set of case studies that explore how this technology changes and challenges social interaction and how participants exploit and adapt cognitive, embodied, technological, and semiotic resources in creating meaningful, collective, virtual spaces of joint social activity. In so doing, we expand the horizon of inquiry and contribute insights that have relevance for the new media ecology. This application of cognitive semiotic analyses of video meetings confronts the nature of “mediation” and its accomplishment, the status of “virtual spaces”, and “social presence.”

From the blue waters of the Baltic and Kattegat to the Red Sands of Burgundy. Per Aage Brandt (April 26, 1944 – November 10, 2021)

Article From the blue waters of the Baltic and Kattegat to the Red Sands of Burgundy. Per Aage Brandt (April 26, 1944 – November 10, 2021) was published on May 1, 2022 in the journal Cognitive Semiotics (volume 15, issue 1).

HOW VISION COLLABORATES WITH THOUGHT TO BRING INFORMATION INTO THE MIND

“To gaze is to think” notes Salvador Dalí. The artist’s observation is illustrative of the tradition of representing perception as analogical with thinking. Analysing perception, Arnheim, for example, claims that there is no difference between a percept and a concept inasmuch as “perception consists in fitting stimulus material with templates,” also called “visual concepts” (Arnheim 1997 [1969]: 27-28). The question that concerns us here is – how can the same visual pattern yield different concepts for different interpreters? In an attempt to answer this question the paper examines the complex dynamics involved in the perception of one of the most controversial works in the history of art – Mona Lisa by Leonardo da Vinci. To account for divergent interpretations of an image, we introduce a dichotomy of immediate and dynamical propositions of perception, a distinction inspired by the Peircean semiotic theory. Immediate propositions (IPs) of perception consist in the attribution of qualities to the perceived object as they are immediately present in the field of vision, as opposed to dynamical propositions (DPs) which bear on the qualities of the given object not necessarily perceived at the instance of perception but rather attributed to it in the past experience. By examining some various ways in which Leonardo’s image has been interpreted, we demonstrate how experience of prior perceptions interferes with the present perception to construct the meaning of a visual pattern. This paper falls within what is considered as the new paradigm of cognitive semiotics inasmuch as it deals with cognitive issues such as visual perception, the processing of visual information and the construction of meaning while taking a semiotic approach, notably, inspired by Charles Sanders Peirce. Our key endeavour consists in determining how the process of visual perception is accomplished. For a thorough consideration of this issue, it was divided into distinct questions as follows: what is information; how is it perceived; what are the constituent elements of perception; what is the role of thought in perception; and what exactly does the collaboration of vision and thought in the process of visual perception consist in? We throw a look at the functioning of the senses, and in particular the sense of vision, so as to determine the role of the senses, as well as the nature of their interaction with the mind in the process of perception. Keywords: vision, mind, thought, information processing, visual perception, meaning