Islands of iconicity: a usage-based approach to capturing meaning-making processes in gesture
Abstract This study presents an example of an integrative, usage-based approach to capturing different levels of representation in co-speech gestures. We propose an annotation procedure that is demonstrated here using an extract from a multimodal corpus of Czech conversations. The 10-min extract captures 3 speakers interacting during a business meeting. The core idea behind our approach is that different aspects of the form-meaning mapping should be treated as dimensions that may interact with each other – these dimensions may include the degree of perceived iconicity based on rating, the type of form-meaning association, or the character of the form-meaning mapping (focusing on cognitive processes of which the gesture may be seen as a manifestation). Such an approach aligns with a current shift in cognitively oriented gesture studies (and beyond) away from categorical and discrete views of multimodal representation, toward more gradual and dynamic accounts.
- Research Article
4
- 10.1684/pnv.2007.0037
- Mar 1, 2007
- Gériatrie et Psychologie Neuropsychiatrie du Viellissement
Aging brings cognitive changes. Language is not immune to these changes. The use of compensation strategies may permit older adults to achieve a performance level identical to the one obtained by younger adults. This research aims to study text comprehension in aging and the reading strategies used for by older and younger adults. Kintsch's cognitive model (1988) allows the identification of different levels of representation within text treatment (linguistic form, macrostructure, microstructure and situation model) and predicts the underlying cognitive components. Eye-tracking analyses during reading permit inference about the moments of reading treatment and detection of reading strategies. Sixty highly educated participants were assessed. They were divided in two age groups (20-40 and 60-80 years old). Participants were asked to read and understand three texts constructed to highlight the features of text comprehension within each one of the different levels of text representation. The amount of detail and the necessity of updating the situation model varied for each text. Eye movements were registered by an eye-tracker (Cambridge research) during the reading process. Specific complementary tasks were administered to evaluate working memory, long-term memory, and executive functions. Variances analyses showed significantly lower performance by older adults regarding: 1) recall of the microstructure of the two texts with a high degree of detail, 2) macrostructure of the text with fewer details, and 3) performance on all tasks that evaluated cognitive components. Aging influenced treatment of levels of text representation depending on text characteristics. However, cluster analysis of the text comprehension and eye-tracker data revealed a group of older adults whose performance in reading comprehension was identical to the performance of younger adults, with the same reading profile. This result seems to show that use of compensation strategies by older adults at the onset of signs of cognitive deterioration is not necessary in reading.
- Research Article
3
- 10.1037/xhp0000496
- May 1, 2018
- Journal of experimental psychology. Human perception and performance
Selective attention refers to the ability to selectively act upon relevant information at the expense of irrelevant information. Yet, in many experimental tasks, what happens to the representation of the irrelevant information is still debated. Typically, 2 approaches to distractor processing have been suggested, namely distractor inhibition and distractor-based retrieval. However, it is also typical that both processes are hard to disentangle. For instance, in the negative priming literature (for a review Frings, Schneider, & Fox, 2015) this has been a continuous debate since the early 1980s. In the present study, we attempted to prove that both processes exist, but that they reflect distractor processing at different levels of representation. Distractor inhibition impacts stimulus representation, whereas distractor-based retrieval impacts mainly motor processes. We investigated both processes in a distractor-priming task, which enables an independent measurement of both processes. For our argument that both processes impact different levels of distractor representation, we estimated the exponential parameter (τ) and Gaussian components (μ, σ) of the exponential Gaussian reaction-time (RT) distribution, which have previously been used to independently test the effects of cognitive and motor processes (e.g., Moutsopoulou & Waszak, 2012). The distractor-based retrieval effect was evident for the Gaussian component, which is typically discussed as reflecting motor processes, but not for the exponential parameter, whereas the inhibition component was evident for the exponential parameter, which is typically discussed as reflecting cognitive processes, but not for the Gaussian parameter. (PsycINFO Database Record
- Research Article
2
- 10.1080/0163853x.2025.2494964
- May 2, 2025
- Discourse Processes
Propositional density, a semantic index, calculates the number of idea units within a sentence. While research has shown that propositional density can influence comprehension, its effects on different levels of memory representation, namely, the surface form, textbase, and situation model, remain unexplored. The current study examined how varying levels of propositional density of the context on which target sentences were embedded affected memory at these different levels of representation. A sample of 120 native English speakers read historical texts with either low or high propositional density and then completed recognition tasks targeting each level. Our results indicated that lower propositional density improved memory for the surface form, while higher density enhanced memory for the textbase. Propositional density had no meaningful effect on the situation model level. These findings suggest a complex relationship between propositional density and memory: Higher density appears to impair memory at the surface form level but may create a desirable difficulty to strengthen memory at the textbase level. However, comprehension, as reflected by the situation model level, remains largely unaffected, suggesting that global comprehension relies more on coherence-building and prior knowledge integration than on local semantic density. These findings inform models of discourse processing and have implications for text design, instructional strategies, and cognitive load management in reading comprehension.
- Research Article
51
- 10.1016/j.jml.2018.05.008
- Jun 11, 2018
- Journal of Memory and Language
Patterns of forgetting
- Discussion
2
- 10.3389/fpsyg.2012.00405
- Oct 18, 2012
- Frontiers in Psychology
Prediction as Internal Simulation: Taking Chances in What to Do Next
- Research Article
186
- 10.1162/jocn_a_01465
- Jan 1, 2020
- Journal of cognitive neuroscience
It has been proposed that hierarchical prediction is a fundamental computational principle underlying neurocognitive processing. Here, we ask whether the brain engages distinct neurocognitive mechanisms in response to inputs that fulfill versus violate strong predictions at different levels of representation during language comprehension. Participants read three-sentence scenarios in which the third sentence constrained for a broad event structure, for example, {Agent caution animate-Patient}. High constraint contexts additionally constrained for a specific event/lexical item, for example, a two-sentence context about a beach, lifeguards, and sharks constrained for the event, {Lifeguards cautioned Swimmers}, and the specific lexical item swimmers. Low constraint contexts did not constrain for any specific event/lexical item. We measured ERPs on critical nouns that fulfilled and/or violated each of these constraints. We found clear, dissociable effects to fulfilled semantic predictions (a reduced N400), to event/lexical prediction violations (an increased late frontal positivity), and to event structure/animacy prediction violations (an increased late posterior positivity/P600). We argue that the late frontal positivity reflects a large change in activity associated with successfully updating the comprehender's current situation model with new unpredicted information. We suggest that the late posterior positivity/P600 is triggered when the comprehender detects a conflict between the input and her model of the communicator and communicative environment. This leads to an initial failure to incorporate the unpredicted input into the situation model, which may be followed by second-pass attempts to make sense of the discourse through reanalysis, repair, or reinterpretation. Together, these findings provide strong evidence that confirmed and violated predictions at different levels of representation manifest as distinct spatiotemporal neural signatures.
- Research Article
14
- 10.1162/coli_a_00420
- Dec 7, 2021
- Computational Linguistics
Natural Language Processing and Computational Linguistics
- Research Article
326
- 10.1080/0950069032000070306
- Nov 1, 2003
- International Journal of Science Education
Chemistry is commonly portrayed at three different levels of representation – macroscopic, submicroscopic and symbolic – that combine to enrich the explanations of chemical concepts. In this article, we examine the use of submicroscopic and symbolic representations in chemical explanations and ascertain how they provide meaning. Of specific interest is the development of students' levels of understanding, conceived as instrumental (knowing how) and relational (knowing why) understanding, as a result of regular Grade 11 chemistry lessons using analogical, anthropomorphic, relational, problem‐based, and model‐based explanations. Examples of both teachers' and students' dialogue are used to illustrate how submicroscopic and symbolic representations are manifested in their explanations of observed chemical phenomena. The data in this research indicated that effective learning at a relational level of understanding requires simultaneous use of submicroscopic and symbolic representations in chemical explanations. Representations are used to help the learner learn; however, the research findings showed that students do not always understand the role of the representation that is assumed by the teacher.
- Research Article
4
- 10.1016/s0187-893x(17)30099-x
- Jan 1, 2012
- Educación Química
O experimento da gota salina e os níveis de representação em química
- Book Chapter
5
- 10.1007/978-3-658-03104-6_4
- Nov 8, 2013
A sound understanding of place value is a crucial competence for successful mathematics learning and the flexible handling of bigger numbers. Firstly, the contribution presents in detail the specific meaning of place value as a fundamental idea as well as results of empirical studies with respect to low achievers. For diagnosing the competencies of low achievers dealing with numbers up to thousand, specific test items were designed that cover different levels of representation and that can be used in interviews or as a paper and pencil test. The contribution presents results of a small case study with fifth- and sixth-graders with special needs using the developed test instrument followed by consequences and indications for classrooms activities. These indications touch aspects like diagnosis with respect to basic competencies, the selection of adequate problems or linking different levels of representation. For low achieving students the topic of place value and its didactical implementation should take a more prominent role for didactical proposals and classroom practice.
- Research Article
- 10.4204/eptcs.11.4
- Nov 30, 2009
- Electronic Proceedings in Theoretical Computer Science
Understanding the behaviour of biological systems requires a complex setting of in vitro and in vivo experiments, which attracts high costs in terms of time and resources. The use of mathematical models allows researchers to perform computerised simulations of biological systems, which are called in silico experiments, to attain important insights and predictions about the system behaviour with a considerably lower cost. Computer visualisation is an important part of this approach, since it provides a realistic representation of the system behaviour. We define a formal methodology to model biological systems using different levels of representation: a purely formal representation, which we call molecular level, models the biochemical dynamics of the system; visualisation-oriented representations, which we call visual levels, provide views of the biological system at a higher level of organisation and are equipped with the necessary spatial information to generate the appropriate visualisation. We choose Spatial CLS, a formal language belonging to the class of Calculi of Looping Sequences, as the formalism for modelling all representation levels. We illustrate our approach using the budding yeast cell cycle as a case study.
- Research Article
7
- 10.1109/tnnls.2021.3083759
- Nov 1, 2022
- IEEE Transactions on Neural Networks and Learning Systems
It is very challenging for machine learning methods to reach the goal of general-purpose learning since there are so many complicated situations in different tasks. The learning methods need to generate flexible internal representations for all scenarios met before. The hierarchical internal representation is considered as an efficient way to build such flexible representations. By hierarchy, we mean important local features in the input can be combined to form higher level features with more context. In this work, we analyze how our proposed general-purpose learning framework-the developmental network-2 (DN-2)-autonomously generates internal hierarchy with new mechanisms. Specifically, DN-2 incrementally allocates neuronal resources to different levels of representation during learning instead of handcrafting static boundaries among different levels of representation. We present the mathematical proof to demonstrate that optimal properties in terms of maximum likelihood (ML) are established under the conditions of limited learning experience and resources. The phoneme recognition and real-world visual navigation experiments that are of different modalities and include many different situations are designed to investigate general-purpose learning capability of DN-2. The experimental results show that DN-2 successfully learns different tasks. The formed internal hierarchical representations focus on important features, and the invariant abstract arise from optimal internal representations. We believe that DN-2 is in the right way toward fully autonomous learning.
- Research Article
34
- 10.1016/s0885-2014(98)90023-5
- Jan 1, 1998
- Cognitive Development
Group collaboration effects and the explicitness of children's knowledge
- Research Article
25
- 10.1021/acs.jchemed.5b00599
- Nov 3, 2015
- Journal of Chemical Education
Computer-based simulations can help students visualize chemical representations and understand chemistry concepts, but simulations at different levels of representation may vary in effectiveness on student learning. This study investigated the influence of computer activities that simulate chemical reactions at different levels of representation in students’ conceptual understanding of the particulate nature of matter (PNM). The participants included 170 second semester general chemistry students and were divided into two groups: one interacting with computer simulations at the particulate level and one at the macroscopic level. Students’ understanding of the PNM was measured using the Particulate Nature of Matter Assessment. In addition, factor analysis was performed to detect latent concepts in the instrument. Results showed that dynamic simulations at the particulate level helped students understand the PNM in chemistry involving particle motion.
- Research Article
24
- 10.1121/1.1394220
- Sep 1, 2001
- The Journal of the Acoustical Society of America
A model is offered here to address an asymmetry of cueing in signal detection [Hafter et al. (1992)] where the effect of frequency uncertainty on the detection of a randomly chosen tone was ameliorated by cueing with a sequence of its harmonics, but detection of a randomly chosen sequence of harmonics was not improved by cueing with their fundamental. The model proposes that signal detection can be based on various levels of neural representation that, for the case at hand, refer to levels organized either by frequency or by complex pitch. Experiments offered to test the model used three-tone complexes for both cues and signals. These stimuli consisted of either three randomly chosen frequencies or three randomly chosen harmonics (from the set 2 f1 to 7 f1) of a randomly chosen fundamental. Support for the idea of cueing and detection at different levels of representation was found in higher performance with uncued detection of harmonic complexes relative to that found with complexes of unrelated tones and by successful cueing of each type of information with cues created to remove uncertainty about the relevant information. A final comparison suggests independence of performance (presumably of the limiting noise) at each of the putative levels of representation.
- Ask R Discovery
- Chat PDF
AI summaries and top papers from 250M+ research sources.