Investigating object orientation effects across 18 languages

ABSTRACTIn the sentence–picture verification (SPV) task, people read sentences implying the shape/size/colour/orientation of objects. They then verify whether pictured objects, which either match or mismatch the implied visual information mentioned in the sentence. Faster verification times on matching trials (match advantage) are considered supportive to the notion that readers perform mental simulations during sentence comprehension. This study advances this work by applying a within-subjects design to the SPV-task, enabling us to directly address the strength of and correlation between the match advantages for the properties shape, size, colour, and orientation. Results showed varying match advantages with colour showing the strongest effect, and no match advantage for orientation. Shape, size, and colour were significantly correlated, whereas there were no significant correlations with orientation. These findings suggest that interpretations of match advantages could benefit from a re-evaluation of mental simulation accountsby distinguishing between intrinsic (shape, size, and colour) and extrinsic (orientation) object properties.

Language comprehenders have been arguing to mentally represent the implied orientation of objects. However, compared to the effects of shape, size, and color, the effect of orientation is rather small. We examined a potential explanation for the relatively low magnitude of the orientation effect: Object size moderates the orientation effect. Theoretical considerations led us to predict a smaller orientation effect for small objects than for large objects in a sentence-picture verification task. We furthermore investigated whether this pattern generalizes across languages (Chinese, Dutch, and English) and tasks (picture-naming task). The results of the verification task show an orientation effect overall, which is not moderated by object size (contrary to our hypothesis) and language (consistent with our hypothesis). Meanwhile, the preregistered picture-picture verification task showed the predicted interaction between object size and orientation effect. We conducted exploratory analyses to address additional questions.

Research suggests that language comprehenders simulate visual features such as shape during language comprehension. In sentence-picture verification tasks, whenever pictures match the shape or orientation implied by the previous sentence, responses are faster than when the pictures mismatch implied visual aspects. However, mixed results have been demonstrated when the sentence-picture paradigm was applied to color (Connell, Cognition, 102(3), 476–485, 2007; Zwaan & Pecher, PLOS ONE, 7(12), e51382, 2012). One of the aims of the current investigation was to resolve this issue. This was accomplished by conceptually replicating the original study on color, using the same paradigm but a different stimulus set. The second goal of this study was to assess how much perceptual information is included in a mental simulation. We examined this by reducing color saturation, a manipulation that does not sacrifice object identifiability. If reduction of one aspect of color does not alter the match effect, it would suggest that not all perceptual information is relevant for a mental simulation. Our results did not support this: We found a match advantage when objects were shown at normal levels of saturation, but this match advantage disappeared when saturation was reduced, yet still aided in object recognition compared to when color was entirely removed. Taken together, these results clearly show a strong match effect for color, and the perceptual richness of mental simulations during language comprehension.

Many studies have shown that mental simulation may occur during language comprehension. Supporting evidence is derived from the matching effects in the sentence-picture verification (SPV) task often used to assess mental simulations of object properties, such as size, orientation, and shape. However, mixed results have been obtained regarding object colour, with researchers reporting matching or mismatching effects. This study investigated the impact of colour information clarity within sentences on the process of mental simulation during language comprehension. Employing the SPV task and using novel objects, we examined whether there is a mental simulation of colour after excluding typical/atypical colour bias and how varying levels of colour information clarity in sentences influence the emergence of matching effects at different stages of comprehension. To address these issues, we conducted two experiments. In Experiment 1, the participants read normal sentences and subsequently engaged in picture verification with a novel object after a 500ms delay. In Experiment 2, the participants encountered sentences containing both clear and unclear colour information and, after either a 0ms or 1500ms interval, completed picture verification tasks with a novel object. Null effects were found in the 500ms condition for normal sentences and the 0ms condition for unclear colour information sentences. A mismatching effect appeared in the 0ms condition after clear colour information sentences, and a matching effect appeared in the 1500ms condition for all sentences. The results indicated that after excluding colour bias, the participants still formed mental simulations of colour during language comprehension. Our results also indicated that ongoing colour simulation with time pressure impacted the participant responses. The participants ignored unclear colour information under time pressure, but without time pressure, they constructed simulations that were as detailed as possible, regardless of whether the implicit colour information in the sentence was clear.

Embodied theories of language comprehension propose that readers construct a mental simulation of described objects that contains perceptual characteristics of their real-world referents. The present study is the first to investigate directly whether implied object size is mentally simulated during sentence comprehension and to study the potential influence of developmental factors on mental simulation by comparing adults' and children's mental simulation processing. Participants performed a sentence-picture verification task in which they read a sentence that implied a large or a small size for an object and then saw a picture of the object that matched or mismatched the implied size. Responses to pictures were faster when implied size and pictured size matched, suggesting that readers activated perceptual information on object size during sentence comprehension. The magnitude of the match effect was equal across age groups. The results contribute to refining and advancing knowledge with respect to the nature of mental simulations.

Comparing the Ability of Perceptual Simulation in Children with Developmental Dyslexia and Typical Children: An Embodied Cognition View

Teaching motivation and strategies to improve mental rotation abilities

Pictures of objects are verified faster when they match the implied orientation, shape, and color in a sentence-picture verification task, suggesting that people mentally simulate these features during language comprehension. Previous studies had an unintended correlation between match status and the required response, which may have influenced participants’ responses by eliciting strategic use of this correlation. We removed this correlation by including color-matching filler trials and investigated if the color-match effect was still obtained. In both a native sample (Experiment 1) and a non-native sample (Experiment 2), we found strong evidence for a color-match advantage on median reaction time and error rates. Our results are consistent with the view that color is automatically simulated during language comprehension as predicted by the grounded cognition framework.

Does experience with spatial school subjects favour girls' mental rotation performance?

Perceptual representations in language comprehension were examined using sentence-picture verification tasks. However, concerns have been raised regarding the suitability of concrete pictures for representing abstract concepts compared to image-schematic diagrams. To assess the perceptual representations of spatial and abstract domains in both first language (L1) and second language (L2) processing, the study tests bilingual speakers' mental imagery on the basis of the simulation-based L1 comprehension model and proposes a simulation-based L2 comprehension model, supported by empirical evidence from an innovative sentence-diagram verification paradigm. 41 adult L1 Mandarin Chinese speakers participated in the study. 21 participants completed the Chinese sentence-diagram verification task (Experiment 1), while 20 participants completed the translation-equivalent version in L2 English (Experiment 2). Participants read a sentence [e.g., A diligent worker walked into the office (spatial sense); A strong team headed into the final (abstract sense)] at their self-paced speed, followed by a congruent (e.g., into diagram) or incongruent diagram (e.g., out-of diagram), and made binary judgments to verify spatial configurations between the sentence and diagram. Semantic rating tasks in both Chinese and English were also conducted to validate congruency between diagrams and sentences in both languages. Results from Experiment 1 indicate overall compatibility effects on L1 Chinese processing, unaffected by directional verbs or abstractness of sense. Results from Experiment 2 reveal interference effects on L2 English processing, with interference observed only after reading sentences encoding spatial senses, not abstract senses. Aligning with previous findings using sentence-picture verification tasks, the current findings confirm the weaker mental simulation effects in L2 processing compared to L1 processing. These findings extend the existing simulation-based L1 comprehension model, provide empirical support for the proposed simulation-based L2 comprehension model, and validate the innovative sentence-diagram verification paradigm for examining image-schematic representations in spatial and abstract language processing among Chinese-English bilinguals. The paradigm holds significant potential for research on perceptual representations in processing a broader range of grammatical and semantic properties during both online and offline L1 and L2 comprehension.

Testosterone level has an influence on cognitive functions, especially spatial abilities. The relationship is, however, bidirectional and brain activity also affects testosterone levels. The aim of this study was to analyze the effects of an intensive 3D mental rotation task on testosterone levels in young healthy men and women. In the mental rotation task, men reached a higher top score (P=0.027) and total score (P=0.004) compared to women. In 8 out of 9 women (P=0.008) but not in men (P=0.129) testosterone levels decreased after one hour of mental rotation testing. A significant gender difference was shown (P<0.0001). In all women, plasma cortisol levels was significantly lower after testing (P=0.004). In men cortisol levels decreased in 7 out of 9 subjects (P=0.039). A significant gender difference was not found (P=0.19). No association was found in women between baseline testosterone levels and mental rotation total score (P=0.810). In men there was a positive correlation between baseline testosterone and mental rotation total score (P=0.015). A significant difference gender difference was seen in the association between testosterone and mental rotation score (P<0.05). Mental rotation stimuli caused significant changes in hormonal levels of testosterone and cortisol. A gender-specific response was detected in testosterone fluctuation.

Recent studies have revealed the instability of the action-sentence compatibility effect (ACE). The current study was designed to demonstrate the hypothesis that the instability of the ACE may be attributed to the instability of focused information in a sentence. A pilot study indicated that the focused information of sentences was relatively stable in the sentence-picture verification task but exhibited significant interindividual variability in the action-sentence compatibility paradigm in previous studies. Experiments 1 and 2 examined the effect of sentence focus on the shape match effect and the ACE by manipulating the focused information of sentences using the focus marker word "" (is). Experiment 1 found that the shape match effect occurred in the original sentence, while it disappeared when the word "" (is) was used to make an object noun no longer the focus of a sentence. Experiment 2 failed to observe the ACE regardless of whether the sentence focus was on the action information. Experiment 3 modified the focus manipulation to observe its impact on the ACE using different fonts and underlines to highlight the focused information. The results indicated that the ACE only occurred when the action information was the sentence focus. These findings suggest that sentence focus influences mental simulation, and the instability of the ACE is likely to be associated with the instability of sentence focus in previous studies. This outcome highlights the crucial role of identifying specific information as the critical element expressed in the current linguistic context for successful simulation.

A brief questionnaire about sense of direction and mechanical experience correlated with Mental Rotation scores. Several new, experimental measures thought to be related to spatial ability did not in fact correlate with Mental Rotation scores.

Are visual processes causally involved in “perceptual simulation” effects in the sentence-picture verification task?

The present study examined the relationship of initial qualitative analysis of movement scores, disembedding scores, and mental rotation scores on terminal qualitative analysis of movement scores. The subjects were 19 female and 17 male undergraduate majors in physical education, 14 from Oklahoma State University and 22 from Southern Utah University, with a mean age of 23.0 +/- 4.5 yr. The test and instructional unit on qualitative analysis of movement were developed by Morrison and Harrison in 1985. The Group Embedded Figures Test was used to discern disembedding scores and the Mental Rotations Test scores on mental rotation. The means and standard deviations for the pretest and posttest measures on the movement analysis test were 72.08 +/- 7.06 and 78.30 +/- 4.21. Analysis indicated instruction improved scores on the qualitative analysis test. Also, initial movement test scores and those on disembedding were significant predictors of scores on the posttest qualitative analysis of movement but not of mental rotation test scores.