Pictorial Input Research Articles

Pictorial vs. linguistic negation: Investigating negation in imperatives across different symbol domains

The processing of negation is typically regarded as one of the most demanding cognitive processes as it often involves the reversal of input information. As negation is also regarded as a core linguistic process, to date, investigations of negation have typically been linguistic in nature. However, negation is a standard operator also within non-linguistic domains. For example, traffic signs often use negation to indicate a prohibition of specific actions (e.g., no left turn). In the current study, we investigate whether processing difficulties that are typically reported within the linguistic domain generalize to pictorial negation. Across two experiments, linguistic negation and pictorial negation were directly compared to their affirmative counterparts. In line with the literature, the results show that there is a general processing benefit for pictorial input. Most interestingly, the core process of negation also benefits from the pictorial input. Specifically, the processing difficulty in pictorial negation compared to affirmation is less pronounced than within the linguistic domain, especially concerning error rates. In the current experiments, pictorial negation did not result in increased error rates compared to the affirmative condition. Overall, the current results suggest that negation in pictorial conditions also results in a slowing of information processing. However, the use of pictorial negation can ease processing difficulty over linguistic negation.

Intelligent Music Player Based on Emotions

This project presents a system to automatically detect emotional dichotomy and mixed emotional experience using a Linux based system. Facial expressions, head movements and facial gestures were captured from pictorial input in order to create attributes such as distance, coordinates and movement of tracked points. Web camera is used to extract spectral attributes. Features are calculated using Fisher face algorithm. Emotion detected by cascade classifier and feature level fusion was used to create a combined feature vector. Live actions of user are to be used for recording emotions. As per calculated result system will play songs and display books list.

A double-edged sword: Metaphor and metonymy through pictures for learning idioms

AbstractThis study examined the impact that metaphorical pictures and semantic transparency had on meaning recall of English idioms for L2 learners. Twenty-seven idioms of differing semantic transparency (low, mid, and high) were selected and two types of pictures were drawn for each idiom. One picture type was a literal only representation while the other was a literal+figurative representation. These idioms and picture conditions (no picture, literal only, and literal+figurative) were counterbalanced and presented to student participants (n=64) via a PowerPoint input treatment. The delayed posttests measured meaning recall of the idioms’ L1 paraphrased meaning. The data showed that in absolute terms, literal-figurative pictures promoted better retention of meaning at all levels of transparency, but this finding was most robust for high-transparency idioms. However, a number of pictures, especially those with metonymical elements, led to overspecification, which has wider implications for pictorial input in general.

Impact of Pictorial Cues on Speaking Fluency and Accuracy among Iranian Pre-Intermediate EFL Learners

This study examined the effect of pictorial cues on Iranian pre-intermediate EFL learners’ speaking accuracy and fluency. To do this study, 54 Iranian pre-intermediate EFL learners were selected out of 80 students in a private English Language Institute. The selected participants were divided into two equal groups; experimental group and control group. After that, both groups were pretested by a speaking pre-test. The experimental group was taught through using the pre-speaking strategies as the researcher provided students with pictorial inputs. On the other hand, the students of the control group were taught through traditional speaking activities including repetition and over-learning. The treatment took 15 sessions of 50 minutes each under the guidance of the supervisor. In the first session, the participants were homogenized. In the second session, they were pretested. During 11 sessions, students were taught by using pictorial input, and in the last session after the treatment the two groups took the speaking post-test. The results of paired t-test and MANOVA revealed that the experimental group had better performance on their accuracy and fluency post-test compared to their pre-test. The results also showed that the experimental group outperformed the control group on the accuracy and fluency post-test. Finally, implications arising from the findings and suggestions for further research were explained.

A Mixed-Method Case Study: Is there Only the Pictorial Input that Affects Word Retention?

This mixed-method case study was conducted to probe how a set of pictures had an influence on a group of EFL university students’ retention of English words. Seven Thai university participants, enrolling in the course of English for Service Industry, were voluntarily engaged in the study. They took a pretest of 45 words they learned in class through the use of pictorial input for one semester, and right after the posttest, they recalled how they could remember the words in an individually stimulated recall protocol session. The result of T-test from Wilcoxon sign-ranked test showed that the pretest and posttest scores were significantly different at the 0.05 level. Interestingly, the qualitative accounts from the stimulated recall revealed that apart from the pictorial input the participants learned in class, they also employed other strategies to help them memorize the vocabulary. The findings from the study; therefore, shed lights on cognitive-metacognitive processing and strategies an individual EFL learner adopted, and most importantly, on how teachers can encourage their learners to orchestrate them and make the best use of pictures in order to learn ESP vocabulary effectively.

Embedded blends and meaning construction in Modest Musorgsky’sPictures at an Exhibition

Conceptual blending is a cognitive theory positing the combination of diverse conceptual spaces for the creation of novel blended spaces. Musical conceptual blending can be intra-musical, pertaining to the combination of diverse structural elements for the creation of new melodies, harmonies or textures, as well as cross-domain, involving the integration of musical and non-musical spaces for the creation of novel analogies or metaphors. This article presents a structural and hermeneutical analysis of “Il vecchio castello” and “Samuel Goldenberg und Schmuÿle” from Modest Musorgsky’s Pictures at an Exhibition in an attempt to disclose both the intra-musical (combination of modal, tonal or coloristic harmonic spaces and rhythmic/textural structure) and extra-musical (contextual, symbolic and programmatic aspects) conceptual blending that the works incorporate. The reductional/prolongational analyses provide input for two distinct Conceptual Integration Networks, the first describing the intra-musical blending of melodic harmonization and the second proposing the cross-domain blending of the musical and pictorial input spaces into a blended hermeneutical space that projects the works’ narrative/programmatic/emotional potential. The proposed analyses show how musical structure promotes creative listening and meaning construction through complex cross-domain integration. This research suggests that conceptual blending theory as an analytical tool can promote a richer structural interpretation and experience of music, even in cases of instrumental, non-vocal compositions.

Word frequency effects in the left IFG in dyslexic and normally reading children during picture naming and reading

Word frequency effects have been reported in numerous neuroimaging studies with typically reading adults, emphasising the role of the left inferior frontal gyrus (LIFG). Within LIFG, different cytoarchitectonic modules (areas 44 and 45) have been related to phonological vs. lexico-semantic processing, respectively. This fMRI study investigated the differential impact of word frequency on LIFG activation in reading and picture naming in primary school children with and without developmental dyslexia. All children showed the typical LIFG frequency effect in both tasks. The effect was comparable in a fronto-orbital region anterior-inferior adjacent to area 45. During reading but not picture naming, a second effect was observed in area 44. Here, the fMRI effect for lexical frequency was stronger for the dyslexic than the normal readers. These findings demonstrate the neural underpinnings of a selective deficit in dyslexic children in the graphemic input lexicon, whereas abstract lexical representations appear to be processed equally well in dyslexic and normally reading children. To conclude, the present fMRI study demonstrated differential impact of word frequency on LIFG activation in primary school children during reading but not picture naming. Apart from extending previous knowledge from studies with adults to childhood, the study sheds further light on a potential neural mechanism for deficient grapheme-to-phoneme conversion in dyslexic children.

Direct access from meaning to orthography in Chinese: A case study of superior written to oral naming

Background: For alphabetic scripts, the obligatory phonological mediation hypothesis about written language processing has been seriously challenged by case reports of acquired dyslexia and dysgraphia. Evidence against the hypothesis mainly comes from superior performance on written production over oral production of the same lexical items. In Chinese, the absence of grapheme‐to‐phoneme conversion, the presence of a character component providing a semantic cue to the meaning of many phonetic compound characters, and the great extent of homophony have led to the view that the orthography is meaning based rather than speech based. However, psycholinguistic studies of character recognition have obtained conflicting results regarding the relative time course of activation of phonological vs semantic information. The work reported here was supported by a grant from the Research Grant Council of Hong Kong (HKU 7157/02H). We are grateful to LKY for his participation in this study. Aims: This paper addresses the phonological mediation hypothesis through describing the performance pattern of a Cantonese‐speaking brain‐injured individual, LKY, on tasks involving oral and written production of single Chinese words. Methods and Procedures: The range of tasks administered to LKY included visual‐spatial analysis, auditory discrimination, written and spoken lexical decision, repetition, oral and written picture naming, reading aloud and writing‐to‐dictation of object names, homophone identification and judgements, and verbal and non‐verbal semantic tests. Outcomes and Results: LKY performed normally on tasks assessing the processing of visual, auditory, orthographic, and verbal input. He was moderately impaired in repeating words and phrases. His performance on the two naming tasks, reading aloud, and writing‐to‐dictation was clearly disrupted. In addition, he was unable to retrieve phonological information from orthographic input, given his poor ability to make homophone judgements and identification. While he could select objects that are functionally related in non‐verbal semantic tests, he was impaired in accessing meaning from verbal materials. The most significant features of his performance pattern were his superior written over oral picture naming and better written naming than writing‐to‐dictation of the same stimuli. Conclusions: The discrepancies in performance between oral and written naming and between the written tasks with pictorial vs oral input were due to deficits at the phonological level. Such dissociations support the view that written production of Chinese, similar to the situation with alphabetic writing systems, is not parasitic on phonology, and thereby disconfirm the obligatory phonological mediation hypothesis.

Preserved reading aloud with semantic deficits: Evidence for a non-semantic lexical route for reading Chinese

This article describes a Cantonese dyslexic patient with a dissociation between reading ability and oral naming, similar to previously reported cases of Chinese dyslexia. Additional semantic tests without pictorial input were given to localize his impairment to the semantic system. His largely preserved reading performance vis-à-vis semantic deficits, together with the absence of assembled phonology in Chinese, support a model of the Chinese lexicon in which reading can be achieved via two different lexical routes, one with semantic mediation and one without. The patient’s poor ability to make homophony judgments of written characters and the high rate of tonal errors suggest that brain injury may have a more detrimental effect on suprasegmental than segmental features of phonological representations.

The effects of different tasks on children's process and product variables in writing

The main problem of the reported study was to examine the relation between writing tasks (pictures vs. texts) and children's strategies used in the writing process, on one side, and their writing products (narratives), on the other. Twenty 12-year-old children were sampled randomly as subjects for the experiment. The experiment was organised as a 2×2×2 three-way factorial design with condition (task variable), group, and gender used as independent variables. After having examined the intercorrelations among a preliminary sample of dependent measurements the number of variables was reduced such that the final sample of dependent measures consisted of three groups of variables: 1) three general strategy variables (attention, feedback, and persistence), 2) six domain-specific strategies, and 3) five measures of the writing products used for the final data analyses. The results of the multivariate analyses indicated that tasks presented as text (input) had significant effects on concentration and persistence (general strategies) and on writing time, frequency of pauses in writing, time for long and short pauses (domain-specific strategies), whereas pictorial input had no substantial effects on the writing process variables under research. On the other hand, pictorial material had significant effects on two of the product variables: accuracy and formal aspects of the writing product. Few significant findings were caused by the group and the gender factors used in the study.

Anomia and dyslexia in Chinese: A familiar story?

We report a study investigating the factors that predict the naming performance of four Chinese-speaking anomic patients. The results showed that the rated familiarity of an item predicts naming performance for all of the patients, a finding that is consistent with data from studies of English-speaking anomic patients. These data are discussed in terms of current models of spoken word production that have been developed largely on the basis of data from English speakers. We also investigated the relationship between anomia and dyslexia for each patient by presenting 232 items to name from pictorial input and from print. The results showed that there was a highly significant correlation between anomia and dyslexia for the same items among three of the four patients, a finding that is consistent with other studies of Chinese-speaking aphasic patients (e.g. Hu et al. 1983). However, for one patient there was a complete dissociation between naming and oral reading of the same items (impaired picture naming co-incident with flawless oral reading), suggesting that spoken word production and oral reading can proceed via separate cognitive systems. We offer a model of spoken word production and oral reading in Chinese that assumes picture naming and oral reading rely upon functionally separate pathways to account for these data.

Ideational Gestures and Speech in Brain-damaged Subjects

Patterns of speech-related (“coverbal”) gestures were investigated in two groups of right-handed, brain-damaged patients and in matched controls. One group of patients ("ldquo;aphasic”) had primarily anomic deficits and the other (“visuo-spatial”) had visual and spatial deficits, but not aphasia. Coverbal gesture was video-recorded during the description of complex pictures and analysed for physical properties, timing in relation to speech and ideational content. Aphasic patients produced a large amount of ideational gestures relative to their lexical production and pictorial input, whereas the related production of the visuo-spatial patients was small. Controls showed intermediate values. The composition of ideational gestures was similar in the aphasic and control groups, while visual subjects produced less iconic gestures (i.e. less gestures which show in their form the content of a word or phrase). We conclude that ideational gestures probably facilitate word retrieval, as well as reflect the transfer of information between propositional and nonpropositional representations during message construction. We suggest that conceptual and linguistic representations should probably be re-encoded in a visuo-spatial format to produce ideational gestures.

The Interaction of Lexical and Non-Lexical Processing Mechanisms: Evidence from Anomia

Patient SF presented with a selective deficit in naming, in the presence of normal auditory, visual and pictorial input processing, and of normal comprehension. The naming disorder was independent of input modality, and resulted in the inability to respond to lowfrequency items. The analysis of the patient's performance in the repeated administration of the same set of pictures for oral naming and for written naming demonstrated that both tasks were disrupted to a similar extent - high consistency values were observed both across and within modality. Thus, the profile of the naming disorder observed in this patient was consistent with damage to phonological and orthographic output lexicons. SF's performance in reading and writing was apparently at odds with this account, as he was able to read aloud words with lexically-assigned stress, and to write to dictation words whose spelling is also determined lexically. The co-occurrence of the inability to activate output lexical representations in both oral and written picture naming, in the context of normal ability to activate the same representations in reading and in writing is consistent with the notion that phonological and orthographic output lexicons are distinct, but interact with nonlexical conversion mechanisms.

The selective impact of question form and input mode on the symbolic distance effect in children

Five experiments are reported on the symbolic distance effect (SDE) and related phenomena with 6- and 9-year-old children. In the first of these, children were asked to judge the relative sizes of animals in verbal and pictorial tests featuring the comparatives “bigger” and “smaller”. A perceptual condition with actual objects was included by way of comparison. A Symbolic Distance Effect was obtained for both lexical and pictorial input. Mode differences were also observed. Pictures produced faster responses than words, and congruity effects occurred only in the pictorial condition. Although performance was similar in tests with either comparative, our subsequent experiments on both 6- and 9-year-olds reveal a significant asymmetry in the child's capacity to verify statements of relation as a function of the direction along the (size) continuum implied by the question. However, important differences between age groups also apparent in the data lead us to conclude that the older subjects develop strategies to overcome this asymmetry by translating certain statements of relation into a form more congruent with their natural modes of encoding.

Visual Superiority Effect in Memory (Pictures are Worth a Thousand Words)

A question of current theoretical interest that concerns the nature of both recall and recognition of displayed information is the contention that there is pictorial superiority effect (PSE) in memory. In recent years there has been considerable discussion about the notion that pictures are better retained than their verbal labels. There is a good deal of evidence from a variety of recall, recognition and pair-associate learning tasks that have shown this contention to be true. Just what do people remember about a picture? Questions concerning the nature of memory for pictures continue to challenge the curiosity of experimental psychologists. Are words and pictures encoded distinctively? That is, does the memory of an item contain information to indicate whether the item was presented as a word or as a picture? Is there a distinctive processing for words and pictures? The purpose of this paper is to discuss studies rendering support to the notion of pictorial superiority in memory. A number of experiments (e.g., Paivio, 1968, 1966) have shown that picture-pairs are much easier to recognize than pairs comprised of their concrete verbal labels or descriptions. Paivio and Yarmey (1966) obtained support for this hypothesis in a study in which pictures and their noun labels were factorially varied on the stimulus and response side of pairs. Further, Paivio and Yarmey demonstrated that in paired-associate learning, retention appears to function as a single effective dimension that extends from abstract nouns, to concrete nouns to pictures (or objects) in increasing order of effectiveness. The question of the superiority effect of pictorial memory has been investigated by psychologists in the field of memory research. There appear to be at least three possible explanations relative to the source of pictures/word differences. The first model (processing level model) suggests that there are analogous processing stages for pictures and words, but the time required for processing each stage is less for pictorial input. The second model (sensory semantic model) suggests that pictorial inputs require fewer processing transformations than verbal inputs before semantic processing occurs. Finally, the dual encoding model suggests two separate but interconnected verbal and non-verbal processing or symbolic systems. This model argues that pictorial stimuli generate simultaneous verbal and non-verbal long-term memory codes; whereas, words don't generally elicit their pictorial code. The finding that memory is generally better for pictorial than verbal labels is not surprising. The sensory and meaning codes for a picture are apparently more differentiating and less susceptible to interferences. Presumably, more associative cues may be attached to stimuli which have rich visual, as well as verbal associates, so that both recall and recognition performance are improved with pictorial stimuli.

A Survey of Electronic Techniques for Pictorial Image Reproduction

This paper is a tradeoff study of image processing algorithms that can be used to transform continuous tone and halftone pictorial image input into spatially encoded representations compatible with binary output processes. A large percentage of the electronic output marking processes utilize a binary mode of operation. The history and rationale for this are reviewed and thus the economic justification for the tradeoff is presented. A set of image quality and processing complexity metrics are then defined. Next, a set of algorithms including fixed and adaptive thresholding, orthographic pictorial fonts, electronic screening, ordered dither, and error diffusion are defined and evaluated relative to their ability to reproduce continuous tone input. Finally, these algorithms, along with random nucleated halftoning, the alias reducing image enhancement system (ARIES), and a new algorithm, selective halftone rescreening (SHARE), are defined and evaluated as to their ability to reproduce halftone pictorial input.

Inferring conceptual graphs

This paper investigates the mechanisms a program may use to learn conceptual structures that represent natural language meaning. A computer program named Moran is described that infers conceptual structures from pictorial input data. Moran is presented with “snapshots” of an environment and an English sentence describing the action that takes place between the snapshots. The learning task is to associate each root verb with a conceptual structure that represents the types of objects that participate in the action and the changes the objects undergo during the action. Four learning mechanisms are shown to be adequate to accomplish this learning task. The learning mechanisms are described along with the conditions under which each is invoked and the effect each has on existing memory structures. The conceptual structure Moran inferred for one root verb is shown.

Cognitive masking in rapid sequential processing: The effect of an emotional picture on preceding and succeeding pictures

This experiment assessed the effect of an emotional pictorial input on the processing of preceding and succeeding pictorial inputs. Thirty-nine individual pictures were successively presented for 165 msec, with a 500-msec dark interval between each item. The 20th input item was either an emotional or a neutral picture; the remaining 38 items were neutral sketches. Postinput recognition data indicated a processing disruption of several pictorial items following the emotional as compared to the neutral input and a possible processing disruption of the item immediately preceding the emotional stimulus. The outcome suggests nonindependence in the processing of separate picture inputs in rapid sequential presentations.

Computer vision

“Intelligent” computers require knowledge of their environment, and the most effective means of acquiring such knowledge is by seeing. Vision opens a new realm of computer applications. Weather analysis from cloud maps and medical diagnosis from x-rays and blood cell counts are two of many high volume data processing tasks that require pictorial input. Man-machine communication will be facilitated if a person can directly show the computer a real object, rather than describe it symbolically. Designers, for example, could then easily obtain a computerized structural analysis of scale models.

Generating computer animated movies from a graphic console

Graphical output via the computer has been available for many years now, but generally only one device at a time has been available to produce the display, and only one mode of output or input has been considered. This paper describes the successful integration of several components to form a graphics system which produces plots, animated crt displays, and motion pictures. Two similar driving programs are employed to create either planar or 3 dimensional dynamic picture sequences from picture language commands and/or other pictorial input.