Spoken Modality Research Articles

One grammar or two? Sign Languages and the Nature of Human Language.

Linguistic research has identified abstract properties that seem to be shared by all languages—such properties may be considered defining characteristics. In recent decades, the recognition that human language is found not only in the spoken modality but also in the form of sign languages has led to a reconsideration of some of these potential linguistic universals. In large part, the linguistic analysis of sign languages has led to the conclusion that universal characteristics of language can be stated at an abstract enough level to include languages in both spoken and signed modalities. For example, languages in both modalities display hierarchical structure at sub-lexical and phrasal level, and recursive rule application. However, this does not mean that modality-based differences between signed and spoken languages are trivial. In this article, we consider several candidate domains for modality effects, in light of the overarching question: are signed and spoken languages subject to the same abstract grammatical constraints, or is a substantially different conception of grammar needed for the sign language case? We look at differences between language types based on the use of space, iconicity, and the possibility for simultaneity in linguistic expression. The inclusion of sign languages does support some broadening of the conception of human language—in ways that are applicable for spoken languages as well. Still, the overall conclusion is that one grammar applies for human language, no matter the modality of expression.

Competition from unseen or unheard novel words: Lexical consolidation across modalities

In four experiments we investigated the formation of novel word memories across modalities, using competition between novel words and their existing phonological/orthographic neighbours as a test of lexical integration. Auditorily acquired novel words entered into competition both in the spoken modality (Experiment 1) and in the written modality (Experiment 4) after a consolidation period of 24 h. Words acquired from print, on the other hand, showed competition effects after 24 h in a visual word recognition task (Experiment 3) but required additional training and a consolidation period of a week before entering into spoken-word competition (Experiment 2). These cross-modal effects support the hypothesis that lexicalised rather than episodic representations underlie post-consolidation competition effects. We suggest that sublexical phoneme–grapheme conversion during novel word encoding and/or offline consolidation enables the formation of modality-specific lexemes in the untrained modality, which subsequently undergo the same cortical integration process as explicitly perceived word forms in the trained modality. Although conversion takes place in both directions, speech input showed an advantage over print both in terms of lexicalisation and explicit memory performance. In conclusion, the brain is able to integrate and consolidate internally generated lexical information as well as external perceptual input.

Latent Semantic Analysis for Multimodal User Input With Speech and Gestures

This paper describes our work in semantic interpretation of a “multimodal language” with speech and gestures using latent semantic analysis (LSA). Our aim is to infer the domain-specific informational goal of multimodal inputs. The informational goal is characterized by lexical terms used in the spoken modality, partial semantics of gestures in the pen modality, as well as term co-occurrence patterns across modalities, leading to “multimodal terms.” We designed and collected a multimodal corpus of navigational inquiries. We also obtained perfect (i.e. manual) and imperfect (i.e. automatic via recognition) transcriptions for these. We automatically align parsed spoken locative references (SLRs) with their corresponding pen gesture(s) using the Viterbi alignment, according to their numeric and location type features. Then, we characterize each cross-modal integration pattern as a 3-tuple multimodal term with SLR, pen gesture type and their temporal relationship. We propose to use latent semantic analysis (LSA) to derive the latent semantics from manual (i.e. perfect) and automatic (i.e. imperfect) transcriptions of the collected multimodal inputs. In order to achieve this, both multimodal and lexical terms are used to compose an inquiry-term matrix, which is then factorized using singular value decomposition (SVD) to derive the latent semantics automatically. Informational goal inference based on the latent semantics shows that the informational goal inference accuracy of a disjoint test set is 99% and 84% when a perfect and imperfect projection model is used respectively, which performs significantly better than (at least 9.9% absolute) the baseline performance using vector-space model (VSM).

When does word frequency influence written production?

The aim of the present study was to explore the central (e.g., lexical processing) and peripheral processes (motor preparation and execution) underlying word production during typewriting. To do so, we tested non-professional typers in a picture typing task while continuously recording EEG. Participants were instructed to write (by means of a standard keyboard) the corresponding name for a given picture. The lexical frequency of the words was manipulated: half of the picture names were of high-frequency while the remaining were of low-frequency. Different measures were obtained: (1) first keystroke latency and (2) keystroke latency of the subsequent letters and duration of the word. Moreover, ERPs locked to the onset of the picture presentation were analyzed to explore the temporal course of word frequency in typewriting. The results showed an effect of word frequency for the first keystroke latency but not for the duration of the word or the speed to which letter were typed (interstroke intervals). The electrophysiological results showed the expected ERP frequency effect at posterior sites: amplitudes for low-frequency words were more positive than those for high-frequency words. However, relative to previous evidence in the spoken modality, the frequency effect appeared in a later time-window. These results demonstrate two marked differences in the processing dynamics underpinning typing compared to speaking: First, central processing dynamics between speaking and typing differ already in the manner that words are accessed; second, central processing differences in typing, unlike speaking, do not cascade to peripheral processes involved in response execution.

Poznanie poza językiem. Przypadek religijnego doświadczenia mistycznego a jego wysłowienie

An analysis of the experiences, insights and proclamations by mystics and prophets from various epochs unveils the characteristics of individuals exposed to long experiences and holistic (or integral) cognition. A mystic transformation affects the mode of speaking and communicating with the world. Language with its stereotypical phrases and connotations certainly limits human cognition. However, in the process of their transpersonal experience, mystics go beyond this barrier and overcome cognitive limitations including the flaws of human language. In order to present an altogether “different” space of reality shared by mystics, and to share their proclamations and teaching with a large audience, mystics apply very specific linguistic tools like the metaphor, allegory, antithesis, paradox, analogy, symbol, quotations from the Bible, hyperbole, parallelism, chiasm and, last but not least, parable. An act of viewing the prophecy and epiphany-related language reflecting the authors’ synoptic (mystic) awareness “from the inside” makes it possible to identify archetypal criteria of the mystic experience and to establish the correct meaning of the key expressions (the sacred) and sacral acts of speech.

Deverbal Nominalisations across Written-spoken Dichotomy in the Language of Science

Scientific language has always been both a source of difficulty for the students of science and, in the recent decades in particular, a theme for speculation for the scholars seeking to describe and analyse language as it is used and conceived. The language of science has always been presumed to be more difficult than the language used in other fields. Scientific concepts and their sophistication are one source of its difficulty. The writing style and dynamics preferred by writers in the fields of science is commonly considered another source of such difficulty. Nominalisation, the use of derivative nouns of verbs and adjectives, which is more prevalent in the language of science compared with the language used elsewhere, is another feature attested to by many scholars. The present study aimed at comparative investigation of the phenomenon of deverbal nominalisations (nominalisations with verb origin) across written and spoken scientific language. The spoken samples were chosen from BBC’s programs in the genre of science and the written samples from science textbooks and credited science magazines (different issues). The samples in each modality included 400,000 words (for each modality) amounting to 800,000 words as the total size of the corpora considered. The findings showed a higher frequency of verbs in spoken modality and of nominalisations in written. Another interesting point was the highest frequency of material process type in both verbs and deverbal nominalisations. (The typology of processes from the Hallidayean systemic functional grammar framework was the basis for classification of verbs and their respective nominalised editions.) Relational process type which claimed the second highest frequency among verbs showed a much lower frequency among nominalisations. The consideration of the slots accommodating the nominalisations in the sentences also revealed insightful points about possible differences between spoken and written language (of science) in terms of the concentration of semantic and lexical density.

Aids to computer-based multimedia learning: a comparison of human tutoring and computer support

Learners are usually provided with support devices because they find it difficult to learn from multimedia presentations. A key question, with no clear answer so far, is how best to present these support devices. One possibility is to insert them intothe multimedia presentation (canned support), while another is to have a human agent provide them (human tutoring). Human tutoring poses potential advantages: it uses spoken modality, displays non-verbal cues and implies social interaction. However, there is mixed evidence regarding these supposed advantages, and prior research comparing human and computer support presents problems. Our goal was to explore whether the advantages of human tutoring actually exist while avoiding the problems of prior research. In one experiment, participants learned Geology from a multimedia presentation including one of three forms of support: human tutoring, canned support or no support. After viewing the presentation, participants solved retention and transfer tests. Results revealed that participants in the human tutoring condition outperformed those in the other two conditions, who did not differ from each other. This means that human tutoring is advantageous, a fact that has implications in the design of support devices in multimedia learning.

Die Illusion der ›einfachen Form‹

Abstract Saints’ lives occupy a fluctuating position between cult and art which challenges their claim to be a devout and unpretentious ‘simple form’. The following contribution traces the interplay in the genre between scepticism toward rhetoric on the one hand and fascination with it on the other. The greater the importance that attaches to a surplus of rhetoric and narration, the more contradictory the narrative order mandated by the genre becomes, and the more testing the conditions under which saints’ lives are narrated. The specific interference of religious and literary modes of speaking is illustrated from examples of narrative saints’ lives from the late middle ages. What tensions does the genre’s ambivalent attitude to rhetoric give rise to when telling stories about saints through the act of narratively transforming the miraculous?

Syntactic Comprehension in Reading and Listening

This study examined syntactic comprehension in French children with dyslexia in both listening and reading. In the first syntactic comprehension task, a partial version of the Epreuve de Compréhension syntaxico-sémantique (ECOSSE test; French adaptation of Bishop’s test for receptive grammar test) children with dyslexia performed at a lower level in the written but not in the spoken modality, compared to reading age–matched children, suggesting a difficulty in handling syntax while reading. In the second task, syntactic processing was further explored through a test of relative clause processing, in which inflectional markers could aid in attributing roles to the elements in a complex syntactic structure. Children with dyslexia were insensitive to inflectional markers in both reading and listening, as was the reading age control group, while only the older normal reader group appeared to make use of the inflectional markers. Overall, the results support the hypothesis that difficulties in comprehension in dyslexia are strongly related to poor reading skills.

Judges of Normality: Mediating Marriage in the Family Courts of Kolkata, India

When legal reform proceeding from feminist movement demands is institutionalized in courts, there may nevertheless be distinct gaps in delivering substantive gender justice. Can feminist reform be enacted through agents who may be resistant to feminist critiques of marriage? This paper evaluates feminist legal reform around marriage by profiling lawyer-free, mediation-based family courts in Kolkata, India, a much-touted experiment in alternate dispute resolution alleged to provide direct and less alienating access to the law for marginalized subjects. Through an ethnography of everyday adjudication, it maps the new modes of mediation that judges and counselors (paralegals or social workers, not lawyers) proudly foreground, in which they see themselves providing better kinship through law and protecting women’s economic interests while keeping families intact. But these scripts reveal the difficulties of stepping outside gendered norms of permissible legal agency, appropriate class behaviors, the degree of violence tolerated, and conjugal authority. The women’s movement itself has become deeply divided around whether lawyer-free venues are better for gender justice given judges’ increased authority and minimization of the violence women experience. In this article I argue that while both law and marriage have been deeply gendered sites of subordination and while one experiment in altered modes of speaking and adjudication cannot change the valences of marriage law, feminists must continue to push for changes in the legal system.

Configuración en el modo escrito de competencias desarrolladas en forma oral como efecto del nivel funcional de ejercicio

This study aimed to establish, in children who start aprendiaje writing, if the functional level of a written exercise mode affects the likelihood that developed in the way comptenecias spoken also be configured as written. 33 children were selected in an initial evaluation showed a low tendency to express conditional relations , causal and functional ( RCCF ) to speak and escribirsobre a pet and a game. Children were assigned to four experimental groups , who exercised the written language in different functional Nuvelo : contextual, supplementary sistitutivo selector and reference. After this all children hey were trained to talk about crabs , linking their physical characteristics and behaviors your fucnión their conditions of occurrence. In a final stage, asked the children to speak and write about the same pet , and the same game as in the initial evaluation. The results indicated that the functional level of the league written exercise affects the likelihood that the skills developed in the spoken mode is also set in the written mode .

Bleaching a dialectal voice in political discourse

In 1973, former President of Tunisia, Habib Bourguiba delivered nine speeches in which he recounted episodes of the national movement. Even though delivered in an official setting and to a highly educated audience, the semi-planned speeches were delivered in the dialectal variety of Arabic, i.e. Tunisian Arabic. The speeches were brought together in a book form titled ḥayātī, ’ārā’ī, jihadi (My Life, My Opinions, My Jihād). Their publication in a book form meant that the original performed texts were modified from a spoken mode to a written mode and ‘translated’ from Tunisian Arabic, a dialectal form of Arabic, to fuṣḥā, the High variety of Arabic (Ferguson 1959). The rewriting of the speeches led to strategic sociolinguistic choices. In the translation process, the linguistic product was regulated by a web of competing institutions of power, sites of linguistic ideologies, and linguistic practices; each of which represents an institution of power whose interests shift strategically between moments of ideological convergence and/or divergence, if not rivalry and connivance, among and between them. The present essay is an attempt to explore those sociolinguistic choices translated into erasures, shifts, modifications, and polishing of the original text. The carefully orchestrated changes in the original text, I believe, aim at re-establishing the authoritative presidential voice, restoring the institutional linguistic status quo, and rendering Bourguiba’s personal ‘ḥkāyāt’, stories, to ‘tārīkh aš-šaʿb t-tūnsi’, ‘the history of the Tunisian people’.

Propietats pragmàtiques i gramaticals en el desenvolupament de la coreferència pronominal

Summary: In this paper, we examine the referential properties of null and overt subject pronouns in Catalan in the semi-spontaneous production of oral and written narrative texts by three groups of speakers/writers: 9–10 year olds, 12–13 year olds, and 16–17 year olds. We aim at determining whether these two types of pronouns play the same role and to what extent they contribute to anaphora resolution and discourse cohesion, both at an intra- and an inter-sentential level. We concentrate on anaphoric relations expressed by 3rd person pronouns in subject position. Our results show that, at an intra-sentential level, null pronouns normally refer to subject antecedents, mainly in the spoken modality, while overt pronouns refer back to either a subject or a non-subject antecedent. This entails that once a discourse element becomes a discourse topic and is the subject of a sentence, a null pronoun is used. Moreover, a parallel tendency is observed at an inter-sentential and discourse level. [Keywords: Reference; pronouns; Catalan; anaphora resolution; written development; oral development]

Freedom of Speech as an Expressive Mode of Existence

This paper adopts Deleuze’s reading of Spinoza’s expressionism and pure semiotics to argue that Spinoza’s Ethics offers an alternative notion of freedom of speech that is based on the potentia of the individual. Its aim is to show how freedom of thought is connected to the problem of individuation that connects our mode of being with our power to speak and think. Rather than treating freedom of speech as an enlightened idea that is in opposition to, for example, religious authority, or the suppression of human rights, this paper argues that freedom of speech should be understood by what Spinoza calls ‘an adequate idea’: an idea that explains the cause of its own production. What is to be considered is: who wants this freedom, in what situation, why, what is at stake? No freedom in itself is ever given. This paper argues for speech as an assembled body that is always in connection with other bodies. It is argued that to understand the power and value of the freedom of speech, we should study the praxis of the utterance as an assembled body, its causal dimensions, and its affective immanent relations with other bodies, and other modes of speaking.

The influence of modality on deep-reasoning questions

This study investigated the influence that modality (print versus spoken text) had on learning with deep reasoning questions. Half the participants were randomly assigned to receive deep-reasoning questions during the learning session. The other half received the same information in the absence of deep-reasoning questions. The participants who received deep reasoning questions were randomly assigned to one of two different groups. One group received deep reasoning questions as on-screen printed text while the other group received deep reasoning questions in a spoken modality via a text to speech engine. Participants who received deep reasoning questions had higher post-test scores than those who did not, a finding that replicated previous research. Additionally, learning was better for the learners who received printed text than spoken messages, a finding that is not compatible with a number of theoretical and empirical claims in the literature.

A comprehensive account of full-verb inversion in English

Despite the abundant corpus-based research on full-inversion constructions in which the subject follows the entire verb phrase in a declarative clause, the analysis of this construction in the spoken mode has traditionally been neglected since it has been suggested that natural speech is not a good source of inversions. The article shows that speech, too, can be a good source of data for full inversion and that the spoken and the written modes do not differ in the overall distribution of the construction but rather in the types of inversion used and in the pragmatic functions these inversions serve in discourse. The article also casts light on the lack of consensus regarding the distribution of full inversion in written fictional and non-fictional texts, and in the pragmatic function the construction serves in both genres.

Does textual feedback hinder spoken interaction in natural language?

The aim of the study was to determine the influence of textual feedback on the content and outcome of spoken interaction with a natural language dialogue system. More specifically, the assumption that textual feedback could disrupt spoken interaction was tested in a human–computer dialogue situation. In total, 48 adult participants, familiar with the system, had to find restaurants based on simple or difficult scenarios using a real natural language service system in a speech-only (phone), speech plus textual dialogue history (multimodal) or text-only (web) modality. The linguistic contents of the dialogues differed as a function of modality, but were similar whether the textual feedback was included in the spoken condition or not. These results add to burgeoning research efforts on multimodal feedback, in suggesting that textual feedback may have little or no detrimental effect on information searching with a real system. Statement of Relevance: The results suggest that adding textual feedback to interfaces for human–computer dialogue could enhance spoken interaction rather than create interference. The literature currently suggests that adding textual feedback to tasks that depend on the visual sense benefits human–computer interaction. The addition of textual output when the spoken modality is heavily taxed by the task was investigated.

Modes of Therapeutic Engagement Part I: Diplomacy and Integrity

Our topic concerns strategies of therapeutic discourse, four relational modes of speaking and listening, adopted with and without awareness: diplomacy, integrity, sincerity, and authenticity. The four modes are offered as conceptual tools to help the therapist build therapeutic relationships, respond to challenges and impasses, and advance group process and insight. They are navigated to cement bonding, build trust, allow troubled relationships to evolve and resolve, and strengthen the abilities to think and relate. Presented in two separate articles, in Part I the topic and its relationship to truth and falsity is introduced, and the modes of diplomacy and integrity are described. The discussion continues, in Part II, where the focus is on sincerity and authenticity, and the interrelationship among the four modes and their differing functions in the totality of the clinical situation is illustrated.

Models of Therapeutic Engagement Part II: Sincerity and Authenticity

Our topic concerns four relational modes of speaking and listening, and their relationship to truth and falsity. In a separate article, Part I introduced the topic and discussed the modes of diplomacy and integrity. The focus of Part II is on sincerity and authenticity, and on the interrelationship among all four modes and their differing functions in the totality of the clinical situation. Together the papers disentangle and address some of the technical issues that concern the relational group psychotherapist regarding topics of intersubjectivity, presentation of self, and self-disclosure. The four overlapping categories while nonexclusive are exhaustive; they supply conceptual references for the therapeutic stance one has adopted, allowing the therapist to be more aware of what he or she is doing and why.

Unity and Duality in Barack Obama's “A More Perfect Union”

Faced with a racialized political crisis that threatened to derail his campaign to become the first African American president of the United States, Barack Obama delivered a speech on race titled “A More Perfect Union.” He begins by portraying himself as an embodiment of double consciousness, but then invites his audience to share his doubled perspective, and finally models a doubled mode of speaking and acting that is captioned by the well-known maxim, the Golden Rule. This speech text thus contributes discursive resources required for the productive doubling necessary for the successful negotiation of contemporary public culture.