Signing Process Research Articles

The Contribution of Event-Related Potentials to the Understanding of Sign Language Processing and Production in the Brain: Experimental Evidence and Future Directions

Functional neuroimaging allows investigation of the timing properties of the brain mechanisms underlying covert language processing. This paper presents a review of the use of the neuroimaging technique called Event-Related Potentials (ERPs) in sign language (SL) research. In the field of neurolinguistics, ERPs have been widely used in the study of spoken language, but their use in SL is still rare. Studying the neurocognitive aspects of SL could lead to a better understanding of the specific processing of SL in the brain. This review outlines the basic theoretical and methodological principles of ERPs. We focus on three groups of ERPs that are particularly relevant to SL processing and production: ERPs focusing on cognition, ERPs focusing on language, and ERPs focusing on movement aspects. We then summarize within each group some ERPs that we consider could be useful for studying the sequence of cognitive processes underlying SL processing and we discuss the current state of the use of ERPs within SL research. According to our analysis of the field, ERPs focusing on language aspects have been used more than ERPs focusing on cognitive and movement-related aspects to study SL. More variability in the type of SLs used is needed to expand the inferences made so far. For the development of the field, we recommend the more frequent use of videos and SL stimulation at a natural pace in order to understand how SL is processed in daily life. The use of a wider variety of ERPs in the study of SL is also recommended. We conclude that ERPs offer a useful tool to address unanswered questions in the field, especially those that call for measuring the building blocks of SL processing in real time. The study of SL cognitive processing in the brain is still in its infancy. One way of developing the field in the coming years would be the more frequent use of the ERP neuroimaging technique.

'Grasp All, Lose All': Raising Awareness Through Loss of Grasp in Seemingly Functional Interfaces

From baroque proto-cybertexts to countercultural gestures by historical avant-gardes, there is a longstanding tradition of disruptive strategies used by artists at the interstices of societies’ demands for order, control, and functionalism. For the avant-gardes and their multiple artistic inf(l)ections, radical changes to the way sensory perception had come to be depicted since Modernism became a central part of their strategy. By placing an emphasis on the confluence between various arts and media, the innovative character of their proposals had much to do with the ways in which they were able to embrace notions that represented modernity, including concepts such as simultaneity, dynamics, motion, and the symbiosis between human and machine. In this manner, they sought to induce estrangement and defamiliarization by using seemingly functional mechanisms to raise awareness through the loss of grasp.Drawing on this idea of raising awareness through what appear to be functional mechanisms, I argue that non-functional/dysfunctional digital interfaces that are part of contemporary artworks that deal with digitally based haptic reading processes (expressly, digital literature) are largely influenced by early avant-garde artistic proposals. Thus, through its metamedial aesthetic and poetic critique of digital media, digital literature reinvents inherited strategies of subversion and disruption previously explored by Modernism, drawing attention to artworks’ processes of signification and affect. As a variation of a rich heritage of experimentation with seemingly functional mechanisms in the arts, such strategies reenact age-old tensions between tradition and innovation while laying the foundation for (re)new(ed) ways of reading and writing in digital multimodal environments.

Understanding cultural aspects of deaf communities in México towards the codesign of automatic sign language processing systems

This paper reports findings from a study through which we seek to understand the socio-economic and cultural context of deaf communities in a city in México in order to inform the codesign of automatic sign language processing systems. We conducted eleven interviews in two school settings with three deaf persons who are sign language users and eight hearing persons who have close relationships with deaf persons. We designed interview scripts to collect data from deaf persons, parents, and sign language teachers and interpreters. In addition to demographic data, we collected data on experience with technology for communication use as well as salient cultural aspects. Key findings from our research relate to perceptions of study participants about themselves and their context: Socio-economic aspects are consistent with previous studies; cultural traits are central for communication, denote a strong sense of identity, and highlight the role of sight and perception. We also report on reflections that were facilitated by the application of existing schemes and frameworks, which allowed us to learn about communication hardships and workarounds, as well as the most important values for the community. These findings shed light on a path we want to further investigate, which aims to understand whether variations exist in the basic needs of deaf signers with respect to the codesign of ASLP systems, independently of the countries where they live.

SignNet II: A Transformer-Based Two-Way Sign Language Translation Model.

The role of a sign interpreting agent is to bridge the communication gap between the hearing-only and Deaf or Hard of Hearing communities by translating both from sign language to text and from text to sign language. Until now, much of the AI work in automated sign language processing has focused primarily on sign language to text translation, which puts the advantage mainly on the side of hearing individuals. In this work, we describe advances in sign language processing based on transformer networks. Specifically, we introduce SignNet II, a sign language processing architecture, a promising step towards facilitating two-way sign language communication. It is comprised of sign-to-text and text-to-sign networks jointly trained using a dual learning mechanism. Furthermore, by exploiting the notion of sign similarity, a metric embedding learning process is introduced to enhance the text-to-sign translation performance. Using a bank of multi-feature transformers, we analyzed several input feature representations and discovered that keypoint-based pose features consistently performed well, irrespective of the quality of the input videos. We demonstrated that the two jointly trained networks outperformed their singly-trained counterparts, showing noteworthy enhancements in BLEU-1 - BLEU-4 scores when tested on the largest available German Sign Language (GSL) benchmark dataset.

Sign and Spoken Language Processing Differences in the Brain: A Brief Review of Recent Research.

Background:It is currently accepted that sign languages and spoken languages have significant processing commonalities. The evidence supporting this often merely investigates frontotemporal pathways, perisylvian language areas, hemispheric lateralization, and event-related potentials in typical settings. However, recent evidence has explored beyond this and uncovered numerous modality-dependent processing differences between sign languages and spoken languages by accounting for confounds that previously invalidated processing comparisons and by delving into the specific conditions in which they arise. However, these processing differences are often shallowly dismissed as unspecific to language.Summary:This review examined recent neuroscientific evidence for processing differences between sign and spoken language modalities and the arguments against these differences’ importance. Key distinctions exist in the topography of the left anterior negativity (LAN) and with modulations of event-related potential (ERP) components like the N400. There is also differential activation of typical spoken language processing areas, such as the conditional role of the temporal areas in sign language (SL) processing. Importantly, sign language processing uniquely recruits parietal areas for processing phonology and syntax and requires the mapping of spatial information to internal representations. Additionally, modality-specific feedback mechanisms distinctively involve proprioceptive post-output monitoring in sign languages, contrary to spoken languages’ auditory and visual feedback mechanisms. The only study to find ERP differences post-production revealed earlier lexical access in sign than spoken languages. Themes of temporality, the validity of an analogous anatomical mechanisms viewpoint, and the comprehensiveness of current language models were also discussed to suggest improvements for future research.Key message:Current neuroscience evidence suggests various ways in which processing differs between sign and spoken language modalities that extend beyond simple differences between languages. Consideration and further exploration of these differences will be integral in developing a more comprehensive view of language in the brain.

A Dialogue with Leone’s Socio-cultural Semiotics of Innovation

Innovation has become emblematic of our times. Indeed, innovation is seen as a vital economic, social, and technological phenomenon, increasingly attracting research interest. However, innovation is, we believe, much more than this; innovation also concerns processes of signification, communication, and inference. Therefore, both the process and product of innovation are open to semiotic explanation and analysis. However, not many semioticians have—so far—addressed innovation (at least not under the rubric of innovation). The prolific Italian semiotician Massimo Leone is the exception to the rule. Leone has tried to lay the groundwork for a semiotics of innovation. We have organized Leone’s semiotics of innovation into sixteen statements, which we address here. Hence, in a critical dialogue with Leone, we introduce Peirce’s concepts of abduction, the three hypoicons (including similarity), and the semiotic mind (of the innovator/receiver) in order to apply a few ideas which we believe can be of relevance when working with a semiotics of innovation conceptually and/or methodologically

Art, face and breathscape: From air to cultural texts

We consider breath as a vast prospect that includes actions and traces of them, that builds images and texts, that involves the human being and the extra-human context; we call this great scenery ‘breathscape’. We then study how breathscape interacts with the human apparatus of the face, both giving rise to signs, but also giving rise to a liminal zone of extremely intriguing interpretative processes on a mereological scale. How and where do the territory of breath and the body interact? Which processes of signification do they give rise to? And which signs are created in their phenomenal and semantic encounter? Art is certainly the most appropriate language for studying this process, as well as for letting opacities emerge and exploring outstanding contrasts. There are various concepts of ‘breathscape’ that, grounded in different cultures, are immediately associated with ancient and contemporary philosophies. As a reverberation from a semiotic interaction and through the discretization and identification of semantic fields relevant to the concerned scenario, and introducing textuality, a phenomenon seen as a crystallization of the transition between outer text (the text of reality) and text (subject/object), we consider those visual texts which are crossed by a common faculty that is both descriptive and inventive: by approaching some inferential and cultural regimes and analysing their specific enunciative practices, we then contribute to their renovation. The texts related to the practices as part of the narratives intrinsic to cultural semiospheres underline the insatiable vastness of epistemological content to be dealt with, and the functional reductionism of the corpus is only a first approach to the field that is intended to shed light on the general panorama and to stimulate subsequent debate and insights.

Contributions to a semiotics of religion: the semiosis from sign to meaning

This article approaches theoretically the religious experience in toto. Considering the semiotics applied to religion, contributions to understand and recognize the relevance of this discipline are proposed. Such approach to the semiotics of religion justifies the aim of the article: to understand the meaning structures of religious experiences. These experiences are diverse, intimate, subjective, but all have an idea of the “transcendent” as a referent and they are based on structures of meaning, expressions, and representations of the sacred, forms, uses and interpretations of religious signs, systems of collective thought and symbolic action. It is intended to advocate that: 1) the semiotics of religion is an interdisciplinary branch of social sciences and humanities and a sort of semiotics of culture; religion is a form of culture, as well communication and social meaning; 2) religion is a semiotic phenomenon; it is sustained by signs, representations, processes of signification and cultural construction of the world, without which there could be no religion. This is followed by a conceptual, theoretical strategy of critical discussion of the structures of meaning on which manifest culture is based through what we say or do, the way we behave and the attitude we have towards signs.

Elizabeth Bishop and the Watery Discourse: The Semiotic Chora at play

This paper theorizes the use of water imagery in Elizabeth Bishop’s selected postmodern poetry namely, “The Map”, “The Imaginary Iceberg” and “The Man-Moth”. Speaking in the language of tears, lake, bay, river, strait, sea and ocean, Bishop personifies water for culture and its language. It is on this basis that Bishop comes close to the French psychoanalyst and linguist, Julia Kristeva’s concept of the semiotic chora. Kristeva calls it a space of maternal discourse, where the fluid realm of mother’s body called chora celebrates the fluidity of semiotic desires and expresses them through the symbolic language of the world outside her body. Bishop revisits this bodily space, time and again, to acquire subjectivity through the mother/child bond. Bishop’s poetic language celebrates this relationship between the semiotic (maternal) and symbolic (paternal) realm of language. The symbolic enables the semiotic to express itself, and the semiotic shatters the rigidity of the symbolic meanings through an ongoing process of signification. This reciprocity, through the semiotic chora, makes Bishop’s identity fluid and always in motion towards multiplicity. This plurality brings newness to Bishop’s poetry and engages the researcher with fresh perspectives of outer vision, inner perception, structural patterns and make her poetry eventful.

New Perspectives on the Neurobiology of Sign Languages.

The first 40 years of research on the neurobiology of sign languages (1960-2000) established that the same key left hemisphere brain regions support both signed and spoken languages, based primarily on evidence from signers with brain injury and at the end of the 20th century, based on evidence from emerging functional neuroimaging technologies (positron emission tomography and fMRI). Building on this earlier work, this review focuses on what we have learned about the neurobiology of sign languages in the last 15-20 years, what controversies remain unresolved, and directions for future research. Production and comprehension processes are addressed separately in order to capture whether and how output and input differences between sign and speech impact the neural substrates supporting language. In addition, the review includes aspects of language that are unique to sign languages, such as pervasive lexical iconicity, fingerspelling, linguistic facial expressions, and depictive classifier constructions. Summary sketches of the neural networks supporting sign language production and comprehension are provided with the hope that these will inspire future research as we begin to develop a more complete neurobiological model of sign language processing.

David Vikgren versus Antti Keksi

David Vikgren Versus Antti Keksi: The Work of Orature in Phonemic poetry
 David Vikgren (1975–) has made a phonemic elaboration of an oral poem by Antti Keksi (1677). Both these poets are situated in Torne Valley in the very north of Sweden. This region was colonised for centuries. For example, the language Meänkieli was forbidden in schools. Nowadays, efforts are made to restore Meänkieli, so David Vikgren’s phonemic work with Keksi’s oral poem has political implications. However, this is also an interesting piece of language materialism, where Vikgren treats the Meänkieli text according to three principles, as antonym, as anagram, and as homophony. This article suggests a method for the signification process of homophony. The semantic meaning of words changes considerably when phonemes are allowed to dominate and reading finds its base in sound associations.

Decomposing spontaneous sign language into elementary movements: A principal component analysis-based approach.

Sign Language (SL) is a continuous and complex stream of multiple body movement features. That raises the challenging issue of providing efficient computational models for the description and analysis of these movements. In the present paper, we used Principal Component Analysis (PCA) to decompose SL motion into elementary movements called principal movements (PMs). PCA was applied to the upper-body motion capture data of six different signers freely producing discourses in French Sign Language. Common PMs were extracted from the whole dataset containing all signers, while individual PMs were extracted separately from the data of individual signers. This study provides three main findings: (1) although the data were not synchronized in time across signers and discourses, the first eight common PMs contained 94.6% of the variance of the movements; (2) the number of PMs that represented 94.6% of the variance was nearly the same for individual as for common PMs; (3) the PM subspaces were highly similar across signers. These results suggest that upper-body motion in unconstrained continuous SL discourses can be described through the dynamic combination of a reduced number of elementary movements. This opens up promising perspectives toward providing efficient automatic SL processing tools based on heavy mocap datasets, in particular for automatic recognition and generation.

Representing Race in Graphs: W.E.B. Du Bois, Corporate Bureaucrats, and Visualization Strategies for Change

In this paper, I draw together W. E. B. Du Bois and corporate bureaucrats to compare the graphical representation of race across three distinct racial epochs: the Progressive, Civil Rights, and post-1980s neoliberal era. I illustrate how, through visual and rhetorical strategies, corporate bureaucrats extend a Du Boisian legacy in constructing popular knowledge of race and racism. I show how they do this by making whiteness visible through data visualizations and rhetorically bundling them to liberal American values of equal opportunity. In examining them as epistemic and semiotic objects, I argue that graphical representations of race compel the enactment of meaningful strategies seen to challenge racial inequalities in the workplace. Yet, insofar as these are employed to equate racism with the absence of equal opportunity in capitalist firms, I argue, they also mask whiteness and reproduce systemic racism. The graphical representation of race, in effect, reveals how the practices of knowledge production and processes of signification are entangled in everyday corporate bureaucracies. Thus, I suggest that we need to reject analytical binaries that pose a bounded distinction between “business” and “social justice” to extend research into the cultural production and productive enactment of racial materiality.

Evidence for visual simulation during sign language processing.

What are the mental processes that allow us to understand the meaning of words? A large body of evidence suggests that when we process speech, we engage a process of perceptual simulation whereby sensorimotor states are activated as a source of semantic information. But does the same process take place when words are expressed with the hands and perceived through the eyes? To date, it is not known whether perceptual simulation is also observed in sign languages, the manual-visual languages of deaf communities. Continuous flash suppression is a method that addresses this question by measuring the effect of language on detection sensitivity to images that are suppressed from awareness. In spoken languages, it has been reported that listening to a word (e.g., "bottle") activates visual features of an object (e.g., the shape of a bottle), and this in turn facilitates image detection. An interesting but untested question is whether the same process takes place when deaf signers see signs. We found that processing signs boosted the detection of congruent images, making otherwise invisible pictures visible. A boost of visual processing was observed only for signers but not for hearing nonsigners, suggesting that the penetration of the visual system through signs requires a fully fledged manual language. Iconicity did not modulate the effect of signs on detection, neither in signers nor in hearing nonsigners. This suggests that visual simulation during language processing occurs regardless of language modality (sign vs. speech) or iconicity, pointing to a foundational role of simulation for language comprehension. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Grounding the Theory of Discursive Resistance: Language, Semiotics and New Testament Theology

Focusing on semantics and semiotics, this article will suggest new and renewed approaches to studying the construction of New Testament theology. First, the relation between Saussure and Peirce will be analyzed because the interpretation of their relationship is crucial for understanding the process of signification. A critical stance will be taken towards Derrida and Eco’s interpretation of signification and towards deconstruction. Applying Benveniste’s development of Saussure’s semantics will introduce a discursive theory. Linguistic signs are not simply linguistic units as such. A sign is about conditions and functions. A sign as a role is a manifestation of participation. For anything to serve as a sign entails participation in a web of relations, participation in a network of meanings, and adoption of a set of rules. In the act of encoding there are elements that resist the free selection of components in encoding, such as narratives and metaphors. Therefore, they also become a means of appropriation: the construction of the sentence is not spontaneous but constrained. When, for instance, the metanarrative of enthronement directs the construction of a Christological statement, the basic theme dominates the process and becomes compelling for the ancient author.

Promoting Epistemic Vigilance Against Racist Attacks on Africans on Mediatised Permanent Education Sites

This paper analyses how tropes of racism against people of African descent manifest in public texts (permanent education sites) of different genres and forms. The selected permanent education sites include the mediatisation of the public and private lives of the British royal family’s Prince Harry and Meghan’s break away from royal duties, the FIFA World Cup and the English Premier League, the Chinese Museum in Hubei, and other public incidences that provoked racial controversies. These activities, controversies and exhibitions are vehicles through which racism is reproduced as part of the historical global capitalist system. This is the ubiquitous public pedagogy of permanent education. The porous but multifarious dominant sitesdiffuse diverse forms of pedagogical address to put into play a limited range of identities, ideologies, and subject positions that both reinforce neoliberal social relations and undermine the capacity for democratic politics. Critical scholarship has a normative duty to be mindful of all cultural activities and productions, their processes of signification, and implications for humanization and democracy. After conveniently selecting specific mass mediated permanent education sites, this paper theorises and historicises the colonial foundations of racism and the European construction of an imagined racial hierarchy. It then problematises China as an emergent global economic powerhouse located in this gradation, and its contemporary self- identities – both official and public – given notable allegations of incidences of Chinese racism against people of African descent. Popular sites of mediation are not studied for their own sake, but to develop a radical pan-African scholarship strategically positioned to influence public education sites for social transformation and justice. Such vigilant critical decolonial scholarship makes necessary com onse to insidious sites ocultural education.

ГЕНЕАЛОГИЯ ЖЕСТА: ОТ КИРЫ МУРАТОВОЙ К РЕНАТЕ ЛИТВИНОВОЙ

The long-term creative collaboration between Kira Muratova and Renata Litvinova began with the film “Uvlechen’ya” (“Pastimes”, 1994), in which Litvinova took part both as an actress and a screenwriter. Since then, Litvinova has become one of the most striking personifications of Muratova’s ornamental film style, which brings about a specific regime of visibility by foregrounding the eccentric corporeality of non-professional actors, or gesturality as a category of bodily and speech performance. This article focuses on the primary scene of gestural genealogy linking the two directors: thus, the pathologist’s gesture from nurse Lilya’s monologue, written and recited by Litvinova in her inner eccentric manner in Muratova’s film, unfolds in a full-length film narration of “Rita’s Last Fairy Tale” (2012), with a phantasmagorical plot and spectacular visuals characteristic of Litvinova’s directorial style. The article addresses, on the one hand, this gesture, expressing concisely the manifold and bidirectional relation between Muratova’s and Litvinova’s films, and, on the other, discusses possible ways of theoretical conceptions of gesture in text and cinema. Gesture is conceived of as a borderline figure of speech and/or of body, aimed at an absent object, whereby the grasping function of the hand makes gestures to a figure of metalepsis which, translated into the language of cinema, emphasises the haptic character of the image. The missing object around which gesticulation arises leads to a discussion of the problematic status of gesture as a sign as well as to the disturbed process of signification during its interpretation. Since gesture only indicates and signals but does not signify, one can speak of the semiotic function of monstration. A gesture appears as a monster in the literal sense of the word, that is, the one who shows itself and, in so doing, warns. Thus, using gestures, to some extent requires adopting the position of a monster – to designate by putting oneself on the show, by making oneself the object of spectacle. Both films and the figure of Litvinova therein are viewed through the prism of monstrosity of gesture and language – it is through the disjunction between showing and speaking that gesture becomes exposed as a pure medium.

Language modality and temporal structure impact processing: Sign and speech have different windows of integration

Language comprehension depends on the ability to temporally process the periodic structure of the language signal. In this study we investigate temporal processing of Spanish Sign Language (LSE), isolating the specific contribution of signal modality through a comparison with spoken Spanish and the contribution of linguistic status by comparing language with a non-linguistic temporally structured visual signal. Twenty-three highly proficient hearing users of LSE performed an intelligibility task with these three types of materials, manipulated with different levels of temporal distortion. The results show that the distortion differently affects the intelligibility of these signals. Spanish is characterized by a threshold of temporal distortion, beyond which intelligibility rapidly decreases and is almost completely lost. Conversely, in LSE and the visual non-linguistic task greater temporal distortion led to a gradual and constant reduction in intelligibility with no clear threshold. LSE is more resilient to temporal manipulation compared to the visual non-linguistic signal: participants’ performance never drops below 50% even with the most severe distortion. Overall, these findings suggest that the temporal processing of language arises from the complex interaction between the properties of the sensory system and the special characteristics of the language signal.

Academic Cultures: Psychology of Education Perspective

This work contributes to a wider range of research, the main objective of which is to investigate models of thought and behaviour that result from belonging to a given academic culture. The academic culture that will be examined is that within the university system, and this research will look at how this culture can take different forms. According to (Bourdieu, 1984), university professors hold an institutionalised form of cultural capital that places them in a dominant position within a field of power. The university system is immersed in a specific culture, and it expresses a given culture and understanding this culture will allow one to understand the system itself (Anolli, 2014). Cultural models are the result of a process of signification, which is understood as the ability of a group or community to elaborate a shared symbolic dimension around an object or symbol at a given historical moment. The cultural models that underlie the professional context orient social and organizational behaviour, which contributes to the construction, on a symbolic level, of identity.

Tra dipendenze intergenerazionali e asimmetrie di genere. Politiche e scelte riproduttive nelle Marche

This essay considers factors affecting low fertility in the Marche region and evaluates their implications for family policies. Low fertility is analysed in the light of the transformations occurred in the last decade, by highlighting the changes in the demographic structure and dy-namics, the modifications produced by the processes of modernization in the ways of making family and parenting, couples’ expectations in the face of social and economic constraints and opportunities characterizing the local contexts. The article pays particular attention to the role of subjective processes of signification in shaping reproductive choices. The methodology combines a quantitative approach, focused on secondary data analysis, with a qualitative one, that allowed the authors to gather the points of view of key informants and couple partners in reproductive age through semi-structured interviews and focus groups.