Non-verbal Resources Research Articles

Trust in interaction studies

Trust is argued to be essential in fostering cooperative communication, whereas a lack of trust is seen as detrimental to these aims. Over the years, there has been a slow but steady stream of research that has aimed to shed light on how trust is accomplished or broken down through discursive-interactional practices. In this mini review, we examine existing studies that take trust as a topic of investigation using micro-analytic, interactional methods, in order to provide readers with an up-to-date overview on new developments in this important field of research. From this review, we conclude that there exist two different, yet complementary, views on trust: Trust as an interactional principle and trust as a discursively accomplished phenomenon. We not only summarize important discursive work that provides a unique lens on how trust may be established and maintained through verbal and non-verbal resources, but also suggest some of the challenges interactional trust research still faces and some important areas for further investigation in which trust is a major concern.

From biblical texts to Balinese paintings

The intersemiotic translation is defined as the interpretation of linguistics signs by means of non-verbal resources. This study aims to investigate the intersemiotic translation of linguistic texts” The Parable of the Ten Virgins” into an image system of Balinese Paintings made by Christian artist Nyoman Darsane. The approach applied in this study is derived from multimodal social semiotic theory in accordance to Halliday’s Systemic Functional Theory (SFT). The source of data is biblical texts which were taken from the New International Version of the Parable of the Ten Virgins of Matthew 25 verse 1 until 13 and its intersemiotic translation in the form Balinese painting. The result of analysis has demonstrated that the metafunctions principle of the different semiotic systems which in this case is texts and painting have complementary and compounding meaning to each other in all levels of meaning including experiential, interpersonal, and textual meanings. Even though there are some untranslated keys of information in those levels of meaning as there are omission strategy implemented from source text to the target image for various reasons such as presenting only essential information.

Speakers Raise Their Hands and Head During Self-Repairs in Dyadic Conversations

People often encounter difficulties in building shared understanding during everyday conversation. The most common symptom of these difficulties are self-repairs, when a speaker restarts, edits or amends their utterances mid-turn. Previous work has focused on the verbal signals of self-repair, i.e. speech disfluences (filled pauses, truncated words and phrases, word substitutions or reformulations), and computational tools now exist that can automatically detect these verbal phenomena. However, face-to-face conversation also exploits rich non-verbal resources and previous research suggests that self-repairs are associated with distinct hand movement patterns. This paper extends those results by exploring head and hand movements of both speakers and listeners using two motion parameters: height (vertical position) and 3D velocity. The results show that speech sequences containing self-repairs are distinguishable from fluent ones: speakers raise their hands and head more (and move more rapidly) during self-repairs. We obtain these results by analysing data from a corpus of 13 unscripted dialogues, and we discuss how these findings could support the creation of improved cognitive artificial systems for natural human-machine and human-robot interaction.

Interpreter visibility in press conferences: a multimodal conversation analysis of speaker–interpreter interactions

This study offers a novel perspective on interpreter visibility by exploring speaker references to interpreters, which differs from previous research that primarily focused on interpreter visibility through their own discourse contributions. Employing a multimodal conversation analysis approach, the study examined the verbal and nonverbal resources utilized by speakers and interpreters in 98 selected excerpts taken from press conference interpreting sessions at the American Institute in Taiwan (AIT). The analysis revealed six distinct topics that denoted the ways in which interpreters were rendered noticeable to the audience through the speaker’s references. These references were context dependent, leading to subsequent speaker–interpreter interactions where interpreters became highly visible. In addition to verbal cues, nonverbal semiotics played a crucial role in demonstrating how interpreters working in rigidly structured press conferences could function as active co-participants of discourse, and how the speaker and interpreter could collaborate to facilitate the interpreter’s visibility and promote a relaxed communicative environment. These findings shed new light on the interpreter’s role, underscoring that it is a dynamic phenomenon requiring analysis in relation to the specific communicative context. This study demonstrated the efficacy of utilizing multimodal conversation analysis as a methodology to explore interactions between speakers and interpreters and to gain a deeper understanding of the complex and nuanced aspects of conference interpreting.

Motion analysis of dyadic talk during joint search and decision-making – A replication study

Human interaction frequently includes decision-making processes during which interactants call on verbal and non-verbal resources to manage the flow of interaction. In 2017, Stevanovic et al. carried out pioneering work, analyzing the unfolding of moment-by-moment dynamics by investigating the behavioral matching during search and decision-making phases. By studying the similarities in the participant's body sway during a conversation task in Finnish, the authors showed higher behavioral matching during decision phases than during search phases. The purpose of this research was to investigate the whole-body sway and its coordination during joint search and decision-making phases as a replication of the study by Stevanovic et al. (2017) but based on a German population. Overall, 12 dyads participated in this study and were asked to decide on 8 adjectives, starting with a pre-defined letter, to describe a fictional character. During this joint-decision task (duration: 206.46 ± 116.08 s), body sway of both interactants was measured using a 3D motion capture system and center of mass (COM) accelerations were computed. Matching of body sway was calculated using a windowed cross correlation (WCC) of the COM accelerations. A total of 101 search and 101 decision phases were identified for the 12 dyads. Significant higher COM accelerations (5.4*10−3 vs. 3.7*10−3 mm/s2, p < 0.001) and WCC coefficients (0.47 vs. 0.45, p = 0.043) were found during decision-making phases than during search phases. The results suggest that body sway is one of the resources humans use to communicate the arrival at a joint decision. These findings contribute to a better understanding of interpersonal coordination from a human movement science perspective.

What do second language speakers really need for real-world interaction? A needs analysis of L2 Chinese interactional competence

A primary undertaking in communicative language teaching is to ensure second language (L2) speakers develop the ability to interact in real-world communicative events, a skill known as interactional competence (IC). Thus far, there have been few needs analysis studies conducted on L2 IC, posing challenges when ascertaining whether the teaching and testing of L2 IC are empirically grounded. Henceforth, the researcher carried out a needs analysis to systematically investigate the IC learning needs of speakers of L2 Chinese, which in itself represents an under-investigated target language in present-day needs analysis literature. A considered methodology was utilized to elicit triangulated information from 18 participants. Content analysis of the elicited data identified seven IC learning categories: (1) social actions, (2) sociopragmatic knowledge, (3) pragmalinguistic knowledge, (4) interactional structure, (5) content knowledge, (6) linguistic resources, and (7) nonverbal resources. Results revealed disaffiliative social actions (e.g. complaining) to be most challenging. The management of social actions was mediated by speakers’ sociopragmatic knowledge (e.g. social distance) and pragmalinguistic knowledge (e.g. formality devices). Other learning categories such as interactional structure (e.g. topic development) and nonverbal resources (e.g. gaze) contributed to successful interaction at a more foundational level. Based on these findings, this article proposes a three-tier IC needs model to conceptualize the hierarchical interrelationships among the seven categories. In addition, the researcher generated lists of IC learning needs for each category and subcategory to assist language educators with developing targeted IC teaching and assessment tasks. The IC needs model and needs lists constitute a pedagogical toolkit for the systematic incorporation of IC in task-based language teaching, language curricula and language programs. This article also discusses how methodological considerations and innovations formulated in the study can contribute to needs analysis research in general.

Doctors, Patients, and Interpreters' Views on the Co-Construction of Empathic Communication in Interpreter-Mediated Consultations: A Qualitative Content Analysis of Video Stimulated Recall Interviews.

Doctors and patients rely on verbal and nonverbal resources to co-construct clinical empathy. In language-discordant consultations, interpreters' communicative actions might compromise this process. We aim to explore doctors, patients, and professional interpreters' perspectives on their own and others' actions during their empathic interaction in interpreter-mediated consultations (IMCs). We analyzed 20 video stimulated recall interviews with doctors, patients, and interpreters using qualitative content analysis. Doctors and patients found ways to connect with each other on the level of empathic communication (EC) that is not limited by interpreters' alterations or disengaged demeanor. Some aspects of doctors and interpreters' professional practices might jeopardize the co-construction of EC in IMCs. The co-construction of EC in IMCs is not only subject to participants' communicative (inter)actions, but also to organizational and subjective factors. These results provide evidence of the transactional process between the behavioral, cognitive, and affective components of clinical empathy in the context of IMCs.

Parole e gesti dell’insegnante nell’ora di italiano a stranieri (e non solo)

Teacher’s words and gestures in didactics of Italian to foreigners&#x0D; (and not only)&#x0D; The notion of contextual continuum&#x0D; This paper starts from an extended concept of “plurilingualism” based on plurimodality in communication (i.e. the use of verbal and nonverbal codes in oral discourses). Assuming that plurilingualism and plurimodality are humans’ natural abilities to adapt verbal and nonverbal resources to the interaction, we propose a methodological approach to the study of teacher’s language and hand gestures in the Italian as a foreign language or L2 classroom (even if our conclusion is valid for the general classroom context). Before discussing the approach, in the first part of the paper we present a brief review of the literature on language and gestures for didactic activities in the contexts of Italian as L1/L2 (or as a foreign language). In the second part the approach called “contextual continuum” is introduced and exemplified. Our aim is to demonstrate that Italian (and not Italian) teacher’s language and gestures in the classroom context are relevant if compared with contexts which are progressively not the classroom. More specifically, in a laboratory context gestures appear significantly simplified and reduced.

Text types of academic writing "Referat/homework" and "abstract/book description" in German and Ukrainian higher education

The article deals with some aspects of academic writing in Ukraine, which is already taught as a special subject in more than a quarter of Ukrainian universities. Differences in the23interpretation of the academic genre "the report" as an oral report at a seminar in the German academic tradition and as a type of written speech of greater volume in the Ukrainian educational tradition (term paper) were considered. It is noted that the term “report” is borrowed from Latin in both languages, but the genre “report” in academic speech shows different characteristics in volume and in the quality “oral / written form”, which in German tradition is a voiced form of writing based paper with the use of nonverbal resources of speaking and modern presentation techniques (visual: pictures, photos, fragments of the movie and audio tools), which demonstrates a hybrid form of the academic genre “report”. The genre of academic writing “abstract” is represented by a number of varieties in both Ukrainian and German traditions of academic speech, but in German it has several designations for these varieties, and in Ukrainian we use one term “annotation (summary)”, which combines all varieties in one generic concept as a philosophical term. An annotation to a scientific conference, an annotation to a scientific article, bachelor's andmaster's thesis, dissertation, monograph, textbook, encyclopedia, reference book, fiction book for sale on the Internet or for sale in a bookstore are considered. However, for all types of genres of academic writing and for types, considered in the proposed article, the characteristic features of scientific style are significant: transparency and brevity of structure (introduction, main part, final part), the high degree of informative speech together with the concise form, the use of the standards of literary language, use of terms and of professional vocabulary, impersonality, logical form of speech, сonsistency of the content expression.

Management of learner-initiated departures through teasing in adult EFL classrooms

ABSTRACT Using Conversation Analysis, this article explores the ways in which teasing is employed as an interactional tool to respond to learner-initiated departures in videotaped adult English as Foreign Language classrooms. The analysis focuses on the moments of classroom interaction where student contributions and behaviours initiate shifts from the ongoing pedagogical trajectory or the behavioural expectations of the classroom. The analyses of the focal extracts reveal that the teachers adopt either a direct or an indirect address mode in their teasing comments, where some non-verbal resources such as gaze, gestures, facial expressions, and embodied enactments are brought into use. The findings of this micro-analytic study contribute both to classroom interaction research and teacher training with an account of how embodied resources are employed for teasing in the management of learner-initiated departures.

Meaning construal through multimodal clusters in the theatrical discourse

The paper presents the results of an empirical study of monomodal and multimodal data collected from the plays by Russian playwrights of the second half of the 20th century (A. Vampilov, A. Kazantsev, N. Sadur) and the theatrical performances based on these plays. The main aim is to find out what semiotic modalities are more frequently involved in the process of recurrent semiosis and, as such, in the multimodal meaning construal. Following the classical as well as contemporary works in the field explored, the research sets out to elaborate the cognitive-semiotic approach to the analysis of the theatre-specific means of multimodal meaning construal to get insights into dynamicity, embodiment and figurativity of artistic expression typical of theatrical performativity and inaccessible to other arts. The data are explored by means of a complex methodology that comprises quantitative and qualitative methods with the diagraph analysis as a key analytical tool which serves to recognize meaningful correlations within and across modalities resulting from mappings between dialogically juxtaposed independent utterances. This paper demonstrates that the involvement of nonverbal resources is mainly manifested in the use of hand and head gestures, facial movements and body repositioning. Next to the quantitative results, the research shows that co-occurring conceptualizations through verbal and nonverbal components, gestures in particular, can trigger off the amplification of meaning or bring about the emergence of new conceptual projections.

Enregistering mask-wearing in the time of a public health crisis

AbstractThis study explores how during the initial outbreak of COVID-19 in Taiwan, mask-wearing emerged as a semiotic register in health communication. Focusing on enregisterment (Agha 2007), I analyze interactional recordings of press conferences by the Taiwanese government, Chinese language lessons offered to international students, and conversations at a convenience store between an Australian student and her Taiwanese friend. The analysis reveals information production and reception on three social scales: public health policies, institutionalized health instruction, and interpersonal-care practices. On each scale, masking is enregistered as a semiotic sign associated with particular social actions, actors, and relations. This enregisterment transpires through an orchestration of verbal and non-verbal resources, conveying ideologies about invisible viruses in everyday life. While inconsistencies exist across scales, masking is consistently overlaid with civic solidarity and moral responsibility. In times of crisis, mask-wearing is an issue of public health and a process of typifying behaviors, identities, and relationships. (COVID-19, enregisterment, scale)*

Empathic and peacebuilding gestures: an analysis of greeting gestures across cultures

Communication process allows people to receive and send messages through verbal and nonverbal resources which play an important role in healthy interpersonal acts. While verbal communication has been the subject of many studies, the present study aims to focus mainly on the nonverbal aspect that is greeting gestures. In this article we shall analyze which greeting gesture, that is widely used across different cultures may evoke a feeling of empathy and thus build peaceful interactions so needed in human communication nowadays.

The Non-verbal Resources of Linguistic Landscape in Dali Ancient City Tourist Attraction, China

The aim of this study is to call for more attention to the non-verbal resources of linguistic landscape research and also provide a certain reference for implement of various countries in the other scenic area of the world. The data collection technique is the comprehensive photographing of signs in various shops, hotels, restaurants at the streets of survey areas. The equipment used is a digital camera. The sampling technique is carried out by random sampling. In order to get the diversity, a wide range of signs is selected. Through analyze, the non-verbal resources of the linguistic landscape of Dali ancient city tourist attraction is that official signs appropriately use the non-text marks to better promote local culture. The use of colors highlights text messages and makes people visually comfortable, in line with Chinese aesthetic habits. Most of the signs have radically abandoned traditional shapes and used non-traditional shapes that are more distinctive and can convey cultural information, which are worthy of promotion. Since the combined sign is formulated in accordance with the official sign standard, the shape and color are more in line with the official standard. The non-text marks on the signs are customized privately, which is more diverse than the unified official watch signs. The non-verbal resources of private signs pay more attention to the transmission of information, and more use of phone numbers combined with brand logos makes the linguistic landscape an advertising effect and maximizes the mass communication of private brands. In addition, in order to comply with the official requirements, the color and shape of the signage do not differ much from store to store. It is worth mentioning that the official signage has a more long-term development perspective, by using the QR code which is very popular nowadays, but the scope of the implement is limited.

Voice and Communication in News Anchors: What is the Impact of the Passage of Time?

IntroductionNews anchors, as high-performance voice professionals, may be subject to voice and communication changes throughout their careers. ObjectiveTo compare the overall degree of vocal deviation, use of vocal and nonverbal resources, pleasantness and expressiveness of Brazilian journalists - news anchors - at 2 time points: at the beginning of their career and late in their career, when they are at an old age. MethodsA longitudinal observational study that analyzed voice samples collected indirectly from 20 professional male journalists at 2 time points with a minimum interval of 25 years: at the beginning of the career (T1) and at present (T2). The journalists were 60 years old or older, and the samples were narratives with similar communicative content. The overall degree of vocal deviation and use of vocal resources (presence/absence of vocal alteration, vocal attack, loudness, pitch, resonance, articulation of speech sounds, speech speed and breath-speech coordination) and nonverbal resources (emphasis, pauses, rhythm and loudness and pitch variations) were analyzed. Nine judges analyzed the expressiveness and pleasantness of the participants’ voices. The results. The mean age of the participants was 38.6 years at T1 and 73.7 years at T2. Differences observed between T1 and T2 included greater overall degree of deviation at T2; greater presence of vocal alterations; and worsening of vocal attack, loudness, pitch, articulation and breath-speech coordination. A directly proportional correlation between pleasantness and expressiveness and an inversely proportional correlation between pleasantness and expressiveness and the overall degree of vocal deviation were observed. ConclusionWorsening of some vocal aspects was observed; however, the participants’ nonverbal characteristics, pleasantness and expressiveness were unchanged throughout their careers.

The Strategic Use of Visuals on Facebook: A Multimodal Analysis of Images and Audience Reactions During the Campaign for the 2019 UK General Election

The article studies the images that British political parties posted on Facebook during the 2019 UK elections. Verbal and nonverbal resources in the visuals were examined to look at the following strategies: Broadcasting Policy, Mobilization, Attacking Opponents, and Image Management. Overall, attacking opponents was the most common strategy for verbal and nonverbal resources. Attacks against opponents were also associated with a higher number of likes and shares. A substantial portion of text in the images was used to talk about policies, in particular by the two largest parties, the Conservative Party and Labour Party. Visuals posted on Facebook by the Labour Party generated the highest number of audience reactions. Specific differences among political parties were also analyzed and discussed.

Video remote interpreting in clinical communication: A multimodal analysis

ObjectiveInvestigating how the spatial and audiovisual conditions in video remote interpreting (VRI) shape communicative interaction in a language-discordant clinical consultation. MethodsWe conducted a multimodal analysis of an authentic VRI-mediated consultation with special reference to spatial arrangements, audiovisual conditions, and the healthcare professional’s use of embodied communicative resources (body orientation, eye gaze, gestures). ResultsThe physician is found to pursue his communicative goals for the consultation by first creating an appropriate spatial and technical environment and then supporting his information-giving and relationship-building actions through the use of nonverbal (embodied) resources like body orientation, gaze and gestures as well as specific turn-management behaviour. ConclusionVRI allows healthcare professionals to access professional interpreters for language-discordant consultations but requires appropriate technical and spatial arrangements as well as users capable of adapting their communicative behaviour to spatial and audiovisual constraints. Practice implicationsAlongside telephone interpreting, VRI is the solution of choice for language-discordant clinical encounters in times of the Covid-19 pandemic. Its use requires appropriate technical and spatial arrangements as well as specific skills on the part of healthcare professionals to cope with inherent audiovisual constraints.

Gaze and other non-verbal resources in student clarification requests: a micro-analytic investigation

ABSTRACT In addressing vocabulary explanations in language classroom settings, a growing number of studies have examined the different ways teachers respond to the vocabulary explanation requests made by students in second language (L2) classrooms. Relatively lacking is an investigation into the nature of vocabulary explanation requests made by students in L2 classroom settings. Based on 14 hours of (50 minutes each) video-recorded data collected in an adult L2 Oral Communication classroom, we discovered that students incorporate gaze and other non-verbal resources in their requests for the clarifications of L2 vocabulary items in three distinct ways: (1) using gaze and gesture with formulaic expressions; (2) using gaze and mimics with the repetition of problematic vocabulary item; and (3) using gaze without talk. Findings contribute to the existing literature with a specification of gaze and other non-verbal resources in the problematisation of vocabulary items and formulation of vocabulary explanation requests.

Sociosemiotic approach to teaching foreign language listening: linguodidactic implications of adopting views on interrelation of verbal and non-verbal semiotic resources in communication.

The following article raises the issue built around teaching foreign language listening by the restrictions imposed by the theory of oral activity that influences the view on the nature of communication and human social behavior. In particular, the conducted analysis of the theoretical underpinnings of social semiotics and multimodal approach to human communication gives scientific credence to assuming the meaning-making potential of non-verbal language comparable to the one of verbal language. In addition to that, it becomes evident that non-verbal language has a tendency to complicate inferencing the meaning of the verbal activity rather than merely simplify it. As a consequence, the linguodidactic basis of teaching listening necessitates revision in this vein. Grounded on multimodal and sociocognitive approaches, the idea is put forward to interpret foreign language listening as a constituent part of sociocognitive activity, the distinguishing features of which are embodied perception and orientation to embodied meaning-making. On the basis of aforementioned, the article enriches psycholinguistic theory of listening with aspects of aural-visual (multimodal) perception as well as offers a view on the alignment of semiotic resources to produce meaning. As a result, there were formulated a set of specific methodological principles intended to govern the teaching process of intertwined aural-visual perception and inferencing of cross-mode meanings and subsequently ensure high performance in foreign language listening in the context of multimodal communication.

Let me listen to where you are. Spatial dimension resources in audio stories can increase imagery, transportation, attention, and recall

ABSTRACT This study analyzes the influence of the audio spatial dimension on the listener’s cognitive processing. Participants listened to different audio stories with four sound verbal and nonverbal resources (dialogue; dialogue and sound effects; dialogue and sound filters; dialogue and sound shots) representing the space and perspective of the scene. They rated the stories concerning imagery, transportation, attention, and performed a cued recall test. The results showed that the audio stories with nonverbal sound resources achieved higher imagery and transportation levels than stories with verbal information. The stories with nonverbal resources, except sound effects, obtained the highest levels of attention and recall. These results reveal the crucial role of the spatial dimension in audio processing.