On-screen Interaction Research Articles

Influence of semiotic resources on peer interactions during collaborative digital multimodal composing

Collaborative digital multimodal composing (CDMC) has recently gained traction in the advent of technology-enhanced language learning, yet scant attention was paid to the influence of semiotic resources on the interaction patterns between learners. The present study attempted to fill the research gap by examining the interactions between English as a Second Language (ESL) learners in an online English for Academic Purpose (EAP) course. Using a multiple case-study design, three pairs of undergraduates completed a collaborative multimodal writing and video-making task for presentation using ZOOM. Influence of semiotic resources can be unveiled through manipulating the mode of presentation. Data of this study includes the verbal exchanges and non-verbal on-screen interactions between the participants while they are working on the tasks. Their interaction patterns were analyzed through conversation analysis and two aspects of collaborations, namely equality and mutuality, were also examined. Their interaction patterns were found to be strikingly different across both tasks, and languaging mainly serves the function of verbalization of content-related issues, such as searching for information and assembling various multimodal elements, with only rare instances of either grammar-based or lexis-based language-related episodes (LRE). Another striking finding is that pairs who are working collaboratively with balanced division and mutual contribution are less susceptible to our manipulation, as compared to those who are demonstrating a dominant-dominant or expert-novice working pattern. Implications of these findings on fostering peer collaborations during CDMC are discussed.

Teaching sign language remotely in the COVID-19 pandemic

ABSTRACT Like language teachers everywhere, Deaf teachers of New Zealand Sign Language (NZSL) had to suddenly move classes online when COVID-19 restrictions were implemented from March 2020. NZSL is conventionally taught through a direct immersion, communicative approach, so adapting instruction to a remote mode required NZSL teachers to develop new technical skills, and adapt teaching formats and strategies to accommodate the constraints of on-screen interaction and learning. The combination of a visual language produced in three dimensions with a screen-mediated mode of interaction creates particular challenges for sign language teachers and learners. This article reports a qualitative enquiry into 14 teachers’ experiences and strategies, with the aim of contributing to practice knowledge for sign language teachers, and to extend the literature on emergency remote teaching (ERT) addressing the teaching of community and signed languages, particularly in the adult community education sector. The study does not evaluate online teaching outcomes, but rather seeks to understand how NZSL teachers adapted to new challenges and what pedagogical knowledge may be carried forward.

Social Communication in Young Children With Sex Chromosome Trisomy (XXY, XXX, XYY): A Study With Eye Tracking and Heart Rate Measures.

Children with sex chromosome trisomy (SCT) have an increased risk for suboptimal development. Difficulties with language are frequently reported, start from a very young age, and encompass various domains. This cross-sectional study examined social orientation with eye tracking and physiological arousal responses to gain more knowledge on how children perceive and respond to communicative bids and evaluated the associations between social orientation and language outcomes, concurrently and 1 year later. In total, 107 children with SCT (33 XXX, 50 XXY, and 24 XYY) and 102 controls (58 girls and 44 boys) aged between 1 and 7years were included. Assessments took place in the USA and Western Europe. A communicative bids eye tracking paradigm, physiological arousal measures, and receptive and expressive language outcomes were used. Compared to controls, children with SCT showed reduced attention to the face and eyes of the on-screen interaction partner and reduced physiological arousal sensitivity in response to direct versus averted gaze. In addition, social orientation to the mouth was related to concurrent receptive and expressive language abilities in 1-year-old children with SCT. Children with SCT may experience difficulties with social communication that extend past the well-recognized risk for early language delays. These difficulties may underlie social-behavioral problems that have been described in the SCT population and are an important target for early monitoring and support.

Verbal and on-screen peer interactions of EFL learners during multimodal collaborative writing: A multiple case-study

Multimodal collaborative writing has become increasingly prevalent in the advent of technology-enhanced language learning, yet scant attention was paid to the verbal and on-screen interactions between learners during the construction process. This study narrowed the research gap and investigated three pairs of EFL learners’ interactions when completing a multimodal collaborative writing task in an English for Academic Purpose (EAP) course at a university in Hong Kong. Using a multiple case-study approach, we examined the dynamics of peer interactions through their interaction patterns, utilization of semiotic and multimodal resources and functions of languaging. Our analyses showed that the three pairs enacted strikingly distinct patterns of interaction, with the more collaborative partners demonstrating a variety of prompting and feedback skills to facilitate their discussion. Surprisingly, the pairs had strong tendencies to attend to the texts rather than multimodal elements during the writing process. Another striking finding is that even non-collaborative partners occasionally engaged in various forms of private speech to keep track of the writing process. Implications of these findings on fostering peer collaborations during multimodal CW are discussed.

Educators’ Experiences of using multilingual pedagogies during emergency remote teaching: a case study of South African universities

ABSTRACT Since the World Health Organisation declared COVID-19 a global pandemic in 2020, universities worldwide have undergone unanticipated changes in how they operate and deliver the academic programme. Chief among these changes has been a wide-scale adoption of emergency remote teaching (ERT). This phenomenological study draws on critical pedagogy to analyse the experiences of South African academics of how the adoption of ERT influenced their use of multilingual pedagogies. Seven lecturers from seven different South African universities were purposively selected to participate based on their prior use of multilingual pedagogies. They teach in fields such as politics, history and education, with some based in traditional universities and others at universities of technology. The participants’ reflections reveal that they essentially found using multilingual pedagogies amid emergency remote teaching very challenging. The challenges can be grouped into three categories: shifts from in-person to on-screen interactions; shifts in the types of multimodal resources used to teach multilingually; and shifts from approaches that intellectualise indigenous languages to approaches that are focused on delivery. According to these educators, multilingual teaching online, particularly in the context of emergency remote teaching, does not bode well for the intellectualisation of indigenous South African languages.

Multilingual and multimodal mediation in online intercultural conversations: a translingual perspective

This study explored peer groups’ mediation processes in the digital space. Building upon translingual research and multimodal discourse analysis, the researcher analysed video transcript excerpts of oral conversations and on-screen interactions among university students of different linguistic and cultural backgrounds. This article focuses on multilingual and multimodal mediation practices, which include polyaccented exemplification, bilingual interpretation, intercultural comparison, transnational definition, polyphonic performance and multimodal instruction (N = 24). The results show that the students attended to dynamic uses of English, Chinese and other languages as multilingual franca while communicating affect and stance through embodied expressions and shared screens in stylised multimodal discourses. The translingual analyses emphasise the students’ collaborative transformations of English as a lingua franca (ELF) communication into playful interactions within digitally mediated networks of relationships. This study suggests that online peer mediation provides useful methods for developing university students’ translingual awareness and capability for participating in intercultural conversations.

Racism and silencing in the media in Aotearoa New Zealand

Racism in the Aotearoa New Zealand media is the subject of scholarly debate that examines how Māori (Indigenous Peoples of New Zealand) are broadcast in a negative and demeaning light. Literature demonstrates evolving understandings of how the industry places Pākehā (New Zealanders primarily of European descent) interests at the heart of broadcasting. We offer new insights by arguing that the media industry propagates a racial discourse of silencing that sustains widespread ignorance of the ways that Pākehā sensibilities mediate society. We draw attention to a silencing discourse through one televised story in 2018. On-screen interactions reproduce and safeguard a harmonious narrative of settler–Indigenous relations that support ignorance and denial of the structuring force of colonisation, and the Television Code of Broadcasting Practice upholds colour-blind perceptions of discrimination and injustice through liberal rhetoric. These processes ensure that the media industry is complicit in racism and the ongoing oppression of Indigenous peoples.

How Wide Is Enough? Effects of Screen Height, Task Type, and Hand Length on Rollable Display Requirements

Although rollable displays must be unrolled for on-screen interaction, it is unknown whether screen height, task type, and hand length affect rollable display requirements. This study examined the effects of screen height, task type, and hand length on the rollable display requirements. A total of 30 young individuals (22.9 ± 2.3 years; 10 in each hand-length group) performed three tasks (web searching, video viewing, and e-mail composition) using three prototypes with different screen heights (H) of 50, 120, and 190 mm. Dependent variables were preferred screen width, preferred screen width-to-height aspect ratio, user satisfaction, gripping comfort, device portability, design attractiveness, and gripping method. As screen height increased, the preferred screen width increased, but the preferred screen aspect ratio decreased. The 95 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">th</sup> -percentile screen width (aspect ratio) of 100 mm (2:1) was required for 50H versus 204 mm (1.7:1) for 120H and 304 mm (1.6:1) for 190H. The highest 95 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">th</sup> -percentile screen aspect ratio of 1.9:1 was required for video viewing. The long-hand-length group preferred significantly wider screens for 190H only. Bilateral grasping was predominantly used for 50H and 120H, whereas non-grasping was for 190H due to limited thumb reach and insufficient screen reaction force. Considering user satisfaction, device portability, and design attractiveness, 120H was recommended, and a screen aspect ratio of 2:1 appeared sufficient for the performance of three mobile tasks on a 120H rollable screen.

Mapping Zazie’s Paris in Louis Malle’s Zazie dans le métro (1960)

ABSTRACTThis article analyses the on-screen interaction between the gamine Zazie and the city of Paris in Louis Malle’s Zazie dans le métro, an overlooked aspect of the film whose critical reception has mostly focused on the topic of adaptation. Through a close reading of Zazie’s spatial practices, it shows that her fragmented and chaotic trajectories in the urban space actively subvert the city’s iconicity. The protagonist gravitates towards affective vortexes in the urban space and gets lost in powerful dérives, which reveal Surrealist and Lettrist/Situationist influences. Zazie’s cartography of Paris plots a correspondingly tumultuous terrain with disjointed juxtapositions, circular movements, and Surrealist predispositions. While closely mirroring the protagonist’s feelings, this space also engages with the city’s previous roles on screen. Finally, the article argues that Zazie’s freedom of movement, anchored in a real and vibrant Parisian landscape, screens a rare image of female subjectivity in urban space.

What Happens When You Click and Drag: Unpacking the Relationship between On-Screen Interaction and User Engagement with an Anti-Smoking Website

ABSTRACTInteractive media provide a physically active experience for users to click, slide, mouseover, and zoom-in/out, but how this increased amount of on-screen interaction is associated with cognitive and attitudinal outcomes has remained unexplored. We tackled this issue by conducting an online study where we recorded the amount of on-screen interaction on a targeted interactive feature, sliders, and correlated it with user engagement, attitudes toward anti-smoking messages, and smoking outcome beliefs, while controlling their baseline smoking outcome beliefs (N = 149). We found that the number of clicks and drags on sliders was positively associated with their attitudes toward anti-smoking messages and smoking outcome belief, but only among nonsmokers who are power users of technology. An increase in perceptual bandwidth influenced by interacting with the slider was significantly correlated with greater user engagement among these users. In contrast, for smokers who are not power users, greater on-screen interaction showed a negative correlation with their user engagement mediated by an increase in defensive processing.

Implementation of an augmented reality interface to reproduce and compare soundscapes

Soundscape design is a growing concern in architectural practice; yet, conveying soundscape properties is a challenge. Quantitative measures like decibel levels offer little utility; birdsong and squealing brakes may be at the same frequency and level, yet one is obviously preferable. Furthermore, visual qualities of the scene have a substantial effect on perception of the acoustic environment. To improve communication between acoustic designers, architects, and clients, an augmented reality interface has been designed to allow comparisons of soundscapes and their relationships to the built environment. The interface consists of a physical scale model, printed data, and a tablet application. The augmented reality application uses a machine vision algorithm to recognize the model and allow on-screen interaction. On the tablet, measurement points are displayed on a live image of the model at their respective locations, and the user may select a point by touch, which will begin sound playback and display an immersive image of the scene. The participant can then look around a spherical image of the scene using the tablet as a movable window, and thereby listen to the environment with appropriate accompanying visual cues. The interface has been a useful tool to communicate urban sound issues.

User Experience of On-Screen Interaction Techniques: An Experimental Investigation of Clicking, Sliding, Zooming, Hovering, Dragging, and Flipping

From scrolling and clicking to dragging, flipping, sliding, hovering, and zooming, the wide array of interaction techniques has vastly expanded the range of user actions on an interface. Each of these interaction techniques affords a distinct action. But do these techniques differ in their ability to engage users and contribute to their user experience? Furthermore, do they affect how users view the content and how much they learn from it? We address these questions via two between-subjects laboratory experiments. Study 1 (N = 128) investigated the relative effects of six on-screen interaction techniques (click-to-download, drag, mouseover, slide, zoom, and 3D carousel) on users' assessment of—as well as their engagement with—an informational website. The site for each condition was identical in content and design, except for the interaction technique used, so that we could isolate the effects of each technique on various cognitive, attitudinal and behavioral outcomes. Study 2 (N = 127) examined the relative effects of four combinations of interaction techniques (slide+click, slide+mouseover, drag+mouseover, and drag+zoom) on the same dependent variables. Data from Study 1 suggest that although the 3D carousel generates more user action, the slide is better at aiding memory. The zoom-in/out tool was the least favored, whereas the mouseover feature fostered greater engagement among power users. Findings from Study 2, which was conducted with a different content domain, replicated the positive effects of slide and negative effects of drag in influencing user experience. Path analyses, using structural equation modeling, revealed the importance of users' assessment of the interface (perceived levels of natural mapping, intuitiveness, and ease of use), which can have significant consequences for user engagement as well as resulting attitudes and behavioral outcomes. Design insights, theories, and techniques to test and capture user experience are discussed.

THE PERFECT CUT

This study examines how politicians act as sources on Dutch television news. It argues that due to the mediatization of politics and a shift towards more interpretive forms of journalism, journalists use politicians' quotes and sound bites first and foremost to support their interpretation of news events. Previous research has shown that because of the growing importance of media logic, journalists are more autonomous and powerful in their relations with sources. This case study shows, however, how the format of news items, especially the use of interviews and quotes, supports the interpretive nature of television news. While there is less on-screen interaction between journalists and politicians on television news, interviews are cut into short sound bites of politicians without the context of the actual interview. Detached reporting of what politicians say because of its newsworthiness has become less important than fitting suitable quotes into predetermined news frames. The analysis is based on a case study of the 2010 local council election coverage by the two major Dutch television news programs, NOS Eight O'Clock News (NOS Achtuurjournaal) and RTL News (RTL Nieuws).

‘A TANGLE OF PEOPLE MESSING AROUND TOGETHER’

This paper explores how Taiwanese women's variety television shows mediate social expectations about women's work, both waged and unwaged. In particular, I argue that the 2011 women's variety programme Shounü Buman Zu 《熟女不 滿族》, lit. unsatisfied mature-women tribe) negotiates both the feminization of everyday domestic risk and expectations surrounding the embodied practices of women's affective labour. I propose that the programme's mediation of these issues takes place at three levels: thematic, performed and formal. First, offering ‘expert’ advice on managing the opportunities and risks associated with women's everyday lives in late-modern consumer culture, the discussion themes on Unsatisfied Shounü readily lend themselves to analysis in terms of the feminization of risk and risk management. Second, the embodied performances of the programme's presenters – including their speech, movement and on-screen interactions – model hegemonic formations of feminine habitus, especially the emotional work it involves. Third, the formal properties of this genre, especially the use of post-production effects, instantiate an amplification of the bodily-affective work of normative feminine social interaction. In representing the labours of femininity on these three levels, these programmes make readily visible some of the defining features of available formations of feminine subjecthood in Taiwan today, while highlighting commercial television's role in the socialization of women's affective capacities.

"Gently Caress Me, I Love Chris Jericho": Pro Wrestling Fans "Marking Out"

"Gently Caress Me, I Love Chris Jericho": Pro Wrestling Fans "Marking Out"