Textual Cues Research Articles

How textual and picture cues influence motor skill learning in short instructional videos: Evidence from eye-tracking experiments

Three eye-tracking experiments were designed to determine the influence of textual and graphic cues on motor skill learning in short instructional videos. Experiment 1 (n = 45) manipulated the colour of picture cues (red arrows, white arrows, and without cue). Experiment 2 (n = 49) manipulated the position of textual cues (image adjacency, image distance, and without textual cues). Experiment 3 (n = 67) examined the effect of combined textual-picture cues and conducted a 2 (Picture Cue: red arrow vs. white arrow) × 2 (Text Cue: image adjacency, vs. image distance) between-subjects design. Results showed that adding picture cues and textual cues helped learners do better with knowledge tests, reduced cognitive load, and spent more time focusing on the instructional video. Moreover, providing both red arrows and text cues away from the image proved to be an effective combination that achieved better learning outcomes.

How can I trust you if you’re fake? Understanding human-like virtual influencer credibility and the role of textual social cues

How can I trust you if you’re fake? Understanding human-like virtual influencer credibility and the role of textual social cues

Mitigating product placement effects induced by repeated exposure: Testing the effects of existing textual disclosures in children’s movies on disclosure awareness

Abstract In certain cases, children can react strongly in favor of product placements inserted in movies. Scholars therefore argue that existing disclosures might be one strategy in protecting children from any negative effects. However, disclosure research indicates that viewers should both notice disclosures and, in cases of textual cues, read them. We conducted an experiment with 139 children (Mage = 8.46; SD = 1.12; girls = 53.2 %) and tested the influence of textual disclosure format on disclosure awareness as well as brand-related effects. When comparing with no disclosure, results showed that exposure to an existing textual disclosure positively affects disclosure awareness while exposure to an expanded version of it (i.e., by using more textual elements) did not. Further analyses revealed that prior movie exposure positively affected brand evaluation. However, disclosure awareness reversed this effect. We conclude that children appear to be socialized with the existing textual disclosure and that disclosure awareness might be of importance in shielding children against persuasive influence.

Visual place recognition from end-to-end semantic scene text features.

We live in a visual world where text cues are abundant in urban environments. The premise for our work is for robots to capitalize on these text features for visual place recognition. A new technique is introduced that uses an end-to-end scene text detection and recognition technique to improve robot localization and mapping through Visual Place Recognition (VPR). This technique addresses several challenges such as arbitrary shaped text, illumination variation, and occlusion. The proposed model captures text strings and associated bounding boxes specifically designed for VPR tasks. The primary contribution of this work is the utilization of an end-to-end scene text spotting framework that can effectively capture irregular and occluded text in diverse environments. We conduct experimental evaluations on the Self-Collected TextPlace (SCTP) benchmark dataset, and our approach outperforms state-of-the-art methods in terms of precision and recall, which validates the effectiveness and potential of our proposed approach for VPR.

Beyond Words: Semiotic Strategies in Indonesian Presidential Campaign Posters

This article investigates the semiotic strategies employed in Indonesian presidential campaign posters, utilizing semiotics as a powerful tool to decode symbols and signs in political communication. The study focuses on the verbal and visual aspects of the posters, aiming to uncover their intended meaning, myth, and ideologies. Through a comprehensive analysis of the visual elements, symbols, and textual cues, the research examines the semiotic strategies employed to convey messages and influence voter perceptions. Drawing on theoretical frameworks of semiotics and political communication, the findings reveal how subtle visual and textual choices shape the interpretation and reception of political messages. The implications extend beyond this specific campaign, offering valuable insights for future electoral discourse and visual communication strategies. Understanding the semiotic strategies in political campaigning enhances our comprehension of the complex dynamics of political communication, facilitating a more nuanced understanding of how political messages are constructed and perceived. This study contributes to a deeper understanding of the role of semiotics in political communication and provides a foundation for further research in the field.

EXPRESS: Who Shares Fake News? Uncovering Insights from Social Media Users' Post Histories

We propose that social-media users’ own post histories are an underused yet valuable resource for studying fake-news sharing. By extracting textual cues from their prior posts, and contrasting their prevalence against random social-media users and others (e.g., those with similar socio-demographics, political news-sharers, and fact-check sharers), researchers can identify cues that distinguish fake-news sharers, predict those most likely to share fake news, and identify promising constructs to build interventions. Our research includes studies along these lines. In Study 1, we explore the distinctive language patterns of fake-news sharers, highlighting elements such as their higher use of anger and power-related words. In Study 2, we show that adding textual cues into predictive models enhances their accuracy in predicting fake-news sharers. In Study 3, we explore the contrasting role of trait and situational anger, and show trait anger is associated with a greater propensity to share both true and fake news. In Study 4, we introduce a way to authenticate Twitter accounts in surveys, before using it to explore how crafting an ad copy that resonates with users’ sense of power encourages the adoption of fact-checking tools. We hope to encourage the use of novel research methods for marketers and misinformation researchers.

Different types of textual cues in educational animations: Effect on science learning outcomes, cognitive load, and self-efficacy among elementary students

Different types of textual cues in educational animations: Effect on science learning outcomes, cognitive load, and self-efficacy among elementary students

HindiPersonalityNet: Personality Detection in Hindi Conversational Data Using Deep Learning with Static Embedding

Personality detection along with other behavioral and cognitive assessment can essentially explain why people act the way they do and can be useful to various online applications such as recommender systems, job screening, matchmaking, and counseling. Additionally, psychometric natural language processing relying on textual cues and distinctive markers in writing style within conversational utterances reveals signs of individual personalities. This work demonstrates a text-based deep neural model, HindiPersonalityNet, of classifying conversations into three personality categories (ambivert, extrovert, introvert) for detecting personality in Hindi conversational data. The model utilizes a gated recurrent unit with BioWordVec embeddings for text classification and is trained/tested on a novel dataset, शख्सियत (pronounced as Shakhsiyat) curated using dialogues from an Indian crime-thriller drama series, Aarya . The model achieves an F1-score of 0.701 and shows the potential for leveraging conversational data from various sources to understand and predict a person's personality traits. It exhibits the ability to capture both semantic and long-distance dependencies in conversations and establishes the effectiveness of our dataset as a benchmark for personality detection in Hindi dialogue data. Further, a comprehensive comparison of various static and dynamic word embedding is done on our standardized dataset to ascertain the most suitable embedding method for personality detection.

Do Anglophone Consumers Truly Trust or Simply Tolerate English Advertising? The Mediating Effect of Ethnic Self-Identification on Attitudes toward English Advertising and Advertising Trust

Language plays a crucial role in shaping consumer purchasing decisions, particularly within ethnic communities where cultural backgrounds and linguistic diversity influence perceptions. For companies operating in multilingual societies, strategic decisions regarding language use in branding and advertising are vital. While existing research has focused on visual cues in Western markets, this study fills a gap by examining the impact of textual cues, especially in emerging markets, on ethnic consumer responses. Understanding these dynamics is essential for effective multicultural advertising, enhancing advertising trust and engagement in emerging economies. A survey of 704 non-native English-speaking consumers in South Africa reveals the significant role of ethnic self-identification in mediating the relationship between attitudes toward English advertisements and advertising trust. It also highlights ethnic self-identification as a key moderator in this connection, suggesting that English proficiency alone does not guarantee trust in English advertising campaigns among Anglophone African consumers. Brands operating in this context should not assume that English language proficiency automatically translates to trust in English advertising.

Enhancing smart home interaction through multimodal command disambiguation

AbstractSmart speakers are entering our homes and enriching the connected ecosystem already present in them. Home inhabitants can use those to execute relatively simple commands, e.g., turning a lamp on. Their capabilities to interpret more complex and ambiguous commands (e.g., make this room warmer) are limited, if not absent. Large language models (LLMs) can offer creative and viable solutions to enable a practical and user-acceptable interpretation of such ambiguous commands. This paper introduces an interactive disambiguation approach that integrates visual and textual cues with natural language commands. After contextualizing the approach with a use case, we test it in an experiment where users are prompted to select the appropriate cue (an image or a textual description) to clarify ambiguous commands, thereby refining the accuracy of the system’s interpretations. Outcomes from the study indicate that the disambiguation system produces responses well-aligned with user intentions, and that participants found the textual descriptions slightly more effective. Finally, interviews reveal heightened satisfaction with the smart-home system when engaging with the proposed disambiguation approach.

Misinformation or hard to tell? An eye-tracking study to investigate the effects of food crisis misinformation on social media engagement

Visual misinformation about ongoing contaminated food crises poses a significant threat to organizational well-being and public health, particularly when people share incorrect images on social media. Corrective responses and highly credible media sources as effective strategies geared toward combating crisis misinformation. Extending Lewandowsky and colleagues’ (2012) corrective strategies for debunking misinformation, the concept of visual misinformation, cognitive process, and the theory of social sharing of emotion, this study aims to advance research on visual misinformation in public relations and crisis communication. A 2 (image veracity: incorrect vs. true) x 2 (corrective strategy: simple rebuttal vs. simple rebuttal + fact elaboration) x 2 (source credibility: high vs. low) between-subjects eye-tracking experiment was conducted to test the effects of these features on visual attention and intention to share. Additionally, we explored the mediation effects of emotional surprise and perceived crisis severity on sharing posts. Results showed visual cues (e.g., images and sources) and textual cues (e.g., corrective strategies) led to different allocations of visual attention. We found visual attention significantly mediated the effects of combined corrective messages on sharing. Additionally, feeling surprised also significantly mediated the effects of messages with low credible sources on sharing. This study provides insights into advancing crisis communication theory and offers evidence-based recommendations for health organizations and practitioners to better fight against food crisis misinformation.

Visual Conjoint vs. Text Conjoint and the Differential Discriminatory Effect of (Visible) Social Categories

AbstractDoes learning political candidates’ social categories through visual cues affect voter preferences? This paper explores this question by conducting a visual conjoint survey experiment with 2324 German voters, varying whether respondents received information on candidates through explicit labels or pretested AI-generated candidate pictures. The results confirm our expectations that the way in which social categories are perceived affects preferences, with visual cues having a more significant effect on voter preferences compared to textual cues, leading to more discriminatory preferences for certain social categories. Moreover, we show that the effect of visual cues is moderated by the visibility of social categories, with visible social categories, such as gender binaries and ethnic in-/out-group, eliciting more discriminatory preferences with visual cues. The study sheds light on how visible and invisible social categories affect political candidates’ preferences and emphasizes the importance of considering the intersectionality of social categories and their relationship with ideology.

Multimodal Sensing for Depression Risk Detection: Integrating Audio, Video, and Text Data.

Depression is a major psychological disorder with a growing impact worldwide. Traditional methods for detecting the risk of depression, predominantly reliant on psychiatric evaluations and self-assessment questionnaires, are often criticized for their inefficiency and lack of objectivity. Advancements in deep learning have paved the way for innovations in depression risk detection methods that fuse multimodal data. This paper introduces a novel framework, the Audio, Video, and Text Fusion-Three Branch Network (AVTF-TBN), designed to amalgamate auditory, visual, and textual cues for a comprehensive analysis of depression risk. Our approach encompasses three dedicated branches-Audio Branch, Video Branch, and Text Branch-each responsible for extracting salient features from the corresponding modality. These features are subsequently fused through a multimodal fusion (MMF) module, yielding a robust feature vector that feeds into a predictive modeling layer. To further our research, we devised an emotion elicitation paradigm based on two distinct tasks-reading and interviewing-implemented to gather a rich, sensor-based depression risk detection dataset. The sensory equipment, such as cameras, captures subtle facial expressions and vocal characteristics essential for our analysis. The research thoroughly investigates the data generated by varying emotional stimuli and evaluates the contribution of different tasks to emotion evocation. During the experiment, the AVTF-TBN model has the best performance when the data from the two tasks are simultaneously used for detection, where the F1 Score is 0.78, Precision is 0.76, and Recall is 0.81. Our experimental results confirm the validity of the paradigm and demonstrate the efficacy of the AVTF-TBN model in detecting depression risk, showcasing the crucial role of sensor-based data in mental health detection.

Gender Roles and Mobile Dating Applications: Exploring Links Between User Characteristics and Traditional Gender Expressions in Self-Presentations.

This linkage study examined the prevalence of traditional gender expressions in the textual and visual profile cues on mobile dating applications (MDA) (nbiographies = 396, npictures = 1352) of 396 young adults' (Mage = 22.39 years, SD = 2.86, 73% women) with attention to users' gender, sexual orientation, and platform type. For 184 users (Mage = 22.10 years, SD = 2.91, 75% women) media content data were linked to self-report survey data. Results showed that individuals aligned their self-presentations with traditional gender roles and expectations, and this link depended on their gender. No significant differences according to individuals' sexual orientation or platform type were found. Individuals' (hyper-) gender orientation also related to engagement in traditional gender expressions. Specifically, women with a stronger feminine gender orientation expressed more traditional femininity in their MDA profiles. For men, no significant associations between (aspects of) a masculine gender orientation and expressing traditional masculinity in their MDA profiles were found. Future research should further disentangle selective gendered self-presentations.

Episodic future thinking: A behavioral intervention to promote weight loss in breast cancer survivors.

e22541 Background: Digital behavioral interventions are a scalable method to increase healthy lifestyle engagement. Episodic future thinking (EFT) is a behavior change method where participants mentally simulate positive, detailed, personal, and future-oriented events. EFT targets delay discounting (DD), a behavioral economic principle that measures valuation of long-term, future rewards. Higher DD correlates with bias of immediate gratification and maladaptive health behaviors. EFT can decrease impulsivity and improve body weight (BW) in those with obesity. However, EFT may impact cancer survivors differently. We report results from a phase II trial evaluating the feasibility of 12 week (wk) EFT in overweight/obese BC survivors. Methods: Eligibility included women with non-metastatic BC, BMI ≥ 25 kg/m2 and completion of BC treatment apart from endocrine therapy. All patients (pts) received lifestyle counseling, referral to nutritionist, and logged exercise/diet in MyFitnessPal. Pts were stratified by baseline DD and randomized 1:1 to 12-wk EFT vs control (Episodic Recent Thinking; ERT) in which pts imagine events in the past. Pts received text cues 3x/day during meals. The primary endpoint was adherence to EFT (percentage of smartphone prompts opened and attended to over 12 wk). Secondary endpoints were 12 wk change in BW and DD. A one-sample t-test evaluated adherence in pts regardless of drop-out and compared to an 80% target. A linear mixed effects model was used for secondary endpoints. Results: Pts were recruited from November 2021 - August 2023 (Table 1, n = 46) at The Ohio State University. In total, 17 (74%) in EFT and 19 (83%) in ERT completed the intervention. The average adherence was consistent with the 80% target (EFT 77%, 95% CI = 0.69 - 0.85; ERT 82%; 95% CI = 0.78 - 0.86). BW at 12 wk was available in 17 (74%) EFT and 19 (83%) ERT pts. There was a significant 12 wk change in BW in all pts (-4.0 lbs, 95% CI -6.08 - -1.79, p < 0.001; -2.03%, 95% CI -3.15 - -0.91, p < 0.001) but no significant difference between groups (lbs: -2.28 EFT vs -5.42 ERT, 95% CI -7.41 - 1.14, p = 0.1; %: -1.07 EFT vs -5.20 ERT, 95% CI -4.01 - 0.37, p = 0.1). There was no significant difference in 12 wk change in DD rate between EFT and ERT (ratio 1.56, 95% CI 0.19 - 12.95, p = 0.7). Conclusions: The high adherence supports the feasibility of implementing EFT and engaging digital health for remotely delivered weight loss in BC survivors. BW loss was globally observed, despite low baseline DD in the majority. Further investigation of EFT in combination with lifestyle intervention is needed to formally assess efficacy in this population. Clinical trial information: NCT05012176 . [Table: see text]

Parental concerns about oral health of children: Is ChatGPT helpful in finding appropriate answers?

Artificial intelligence (AI) is becoming an important part of our lives owing to increased data availability and improved power of computing. One of the recently launched modalities of AI, ChatGPT, is being enormously used worldwide for different types of tasks. In medical context, its use is being explored for clinical queries, academia, research help, etc. Further, literature suggests that parents seek information about health of their children using different Internet resources and would surely turn toward ChatGPT for the same, as this chatbot model is easy to use, generates "one" response, and is available without any subscription. ChatGPT generates a response using text cues and applying different algorithms on prepublished literature but is still in its naïve state; hence, it is imperative to validate the generated responses. Accordingly, we planned this study to determine the clarity, correctness, and completeness of some Frequently asked questions (FAQs) about child's oral health, from a mother's perspective. The study design was a vignette-based survey and included a set of 23 questions, for which ChatGPT was interviewed from the perspective of an imaginary parent. The answers responded by ChatGPT were copied "verbatim," and a Google survey form was designed. The survey form was validated and then sent to 15 pediatric dentists, and the responses were mainly collected on the Likert's scale with a provision of one open-ended question aiming to determine "what they would have added" to this generated response as an expert in the field. The responses on Likert's scale were condensed and values ≥4 were considered 'adequate and acceptable' while scores ≤3, were considered 'inadequate'. The generated responses and comments mentioned by different respondents in the open-ended question were critiqued in reference to the existing literature. Overall, the responses were found to be complete and logical and in clear language, with only some inadequacies being reported in few of the answers.

Towards Efficient and Effective Text-to-Video Retrieval with Coarse-to-Fine Visual Representation Learning

In recent years, text-to-video retrieval methods based on CLIP have experienced rapid development. The primary direction of evolution is to exploit the much wider gamut of visual and textual cues to achieve alignment. Concretely, those methods with impressive performance often design a heavy fusion block for sentence (words)-video (frames) interaction, regardless of the prohibitive computation complexity. Nevertheless, these approaches are not optimal in terms of feature utilization and retrieval efficiency. To address this issue, we adopt multi-granularity visual feature learning, ensuring the model's comprehensiveness in capturing visual content features spanning from abstract to detailed levels during the training phase. To better leverage the multi-granularity features, we devise a two-stage retrieval architecture in the retrieval phase. This solution ingeniously balances the coarse and fine granularity of retrieval content. Moreover, it also strikes a harmonious equilibrium between retrieval effectiveness and efficiency. Specifically, in training phase, we design a parameter-free text-gated interaction block (TIB) for fine-grained video representation learning and embed an extra Pearson Constraint to optimize cross-modal representation learning. In retrieval phase, we use coarse-grained video representations for fast recall of top-k candidates, which are then reranked by fine-grained video representations. Extensive experiments on four benchmarks demonstrate the efficiency and effectiveness. Notably, our method achieves comparable performance with the current state-of-the-art methods while being nearly 50 times faster.

Learning Task-Aware Language-Image Representation for Class-Incremental Object Detection

Class-incremental object detection (CIOD) is a real-world desired capability, requiring an object detector to continuously adapt to new tasks without forgetting learned ones, with the main challenge being catastrophic forgetting. Many methods based on distillation and replay have been proposed to alleviate this problem. However, they typically learn on a pure visual backbone, neglecting the powerful representation capabilities of textual cues, which to some extent limits their performance. In this paper, we propose task-aware language-image representation to mitigate catastrophic forgetting, introducing a new paradigm for language-image-based CIOD. First of all, we demonstrate the significant advantage of language-image detectors in mitigating catastrophic forgetting. Secondly, we propose a learning task-aware language-image representation method that overcomes the existing drawback of directly utilizing the language-image detector for CIOD. More specifically, we learn the language-image representation of different tasks through an insulating approach in the training stage, while using the alignment scores produced by task-specific language-image representation in the inference stage. Through our proposed method, language-image detectors can be more practical for CIOD. We conduct extensive experiments on COCO 2017 and Pascal VOC 2007 and demonstrate that the proposed method achieves state-of-the-art results under the various CIOD settings.

The potency of audiovisual attractiveness and influencer marketing: The road to customer behavioral engagement

Digital technologies have signified audiovisual (vs. textual) cues for customers on online booking platforms (e.g., Airbnb and hotels). Predicating on the current literature, this study investigates the influencers' audiovisual attractiveness, customer-influencer interaction, and their consequent customer behavioral engagement using three experimental studies. This research involved an analysis using IBM SPSS 28.0, which included independent sample t-test and One-way ANOVA. Additionally, moderation analysis was conducted using the PROCESS macro model 4. In order, Study 1 reveals that influencers' audiovisual attractiveness at high (vs. low) significantly and directly manipulate customer engagement. In Study 2, using Process Model 4, customer-influencer interaction significantly mediates the relationships between influencers' audiovisual attractiveness and customer behavioral engagement. Last, Study 3 incorporates the moderation of influencers' audiovisual expertise between customer-influencer interaction and customer behavioral engagement. The two-way between-subjects analysis anchors that when influencers’ audiovisual expertise is high (vs. low), and customers having good (vs. bad) interactions, it engenders higher effects on customer behavioral engagement. The findings add to the literature on consumer engagement on booking platforms and underscore the potency of audiovisual attractiveness to engage customers. We also acknowledge the shortcomings and provide recommendations for future studies.

Hierarchical graph contrastive learning of local and global presentation for multimodal sentiment analysis

Multi-modal sentiment analysis (MSA) aims to regress or classify the overall sentiment of utterances through acoustic, visual, and textual cues. However, most of the existing efforts have focused on developing the expressive ability of neural networks to learn the representation of multi-modal information within a single utterance, without considering the global co-occurrence characteristics of the dataset. To alleviate the above issue, in this paper, we propose a novel hierarchical graph contrastive learning framework for MSA, aiming to explore the local and global representations of a single utterance for multimodal sentiment extraction and the intricate relations between them. Specifically, regarding to each modality, we extract the discrete embedding representation of each modality, which includes the global co-occurrence features of each modality. Based on it, for each utterance, we build two graphs: local level graph and global level graph to account for the level-specific sentiment implications. Then, two graph contrastive learning strategies is adopted to explore the different potential presentations based on graph augmentations respectively. Furthermore, we design a cross-level comparative learning for learning local and global potential representations of complex relationships.