Agent Trust Research Articles

Making a human's trust repair for an agent in a series of tasks through the agent's empathic behavior

As AI technology develops, trust in agents (including robot, AI and anthropomorphic agent) is becoming more important for more AI applications in human society. Possible ways to improve the trust relationship include empathy, success-failure series, and capability (performance). Appropriate trust is less likely to cause deviations between actual and ideal performance. In this study, we focus on the agent's empathic behavior and success-failure series to increase trust in agents. We experimentally examine the effect of empathic behavior from agent to person on changes in trust over time. The experiment was conducted with a two-factor mixed design: empathic behavior (available, not available) and success-failure series (phase 1 to phase 5). An analysis of variance (ANOVA) was conducted using data from 200 participants. The results showed an interaction between the empathic behavior factor and the success-failure series factor, with trust in the agent repairing when empathic behavior was present. This result supports our hypothesis. This study shows that designing agents to be empathic behavior is an important factor for trust and helps humans build appropriate trust relationships with agents.

Enhancing user experience and trust in advanced LLM-based conversational agents

This study explores the enhancement of user experience (UX) and trust in advanced Large Language Model (LLM)-based conversational agents such as ChatGPT. The research involves a controlled experiment comparing participants using an LLM interface with those using a traditional messaging app with a human consultant. The results indicate that LLM-based agents offer higher satisfaction and lower cognitive load, demonstrating the potential for LLMs to revolutionize various applications from customer service to healthcare consultancy and shopping assistance. Despite these positive findings, the study also highlights significant concerns regarding transparency and data security. Participants expressed a need for clearer understanding of how LLMs process information and make decisions. The perceived opacity of these processes can hinder user trust, especially in sensitive applications such as healthcare. Additionally, robust data protection measures are crucial to ensure user privacy and foster trust in these systems. To address these issues, future research and development should focus on enhancing the transparency of LLM operations and strengthening data security protocols. Providing users with clear explanations of how their data is used and how decisions are made can build greater trust. Moreover, specialized applications may require tailored solutions to meet specific user expectations and regulatory requirements. In conclusion, while LLM-based conversational agents have demonstrated substantial advantages in improving user experience, addressing transparency and security concerns is essential for their broader acceptance and effective deployment. By focusing on these areas, developers can create more trustworthy and user-friendly AI systems, paving the way for their integration into diverse fields and everyday use.

Principal–agent trust and adoption of digital financial services: Evidence from India

AbstractThe high reliance on cash is a lifestyle for many people, and their low proficiency with smartphone usage acts as an inertial barrier to using digital financial services (DFS). Given low financial or digital literacy, online banking transactions have to be promoted and mediated by DFS agents. This study examines the role of digital platforms, intermediaries such as banks, mobile money agents, payment banks, post offices, and the government's increasing use of digital technologies. This usage of digital platforms is based on the trust relationship between the principals, who are owners of their bank accounts, and agents of DFS, who are intermediaries. A linear probabilistic model is employed to determine the determinants behind the low adoption of DFS between the necessity and nonnecessity groups. We observe that demand‐side factors like education level, identity proof (documentation), and level of smartphone use proficiency increase the probability of a principal's involvement in a digital mode of payment. At the same time, supply‐side factors like digital infrastructure and trust deficit due to unviable commission incentives to DFS agents decrease the principal's likelihood of using digital platforms.

Behave Yourself! Behavioral Indicators of Trust in Human-Agent Teams

As advancements in artificial intelligence accelerate, there is a rise in the complexity and number of autonomous agents placed in human-agent teams (HATs). With this expansion, it is important to understand how trust in agent teammates evolves and is influenced by contextual events. In support of this, significant research has focused on the factors that influence human trust in an agent and elements that negatively impact this trust. In this research, human trust in agent teammates is typically measured via self-report surveys. Although a reliable format, surveys are not without limitations, as they can be disruptive and lack the temporal resolution needed to capture nuanced and dynamic changes in trust. As a result, researchers are adopting alternative measurement approaches that aim to unobtrusively capture measures that correlate with trust using behavioral and physiological sensors that capture measures such as voice pitch and eye tracking. However, most of these studies utilize only one unobtrusive measure, which can be indicative of various states (e.g., heart rate varies with psychological and physical responses). Given the lack of selectivity in such measures, it may be necessary to utilize multiple unobtrusive measures at once, such as real-time recorded behavioral indicators, to effectively capture changes in these indicators that are due to changes in trust over time. This paper investigated the following research questions: (1) do multiple unobtrusive indicators provide a more holistic picture of trust compared to a single behavioral indicator, and (2) are certain behavioral indicators more indicative of self-reported trust at different times during a mission (e.g., before/after a trust violation event)? Results indicated that following a significant event, changes in self-reported trust are associated with specific behaviors related to improving or mitigating perceived deficiencies in their agent teammates. These insights highlight the opportunity to integrate dynamic behavioral measures in trust assessment frameworks, specifically in scenarios where a significant event compromises trust.

In human–machine trust, humans rely on a simple averaging strategy

With the growing role of artificial intelligence (AI) in our lives, attention is increasingly turning to the way that humans and AI work together. A key aspect of human–AI collaboration is how people integrate judgements or recommendations from machine agents, when they differ from their own judgements. We investigated trust in human–machine teaming using a perceptual judgement task based on the judge–advisor system. Participants (n=89\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$n=89$$\\end{document}) estimated a perceptual quantity, then received a recommendation from a machine agent. The participants then made a second response which combined their first estimate and the machine’s recommendation. The degree to which participants shifted their second response in the direction of the recommendations provided a measure of their trust in the machine agent. We analysed the role of advice distance in people’s willingness to change their judgements. When a recommendation falls a long way from their initial judgement, do people come to doubt their own judgement, trusting the recommendation more, or do they doubt the machine agent, trusting the recommendation less? We found that although some participants exhibited these behaviours, the most common response was neither of these tendencies, and a simple model based on averaging accounted best for participants’ trust behaviour. We discuss implications for theories of trust, and human–machine teaming.

Post-COVID inflation and the monetary policy dilemma: an agent-based scenario analysis

AbstractThe economic shocks that followed the COVID-19 pandemic have brought to light the difficulty, both for academics and policy makers, of describing and predicting the dynamics of inflation. This paper offers an alternative modelling approach. We study the 2020–2023 period within the well-studied Mark-0 Agent-Based Model, in which economic agents act and react according to plausible behavioural rules. We include a mechanism through which trust of economic agents in the Central Bank can de-anchor. We investigate the influence of regulatory policies on inflationary dynamics resulting from three exogenous shocks, calibrated on those that followed the COVID-19 pandemic: a production/consumption shock due to COVID-related lockdowns, a supply chain shock, and an energy price shock exacerbated by the Russian invasion of Ukraine. By exploring the impact of these shocks under different assumptions about monetary policy efficacy and transmission channels, we review various explanations for the resurgence of inflation in the USA, including demand-pull, cost-push, and profit-driven factors. Our main results are fourfold: (i) without appropriate fiscal policy, the shocked economy can take years to recover, or even tip over into a deep recession; (ii) the success of monetary policy in curbing inflation is primarily due to expectation anchoring, rather than to the direct economic impact of interest rate hikes; (iii) however, perhaps paradoxically, strong inflation anchoring is detrimental to consumption and unemployment, leading to a narrow window of “optimal” policy responses due to the trade-off between inflation and unemployment; (iv) the two most sensitive model parameters are those describing wage and price indexation. The results of our study have implications for Central Bank decision-making, and offer an easy-to-use tool that may help anticipate the consequences of different monetary and fiscal policies.

REPUTATION AND TRUST OF PRINCIPALS AND AGENTS IN INTERORGANISATIONAL RELATIONS — THE CASE OF INTERDISCIPLINARY RESEARCH COOPERATION

Trust is the most important prerequisite for the emergence and effectiveness of interorganisational innovation cooperation, due to uncertainty and contract incompleteness. Nevertheless, little attention has been paid to the question of whether the trust of principals in a cooperation depends on different preconditions than that of agents. Based on a sample of 364 scientists from 100 German publicly funded interdisciplinary cooperations in medical research, we find that the trust of principals mainly depends on their partners’ reputation of ability, whereas the trust of agents on their partners’ reputation of benevolence. Surprisingly, culture- and personality-related factors prove to be ineffective. These results are of major relevance for the trust generation in innovation cooperation.

Trust in Human-Agent Teams: A Multilevel Perspective and Future Research Agenda

As technology continues to advance, interest in how humans perceive and interact with autonomous agents has increased, spurring ample research within human-agent teams (HATs). However, the word team is a bit of a misnomer, in that much of this research has examined one human in relation to one agent. We extend the HAT literature by applying a multilevel lens to develop future research questions regarding trust in heterogeneous HATs. First, we assert that trust is an attitude that is multireferent, suggesting research should explore not just humans’ trust in agents, but many other perspectives such as agents’ trust in humans. Second, we assert that trust is multilevel, suggesting research should explore higher-level emergent forms of trust. Third and fourth, we assert trust is dynamic and event-based, suggesting research should explore discontinuous changes in trust in response to events such as agent- and human-enacted trust violation and repair.

Pairing in-vehicle intelligent agents with different levels of automation: implications from driver attitudes, cognition, and behaviors in automated vehicles

ABSTRACT In-vehicle intelligent agents (IVIAs) have been developed to improve user experience in autonomous vehicles. Yet, the impact of the automation system on driver behavior and perception toward IVIAs is unclear. In this study, we conducted three experiments with 73 participants in a driving simulator to examine how automation system parameters (the level of automation system and IVIA features) influence driver attitudes, cognition, and behaviors when driving or riding in a simulated vehicle. We focused on subjective evaluations of driver-agent interaction and driver trust toward IVIAs to assess driver attitudes, driver situation awareness, and visual distraction to capture their cognition, and their driving performance to understand their behaviors. Our results show that the level of automation system affects drivers’ attitudes toward agent capabilities (e.g. perceived intelligence). Embodiment benefits are more pronounced with Level 5 systems, while speech style, in general, is more influential in determining affective aspects of user attitudes (e.g. Warmth, Likability). As the level of automation increases, drivers engage in more visual distractions. In addition, conversational speech style in general encouraged safer driving behaviors indicated by more stable lateral control under lower levels of automation. Our findings uncover the path of how system parameters affect driver behaviors through system evaluation and trust in agents. These findings have important implications for the development of cohesive user experiences in future transportation systems.

The Impacts of Ethical Sales Behaviour on Customer Loyalty in the Life Insurance Industry in Pakistan

The life insurance industry in Pakistan is highly competitive, with customer loyalty playing a crucial role in long-term success. Ethical sales behaviour emerges as a key differentiator, potentially fostering trust and driving positive customer experiences. This research investigates the relationship among ethical sales behaviour, customer trust in company and trust in sales agent, which directly affect customer loyalty in the life insurance industry. The quantitative survey targets customers of five major life insurance companies, ensuring a representative sample across diverse demographics. The questionnaires were sent to all companies customer on a random basis. The questionnaires are distributed with the help of office employee of the companies.The finding of the study shows that there is a direct and positive relation between sales agent ethical sales behaviour and customer loyalty. Furthermore, the trust of the customer in the sales agent and trust of the customer in a company has direct positive mutual relation and customer trust both affect customer loyalty. By demonstrating the tangible benefits of ethical sales behaviour, the study can encourage companies to implement and prioritize ethical practices. The identified key drivers of customer loyalty can guide training programs and performance evaluations for insurance agents, ultimately fostering a culture of ethical conduct within the industry.

Expediting financial inclusion in Tanzania using FinTech: the perspective of diffusion of innovation theory

PurposeThe purpose of this paper is examining how financial inclusion can be enhanced in rural and remote areas in emerging economies through the adoption of Financial technologies. The study examines the direct influence of technological characteristics, facilitating conditions, trust and perceived risk on adoption of FinTech. Furthermore, the study examines the mediating effects of facilitating conditions and moderating effects of education level in adoption of FinTech.Design/methodology/approachThe study employed quantitative research design. Stratified and simple random sampling were employed to identify areas in which respondents will be collected. Drop and Pick method was further employed to collect 275 valid and reliable responses. Structural equation modeling was used to analyze the data and assess the reliability and validity of the measurement model. Structural model analysis was employed to examine the hypothesized relationship.FindingsThe results show that compatibility, complexity agent trust, perceived risk and facilitating have significant influence on adoption of FinTech. Furthermore, the findings show that facilitating conditions mediate the relationship between all technological characteristics and FinTech adoption except the relationship between compatibility and FinTech. The study also has found that education level moderates the relationship between observability and FinTech adoption as well as trialability and FinTech adoption.Research limitations/implicationsThis study employed cross-sectional research-based techniques in collecting data for analyzing the hypothesized relationship. However, the drawback of cross-sectional may impact the findings of this study over a long time. Therefore, future studies must consider conducting longitudinal studies which may collect data over some time for analysis of the FinTech adoption. Furthermore, the generalization of these findings should be considered carefully by other countries which are not economically similar to Tanzania and developed countries. This also provides room for future studies to examine the same area in countries with different economic development.Originality/valueThis study contributes to knowledge into two folds: First there is limited knowledge on adoption of FinTech in emerging economies; the study addresses the knowledge gap by providing empirical findings which will help scholar. Furthermore, less studies have examined the moderating effects of facilitating condition in adoption of technology. This study addresses the knowledge gap by examining the moderating influence of moderating influence of facilitating conditions. Also, this study provides empirical findings which will help policymakers to implement strategy that will speed-up financial inclusion in rural and isolated areas in emerging economies.

The Effect of AI Agent Gender on Trust and Grounding

Artificial intelligence (AI) agents are widely used in the retail and distribution industry. The primary objective was to investigate whether the gender of AI agents influences trust and grounding. This paper examined the influence of AI agent gender and brand concepts on trust and grounding within virtual brand spaces. For this purpose, it used two independent variables: brand concept (functional vs. experiential) and AI agent gender (male vs. female). The dependent variables included AI agent trust and grounding. The study revealed that in virtual brand spaces centered around a functional concept, male AI agents generated higher levels of trust than female AI agents, whereas, when focused on an experiential concept, female AI agents induced higher levels of grounding than male AI agents. Furthermore, the findings indicate that the association between customers’ identification with AI agents and recommendations for actual brand purchases is mediated by trust and grounding. These findings support the idea that users who strongly identify with AI agents are more inclined to recommend brand products. By presenting alternatives that foster the establishment and sustenance of a meaningful, sustainable relationship between humans and AI, this study contributes to research on human–computer interactions.

Trust dynamics in human interaction with an industrial robot

ABSTRACT Trust is important for collaboration. In hybrid teams of humans and robots, trust enables smooth collaboration and reduces risks. Just as collaboration between humans and robots differs from interpersonal collaboration, so does the nature of trust in human-robot interaction (HRI). Therefore, further investigations on trust formation and dissolution in HRI, factors affecting it, and means for keeping trust on an appropriate level are needed. However, our knowledge of interpersonal trust and trust in autonomous agents cannot be transferred directly to HRI. In this paper, we present a study with 32 participants on trust formation and dissolution as well as forecasting to influence trust in an industry robot. Results show differences in dynamics and factors of trust formation and dissolution. Additionally, we find that the effect of forecasting on trust depends on task success. These findings support the design of trustful human-robot interaction and corresponding robotic team members.

Blockchain and agency theory in supply chain management: A question of trust

This paper aims to explore how does the introduction of blockchain technologies impact trust in existing supply chain relationships. To do so, we conducted a multiple case study through the lens of agency theory. Three Swiss companies which are carrying out blockchain projects for their supply chain have been analyzed in semi-structured interviews and document analysis. Two significant findings of the impact on principal-agent relationships are stressed. Firstly, blockchain technologies can reduce the asymmetry of information, increase the control of agents, and consequently minimize the trust in the principal towards the agents. Secondly and paradoxically, the control mechanisms inherent in these technologies can decrease the agent's trust in the principal. Thus, questioning trust is necessary, both as an antecedent and an unexpected consequence when adopting blockchain technologies. Finally, we provide guidelines for initiating the adoption of blockchain technologies in supply chains.

Integrity-based Explanations for Fostering Appropriate Trust in AI Agents

Appropriate trust is an important component of the interaction between people and AI systems, in that “inappropriate” trust can cause disuse, misuse, or abuse of AI. To foster appropriate trust in AI, we need to understand how AI systems can elicit appropriate levels of trust from their users. Out of the aspects that influence trust, this article focuses on the effect of showing integrity. In particular, this article presents a study of how different integrity-based explanations made by an AI agent affect the appropriateness of trust of a human in that agent. To explore this, (1) we provide a formal definition to measure appropriate trust, (2) present a between-subject user study with 160 participants who collaborated with an AI agent in such a task. In the study, the AI agent assisted its human partner in estimating calories on a food plate by expressing its integrity through explanations focusing on either honesty, transparency, or fairness. Our results show that (a) an agent who displays its integrity by being explicit about potential biases in data or algorithms achieved appropriate trust more often compared to being honest about capability or transparent about the decision-making process, and (b) subjective trust builds up and recovers better with honesty-like integrity explanations. Our results contribute to the design of agent-based AI systems that guide humans to appropriately trust them, a formal method to measure appropriate trust, and how to support humans in calibrating their trust in AI.

“My Name is Alexa. What’s Your Name?” The Impact of Reciprocal Self-Disclosure on Post-Interaction Trust in Conversational Agents

The use of conversational AI agents (CAs), such as Alexa and Siri, has steadily increased over the past several years. However, the functionality of these agents relies on the personal data obtained from their users. While evidence suggests that user disclosure can be increased through reciprocal self-disclosure (i.e., a process in which a CA discloses information about itself with the expectation that the user would reciprocate by disclosing similar information about themself), it is not clear whether and through which mechanism the process of reciprocal self-disclosure influences users’ post-interaction trust. We theorize that anthropomorphism (i.e., the extent to which a user attributes humanlike attributes to a nonhuman entity) serves as an inductive inference mechanism for understanding reciprocal self-disclosure, enabling users to build conceptually distinct cognitive and affective foundations upon which to form their post-interaction trust. We found strong support for our theory through two randomized experiments that used custom-developed text-based and voicebased CAs. Specifically, we found that reciprocal self-disclosure increases anthropomorphism and anthropomorphism increases cognition-based trustworthiness and affect-based trustworthiness. Our results show that reciprocal self-disclosure has an indirect effect on cognition-based trustworthiness and affect-based trustworthiness and is fully mediated by anthropomorphism. These findings conceptually bridge prior research on motivations of anthropomorphism and research on cognitive and affective bases of trust.

Implications of trust, preparedness, risk perceptions, and local context on deprivation costs and disaster relief planning

Deprivation Costs Functions (DCFs) are key to designing effective relief distribution operations after disasters. In this paper, data are collected in Colombia and Ecuador to estimate DCFs for water and food, considering the influence of individuals' attitudes on preparedness, risk perception, and trust in response agents. Hybrid Choice Models are used to analyze and compare the estimated DCFs. The analyses confirm that DCFs differ by commodity. Additionally, socioeconomic characteristics of individuals influence their willingness to pay for critical supplies to reduce their own suffering. Preparedness, risk perceptions and trust in response agents impact individuals' behavior in disaster situations. The results also show that community and religious groups are considered the most trusted response agents in both countries. As a result, their involvement in official relief efforts should be more articulated. Colombia and Ecuador show significant differences in their estimated DCFs, confirming that deprivation costs are context-specific by nature. As such, DCFs should not be directly transferred among disaster locations. The findings from this study will support decision-makers in designing effective preparedness and response plans that are based on trust relationships that serve as foundations for community resilience.

Trust in Robot–Robot Scaffolding

The study of robot trust in humans and other agents is not explored widely despite its importance for the near future human-robot symbiotic societies. Here we propose that robots should trust partners that tend to reduce their computational load, which is analogous to human cognitive load. We test this idea by adopting an interactive visual recalling task. In the first set of experiments, the robot can get help from online instructors with different guiding strategies to decide which one it should trust based on the computational load it experiences during the experiments. The second set of experiments involves robot-robot interactions. Akin to the robot-online instructor case, the Pepper robot is asked to scaffold the learning of a less capable ‘infant’ robot (Nao) with or without being equipped with the cognitive abilities of theory of mind and task experience memory to assess the contribution of these cognitive abilities to scaffolding performance. Overall, the results show that robot trust based on computational/cognitive load within a sequential decision-making framework leads to effective partner selection and robot-robot scaffolding. Thus, using the computational load incurred by the cognitive processing of a robot may serve as an internal signal for assessing the trustworthiness of interaction partners.

Predictors of users' adherence to a fully automated digital intervention to manage insomnia complaints.

Fully automated digital interventions show promise for disseminating evidence-based strategies to manage insomnia complaints. However, an important concept often overlooked concerns the extent to which users adopt the recommendations provided in these programs into their daily lives. Our objectives were evaluating users' adherence to the behavioral recommendations provided by an app, and exploring whether users' perceptions of the app had an impact on their adherence behavior. Case series study of individuals completing a fully automated insomnia management program, conducted by a virtual agent, during December 2020 to September 2022. Primary outcome was self-reported adherence to the behavioral recommendations provided. Perceptions of the app and of the virtual agent were measured with the Acceptability E-Scale and ECA-Trust Questionnaire. Insomnia was evaluated with the Insomnia Severity Index at baseline (phase 1), after 7days of sleep monitoring (phase 2) and post-intervention (phase 3). A total of 824 users were included, 62.7% female, mean age 51.85 (±12.55) years. Of them, 32.7% reported having followed at least one recommendation. Users' trust in the virtual agent and acceptance of the app were related to a pre-intervention effect in insomnia severity (phase 2). In turn, larger pre-intervention improvements predicted better adherence. Mediational analyses showed that higher levels of trust in the virtual agent and better acceptance of the app exerted statistically significant positive effects on adherence (β = 0.007, 95% CI, 0.001-0.017 and β = 0.003, 95% CI 0.0004-0.008, respectively). Users' adherence is motivated by positive perceptions of the app's features and pre-intervention improvements. Determinants of adherence should be assessed, and targeted, to increase the impact of fully automated digital interventions.

Trust in Artificial Intelligent Agent while Completing a Procedural Construction Task

The use of AI-enabled recommender systems in construction activities has the potential to improve worker performance and reduce errors; however, the accuracy of such systems in providing effective suggestions is dependent on the quality of their training data. A within-subjects experimental study was conducted using a simulated recommender system for installation tasks to investigate the effect of system reliability and construction task complexity on worker trust, workload, and performance. Results indicate that overall trust in the AI agent was higher for the highly reliable condition but remained consistent across various levels of task complexity. The workload was found to be higher for low reliability and complex conditions, and the effect of reliability on performance was influenced by task complexity. These findings offer insights for designing recommender systems to support construction workers in completing procedural tasks.