Addressing the notion of trust around ChatGPT in the high-stakes use case of insurance

Abstract

The public discourse concerning the level of (dis)trust in ChatGPT and other applications based on large language models (LLMs) is loaded with generic, dread risk terms, while the heterogeneity of relevant theoretical concepts and empirical measurements of trust further impedes in-depth analysis. Thus, a more nuanced understanding of the factors driving the trust judgment call is essential to avoid unwarranted trust. In this commentary paper, we propose that addressing the notion of trust in consumer-facing LLM-based systems across the insurance industry can confer enhanced specificity to this debate. The concept and role of trust are germane to this particular setting due to the highly intangible nature of the product coupled with elevated levels of risk, complexity, and information asymmetry. Moreover, widespread use of LLMs in this sector is to be expected, given the vast array of text documents, particularly general policy conditions or claims protocols. Insurance as a practice is highly relevant to the welfare of citizens and has numerous spillover effects on wider public policy areas. We therefore argue that a domain-specific approach to good AI governance is essential to avoid negative externalities around financial inclusion. Indeed, as a constitutive element of trust, vulnerability is particularly challenging within this high-stakes set of transactions, with the adoption of LLMs adding to the socio-ethical risks. In light of this, our commentary provides a valuable baseline to support regulators and policymakers in unravelling the profound socioeconomic consequences that may arise from adopting consumer-facing LLMs in insurance.