Tell me a story: a framework for critically investigating AI language models

Abstract

ABSTRACT Large language models are rapidly being rolled out into high-stakes fields like healthcare, law, and education. However, understanding of their design considerations, operational logics, and implicit biases remains limited. How might these black boxes be understood and unpacked? In this article, we lay out an accessible but critical framework for inquiry, a pedagogical tool with four dimensions. Tell me your story investigates the design and values of the AI model. Tell me my story explores the model’s affective warmth and its psychological impacts. Tell me our story probes the model’s particular understanding of the world based on past statistics and pattern-matching. Tell me ‘their’ story compares the model’s knowledge on dominant (e.g. Western) versus ‘peripheral’ (e.g. Indigenous) cultures, events, and issues. Each mode includes sample prompts and key issues to raise. The framework aims to enhance the public’s critical thinking and technical literacy around generative AI models.