Artificial intelligence and the situational rationality of diagnosis: Human problem‐solving and the artifacts of health and medicine

Abstract

AbstractWhat is the problem‐solving capacity of artificial intelligence (AI) for health and medicine? This paper draws out the cognitive sociological context of diagnostic problem‐solving for medical sociology regarding the limits of automation for decision‐based medical tasks. Specifically, it presents a practical way of evaluating the artificiality of symptoms and signs in medical encounters, with an emphasis on the visualization of the problem‐solving process in doctor‐patient relationships. In doing so, the paper details the logical differences underlying diagnostic task performance between man and machine problem‐solving: its principle of rationality, the priorities of its means of adaptation to abstraction, and the effects of seeking optimization in the problem‐solving process. Using these parameters as a heuristic for evaluating the capacity of AI to address issues of diagnostic error through design, the paper presents a conceptual review of the discipline of AI in medicine. Studies relying on procedural rationality describe models that treat diagnosis as a “natural artifact” by employing symbolic methods designed to simulate human problem‐solving. Research adhering to probabilistic rationality describes models that treat diagnosis as a “natural artifact” of an ecological image by utilizing sub‐symbolic methods designed to simulate neural networks. Research guided by situational rationality describes models that require treating diagnosis as a “socio‐cognitive artifact,” the artificiality of which is organized in discourses of patient‐centered decision‐making. The paper concludes with a commentary on the ethical application of AI in health and medicine, given the logical differences underlying diagnostic task performance.