Abstract
ObjectiveWe explore users’ and observers’ subjective assessments of human and automation capabilities and human causal responsibility for outcomes.BackgroundIn intelligent systems and advanced automation, human responsibility for outcomes becomes equivocal, as do subjective perceptions of responsibility. In particular, actors who actively work with a system may perceive responsibility differently from observers.MethodIn a laboratory experiment with pairs of participants, one participant (the “actor”) performed a decision task, aided by an automated system, and the other (the “observer”) passively observed the actor. We compared the perceptions of responsibility between the two roles when interacting with two systems with different capabilities.ResultsActors’ behavior matched the theoretical predictions, and actors and observers assessed the system and human capabilities and the comparative human responsibility similarly. However, actors tended to relate adverse outcomes more to system characteristics than to their own limitations, whereas the observers insufficiently considered system capabilities when evaluating the actors’ comparative responsibility.ConclusionWhen intelligent systems greatly exceed human capabilities, users may correctly feel they contribute little to system performance. They may interfere more than necessary, impairing the overall performance. Outside observers, such as managers, may overweigh users’ contribution to outcomes, holding users responsible for adverse outcomes when they rightly trusted the system.ApplicationPresenting users of intelligent systems and others with performance measures and the comparative human responsibility may help them calibrate subjective assessments of performance, reducing users’ and outside observers’ biases and attribution errors.
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.