Competitive Benchmarking: An IS Research Approach to Address Wicked Problems with Big Data and Analytics

Abstract

Wicked problems like sustainable energy and financial market stability are societal challenges that arise from complex sociotechnical systems in which numerous social, economic, political, and technical factors interact. Understanding and mitigating these problems requires research methods that scale beyond the traditional areas of inquiry of information systems (IS) individuals, organizations, and markets and that deliver solutions in addition to insights. We describe an approach to address these challenges through competitive benchmarking (CB), a novel research method that helps interdisciplinary research communities tackle complex challenges of societal scale by using different types of data from a variety of sources such as usage data from customers, production patterns from producers, public policy and regulatory constraints, etc. for a given instantiation. Further, the CB platform generates data that can be used to improve operational strategies and judge the effectiveness of regulatory regimes and policies. We describe our experience applying CB to the sustainable energy challenge in the Power Trading Agent Competition (Power TAC) in which more than a dozen research groups from around the world jointly devise, benchmark, and improve IS-based solutions.