The influence of leadership on the adoption of agile practices and principles in software development teams

Abstract

With the growth of globalisation and significant developments in science and technology, software development today requires the capability to deal with fast-paced changes, increased competition and higher customer expectations. In light of this, many organisations have moved away from traditional approaches to software development, preferring instead to adopt lightweight, iterative approaches commonly termed ‘agile methodologies’.Despite the popularity of agile practices and principles, adoption rates and success in using them vary. Few academic studies exist examining how agile practices and principles are adopted in practice, or why there is such variation in their use. This study addresses this gap by drawing on transformational-transactional leadership theory to understand how agile practices and principles are adopted, and what factors influence this adoption. It also draws on existing research highlighting a distinction between cognitive and behavioural change and applies this to agile adoption.A qualitative approach with an interpretive perspective was taken for this research, adopting a case study research method. A single case, SoftwareCorps, was chosen with data collected over three and a half years, predominantly through participant observation and two phases of semi-structured interviews. Overall, 29 individuals participated in 47 interviews. This study was undertaken in two phases. In the first phase, the journey of agile adoption at SoftwareCorps was explored, and factors that influenced agile adoption attempts considered. In the second phase, the focus was narrowed to explore how leadership influenced the adoption of agile practices and principles. A model of leadership for agile adoption was subsequently developed, combining leadership theory and insights from the empirical data in this study.Findings from phase one demonstrated that effective and open communication, coaching, understanding the existing system of working, removal of barriers, sponsorship and buy-in, piloting with volunteers, selection of Scrum, and leadership were all influential factors in the adoption of agile at SoftwareCorps. However, leadership was found to be the most dominant theme. Additionally, data analysis in phase one revealed a distinction between behavioural and cognitive changes required for the adoption of agile practices.Findings from phase two revealed that a leader’s interpretation of agile and style of leadership shaped the type of change (i.e. behavioural or cognitive change) driven in software development teams at SoftwareCorps. A transformational leadership style, combined with an interpretation of agile as an approach to problem solving and a drive to create cognitive change, was found to be conducive to successful agile adoption. A transactional leadership style with an interpretation of agile as a process and a focus exclusively on behavioural change was inhibiting to agile adoption. The findings of this study suggest that successful agile adoption requires software development teams to be supported by individuals whose leadership style compliments agile values and principles (such as empowering, communicating and collaborating). Additionally, successful agile adoption is facilitated by leaders whose interpretation of agile is as an approach to problem-solving, rather than merely a set of procedures for developing software. This study contributes towards a greater understanding of factors influential to the adoption of agile practices and principles and provides a model to explain how leadership may influence agile adoption. In so doing, this study helps to explain why variation in agile adoption levels may occur.