Promoting scientific software quality through transition to continuous integration and continuous delivery

Abstract

Software quality has been an issue for decades in many areas of scientific modelling for environment applications. Much of the software that has been developed is well-suited to supporting research investigation and application in one-off curated application environments, such as delivering solutions through community-based participatory approaches. However, when releasing the software into constrained production environments, with everyday operational challenges such as missing data, unintended user inputs, variable data quality, and values outside the bounds of those previously tested, lacks in software quality can become apparent. The lacks can result in lost time and effort to get the model running again, loss of trust in the model by users and, potentially, cessation in model use, and early and unexpected end-of-life. There have been many efforts through the years to encourage improved software quality, through approaches such as test-driven development, paired programming, more rigorous documentation, and better and broader user acceptance testing. The basis of many of these approaches is the art and science of software testing, through the whole development chain from unit testing, regression testing, integration testing, functional testing and to user testing. Continuous Integration/ Continuous Development (CI/CD)/ Continuous Deployment uses an automated or semi-automated process pipeline that promotes or progresses software through various stages of 'readiness' in different system realms, from development to testing to production. Faithful application of CI/CD promotes good quality software by requiring the software to pass a series of pre-determined tests before it is accepted into the next realm. Models must meet quality levels and pass automated tests to be able to be promoted. Additionally, governance gateways are used to check non-automated workflow components, such as manual testing. This paper explores the challenges and lessons learned in adoption and application of secure CI/CD in an operational environmental modelling enterprise, and suggests a minimum viable good practice approach for application to scientific environmental modelling.