De-Mystifying Data Testing and Applying Automation

Abstract

Being Agile has become a norm rather than a special need. To stay Agile in today’s world requires significant thought and innovative solutions. The testing industry has matured during the past decade with hundreds of open source tools and frameworks, specifically in the area of automation. QA teams have significantly benefitted by this evolution, enabling them to satisfy the demand of being Agile throughout the lifecycle and stay at par with technology advancements. One of the most important objectives of data-testing is to recommend the corrective measures the back-end integration teams need to introduce in the development life cycle (SDLC). Data validation definitely plays an important role and there are lots of techniques and tools available in the market. However, end-to-end automation penetration is comparatively low in back-end data testing and ETL test automation since the data transformation predominantly happens through ETL processes on major enterprise systems. There is a clear market and industry demand for automation in data testing. This space is gaining importance with the sole reason being quantity (size) to be handled along with the quality of data. This paper explains the essentials of data testing strategy how data quality and data validation checks play an important role; where and how to bring-in automation; and finally the method for arriving at faster, accurate root-cause analysis. It can be argued that data quality checks are implicitly covered as part of validation, however it is always recommended to address the problem at the source rather than at the destination. According to analyst findings in public domain, significant revenue wastages are reported due to poor data quality. The approach defined in this paper will benefit QA-testing teams involved in back-end data testing. It will improve their understanding and enable them to apply correct techniques as they move forward. Automation for data-testing is considered only for people with a technical background. A proper understanding of what exactly happens at back-end once data is processed from front-end, will enable a non-technical person to understand, enjoy, and appreciate the benefits of automation.