Software testing, a linchpin of quality assurance in software development, encompasses both manual and automation approaches. While manual testing ensures meticulous scrutiny, automation accelerates efficiency, reducing time and costs in dynamic landscape of software development. This research investigates the impact of critical success factors on productivity gain in software testing, with a focus on automation, within CMMI Level 5 companies. Drawing from a dataset comprising thirty real-time projects spanning banking, retail, and industrial applications, study explores dynamics of software development under agile and waterfall models. Interviews with developers and testers reveal insights into key aspects influencing productivity, including skills, experiences, and training. Emphasizing the need for strategic automation in projects subject to changes and maintenance, the research analyzes over thirty projects, scrutinizing variables such as manually designed test cases, execution time, and subsequent automation of test cases. Strategic selection of test cases for automation emerges as a cost-efficient practice. Results highlight a correlation between manual and automated test cases, indicating productivity gains. Research introduces a productivity gain metric, showcasing a break-even point where automation significantly reduces testing time. Overall, the findings offer a comprehensive understanding of software testing within CMMI Level 5, guiding organizations toward efficient automation practices and improved productivity.