Rain gauge networks deliver crucial observations for many water-related applications but can be expensive to purchase and operate, which makes optimization of the number and locations of gauges an important task. Traditional optimization approaches often focus on kriging-based methods that are computationally expensive, which limits the scale of the optimization to small areas or a very limited number of gauges. This study presents a novel workflow with high computational efficiency that is able to handle optimization problems on large rain gauge networks. This is accomplished by developing a fast parametric emulator of the commonly used ordinary kriging approach. The results show that the developed emulator is able to accurately reproduce the original kriging uncertainty estimates with a computational speed up of a factor of 3000. In order to determine the best locations for new gauges, a greedy optimization heuristic that relies on sequential placement of gauges is developed. The sequential optimization can lead to sub-optimal solutions by itself, so a resubstitution mechanism is introduced to correct for this. The workflow is applied to the national gauging network of Denmark with 291 gauges, where it is able to optimally place 175 new gauges within 1 h of running time. A similar optimization with a traditional kriging approach would have taken approximately 125 days to complete highlighting the value the workflow.