Sequential infiltration synthesis (SIS), also known as vapor phase infiltration (VPI), is a quickly expanding technique that allows growth of inorganic materials within polymers from vapor phase precursors. With an increasing materials library, which encompasses numerous organometallic precursors and polymer chemistries, and an expanding application space, the importance of understanding the mechanisms that govern SIS growth is ever increasing. In this work, we studied the growth of polycrystalline ZnO clusters and particles in three representative polymers: poly(methyl methacrylate), SU-8, and polymethacrolein using vapor phase diethyl zinc and water. Utilizing two atomic resolution methods, high-resolution scanning transmission electron microscopy and synchrotron X-ray absorption spectroscopy, we probed the evolution of ZnO nanocrystals size and crystallinity level inside the polymers with advancing cycles─from early nucleation and growth after a single cycle, through the formation of nanometric particles within the films, and to the coalescence of the particles upon polymer removal and thermal treatment. Through in situ Fourier transform infrared spectroscopy and microgravimetry, we highlight the important role of water molecules throughout the process and the polymers' hygroscopic level that leads to the observed differences in growth patterns between the polymers, in terms of particle size, dispersity, and the evolution of crystalline order. These insights expand our understanding of crystalline materials growth within polymers and enable rational design of hybrid materials and polymer-templated inorganic nanostructures.