Microgels are commonly applied as solute carriers, where the size, density, and functionality of the microgels depend on solute binding. As representatives for ionic solutes with high affinity for the microgel, we study here the effect of superchaotropic Keggin polyoxometalates (POMs) PW12O403- (PW) and SiW12O404- (SiW) on the aqueous swelling and internal structure of nonionic poly(N-isopropylacrylamide) (pNiPAM) microgels by light scattering techniques and small-angle X-ray scattering. Due to their weak hydration, these POMs bind spontaneously to the microgels at millimolar concentrations. The microgels thus become charged and swell at low POM concentration, surprisingly without strongly increasing the volume phase transition temperature, and deswell at higher POM concentration. The swelling arises because of the osmotic pressure of dissociated counterions of the POMs, while the deswelling is due to POMs acting as physical cross-links in the microgels under screened electrostatics in NaCl or excess POM solution. This swelling/deswelling transition is sharper for PW than for SiW related to the lower charge density, weaker hydration, and stronger binding of PW. The POMs elicit qualitatively and quantitatively different swelling effects from ionic surfactants and classical salts. Moreover, the network softness and topology govern the swelling response upon POM binding. The softer the microgel, the stronger is the swelling response, while, inside the microgel, regions of high polymer density swell/contract more upon electric charging/cross-linking than regions with low polymer density. POM binding thus enables fine-tuning of microgel properties and highlights the role of network topology in microgel swelling. Because POMs decompose at an alkaline pH, these POM/microgel systems also exhibit pH-responsive swelling in addition to the typical temperature responsiveness of pNiPAM microgels.