Patterns of transport and residence time influence the morphology, ecology and biogeochemistry of shallow coastal bay systems in important ways. To better understand the factors controlling residence time and exchange in coastal bays, a three-dimensional finite-volume coastal ocean model was set up and validated with field observations of circulation in a system of 14 shallow coastal bays on the Atlantic coast of the USA (Virginia Coast Reserve). Residence times of neutrally buoyant particles as well as exchange among the bays in the system and between the bays and the ocean were examined with Lagrangian particle tracking. There was orders of magnitude variation in the calculated residence time within most of the bays, ranging from hours in the tidally refreshed (repletion) water near the inlets to days–weeks in the remaining (residual) water away from the inlets. Residence time in the repletion waters was most sensitive to the tidal phase (low vs. high) when particles were released whereas residence time in the residual waters was more sensitive to wind forcing. Wind forcing was found to act as a diffuser that shortens particle residence within the bays; its effect was higher away from the inlets and in relatively confined bays. Median residence time in the bays significantly decreased with an increase in the ratio between open water area and total area (open water plus marsh). Exchange among the bays and capture areas of inlets (i.e., exchange between the bays and the ocean) varied considerably but were insensitive to tidal phase of release, wind, and forcing conditions in different years, in contrast to the sensitivity of residence time to these factors. We defined a new quantity, termed shortest-path residence time, calculated as distance from the closest inlet divided by root-mean-square velocity at each point in model domain. A relationship between shortest-path residence time and particle-tracking residence time provides a means of estimating residence time over an entire model domain.