The purpose of this study is to clarify the vertical prokaryotic distribution in groundwater in a terrestrial subsurface sedimentary environment with a geological complex. Six groundwater samples were collected from a coastal 1200-m-deep borehole in which digging strata deposited between 2.3 and 1.5 Ma in Horonobe, Hokkaido, Japan. The studied succession was divided into three vertical zones that were geochemically differentiated according to their chloride contents and water-stable isotopes. The upper zone (UZ; shallower than 500 m) primarily contained fresh water supplied by penetrating meteoric water, the connate water zone (CWZ; deeper than 790 m) contained paleo-seawater, and the diffusion zone (DZ; 500–790 m depth) located between UZ and CWZ. Fluctuations in the prokaryotic density and constituents were observed across these three zones. The prokaryotic density decreased from UZ toward DZ, and the density of DZ was two orders of magnitude lower than that of UZ and CWZ. High prokaryotic activity was observed in CWZ below DZ. The upward expansion of prokaryotic distribution from CWZ, where high prokaryotic potential expressed by biomass can be maintained almost equivalent to that in the marine environment, probably occurred on a geological timescale from 80 ka to 1.3 Ma, as shown by the groundwater age of DZ. The DZ is a zone where the geochemistry has changed drastically owing to the mixing of penetrating meteoric water and the diffusion of deep paleo-seawater, preserving a unique subsurface environment. This chemically mixed zone might be considered as a buffering zone for prokaryotes to prevent the expansion of prokaryotic density and activity provided by diffusion and their in situ growth from both above and below the zones, which is expected to be maintained over a geological timescale. This zone is considered important for using subsurface space in the deep subsurface environment of the island arc.