Heterogeneity in the physical properties of tight sandstones is mainly caused by complex diagenetic interactions in various combinations during the burial process. The tight sandstone oil in the Linnan Sag of Bohai Bay Basin has great development potential but it is unclear how effective reservoirs were formed, and this is a significant reason why exploration in the deep reservoir is so difficult. Here we studied the diagenetic evolution of the tight sandstone of Lower Member 3 of the Shahejie Formation, Linnan Sag, Bohai Bay Basin, by using cores, thin sections, XRD, SEM, grain-size analysis, high-pressure mercury intrusion porosimetry, fluid inclusions, and carbon and oxygen stable isotopes. Compaction, replacement, cementation and dissolution diagenesis are identified; all four processes mainly occur in the A phase of mesodiagenesis. The interaction of formation burial, thermal evolution of organic matter and dehydration of gypsum leads to the alternating presence of three stages of alkaline fluids and two stages of acidic fluids. The average dissolution porosity of the dissolution sandstone facies is greater than the lower limit of the porosity for the effective reservoir in the study area, which is key to the formation of effective reservoirs. By determining differential diagenetic evolution sequences of tight sandstone reservoirs under the constraints of different lithologies where sandstone is interbedded with mudstone, we are able to elucidate the formation of the effective reservoir as follows: (1) In the eodiagenetic stage, a large amount of early carbonate cement developed in the sandstone adjacent to the mudstone, filling the primary pores and forming a cementation sandstone facies. Smaller amounts of early carbonate cement developed in the interior of the sandstone, which “fixed” the primary pores, and has a positive effect on the formation of a large number of secondary pores in the dissolution stage. (2) In the early stage of mesodiagenesis (A1), due to the filling of primary pores, it is difficult for acidic fluids to enter the cementation sandstone facies adjacent to mudstone, and the dissolution is weak. But dissolution within the sand body was strong, producing a large number of dissolution pores and forming a dissolution sandstone facies that clearly enhanced the physical properties. (3) In a second early stage of mesodiagenesis (A2), dissolution still played a dominant role in the dissolution sandstone facies, due to the development of dissolution pores in the previous stage (it is easy for acidic fluids to enter the dissolution sandstone facies). The physical properties of the dissolution sandstone facies were once again enhanced and an effective reservoir gradually began to form. (4) At the end of mesodiagenesis stages A1 and A2 (26.2–24.6 Ma; 7.5–2.8 Ma), the reservoir experienced two phases of petroleum charging. The charging of the cementation sandstone facies developed at the margins of the sandstones was difficult due to the filling of primary pores. The dissolution sandstone facies are oil-bearing, with higher porosity, which indicates an effective reservoir. By coupling the thermal evolution of organic matter and the dehydration of gypsum with differential diagenetic evolution and the porosity evolution of reservoir, we are able to establish a reservoir genetic model that involves alternating seepage of organic acids and alkaline brines.