Accurate characterization of gas transport in shale is crucial for many industrial processes and scientific issues, such as CO2 sequestration in depleted shale and shale gas production. Gas transport in shale involves complex chemical, hydraulic, and mechanical effects and is generally characterized by permeability. Laboratory observations show some “abnormal” permeability evolution behavior which cannot be explained by classical shale permeability models. In this research, we develop a generic shale permeability model that couples multiple gas transport mechanisms and explains complex permeability evolution. Shale is idealized as an improved sugar-cube structure with inorganic flow channels and composite flow channels penetrating organic and inorganic zones. In organic zones, gas rarefaction effects, multilayer adsorption, multilayer-adsorption-induced swelling, stress sensitivity, and surface diffusion are considered. For inorganic zones, gas rarefaction effects and stress sensitivity control gas transport. Based on this conceptual geometry, the variation of flow channel size, effective stress, porosity, tortuosity, and permeability is coupled. The permeability model can be degraded into different forms that are suitable for constant confining pressure, constant effective stress, and constant average pore pressure conditions. The model is verified against experimental data involving different permeability evolution trends. The comparison among this model and several classical models further demonstrates its advantage and uniqueness. Simulation results show that shale permeability is determined by the competitive effects of gas rarefaction phenomenon, sorption-induced organic matter swelling, and effective stress variation. Before and after pore pressure reaches a switch pressure value, the dominant mechanisms for permeability evolution are different. We use the switch pressure to drawn two controlling factor diagrams that demonstrate the dominant mechanisms in different realms of permeability evolution during gas injection or depletion.