During hydraulic fracturing for shale gas development, water softening and creep deformation of shale rock are two common engineering problems that severely impair production performance. In order to understand the mineralogical and microstructural control on this mechanism from microscale, a brittle black shale from Lower Silurian Longmaxi Formation (named as LMX shale) and a clayey oil shale from Paleogene Youganwo Formation (YGW shale), which represent two members of shale family with different mineral constitutes and microstructures, were selected for hydration process under various durations (0.5 h–96 h) and temperatures (25 °C and 50 °C). The accompanied changes of microstructural and mechanical properties during hydration at microscale were investigated via nuclear magnetic resonance (NMR) and nanoindentation, respectively. Based on NMR T2 spectra, peak T2 time and water-infiltrated pore increment for both LMX and YGW shales rise sharply within the first 12 h, whereas remain invariable hereafter. And the values increase at higher temperature. For YGW shale, the maximum peak T2 time is lower but the incremental porosity is higher than LMX shale, respectively. Such difference indicates an occurrence of gradually enlarged and connected pores inside LMX shale after hydration, while a greater number of disconnected pores have been developed in YGW shale, comparatively. For both shales, great strength deterioration and creep enhancement are monitored within the first 12 h of hydration. The modulus reduction could be as high as 26.27% and 80.16% after 96 h soaking at 50 °C for LMX and YGW shales, respectively. Creep strain rate sensitivity increases from 0.026 to 0.234 for LMX shale, and 0.011 to 0.099 for YGW shale with intensified softening degree. The softening mechanism for LMX brittle shale is dominated by secondary pores and cavities generated as a result of dissolution of carbonate cementation and grain detachment after water infiltration, while microstructural integrity of YGW oil shale is degraded by clay swelling. Due to different mineral constitutes and microstructures, creep deformation for clayey YGW shale is mainly mediated by dislocation movement of fine clay grains, whereas diffusion creep plays the key role in coarse-grained LMX shale. This comparative study with two distinct shale samples provides the mineralogy- and microstructure-based framework for interpreting water softening process and creep mechanism of shale, which benefits hydraulic fracture design in different shale formations so as to enhance shale oil/gas production.