This paper presents a newly observed phenomenon of upscaling of rocks’ properties using big data nanoindentation and analytics involving Gaussian mixture modeling (GMM), leading to characterizing the cross-scale mechanical properties of four shales and one sandstone. A large number (i.e., ~ 500) of statistical indentation measurements to depths of 6–8 μm were performed on each rock, resulting in continuous depth-dependent hardness and Young’s modulus data from unknown phases, which were then segmented at various depths to extract an array of discretized subdatasets. Two-dimensional GMM of each subdataset yields the number, fraction, and properties of mechanically distinct phases, and re-assembly of these results leads to clearly discernible property-depth curves. Such improved data analytics consisting of data segmentation, GMM deconvolution, and re-assembly enables the transformation of a massive number of chaotic curves from unknown phases into a few discernible lines corresponding to identified phases, from which the mechanical properties of individual phases are accurately determined at relatively small depths. With increasing depth, initially unique mechanical properties of individual phases undergo multistage merging at the intermediate mesoscale and progressively homogenize into a unified value at large depths or macroscale (e.g., > ~ 5 μm), which is regarded as the bulk rock’s properties. More importantly, such depth-dependent transition and progressive merging and homogenization actually manifest the micromechanics of nanoindentation on a heterogeneous composite, including the indentation surround effect and rock’s microstructure (e.g., sizes and spacings of different solid particles and their properties). Compared to different micromechanical upscaling models, this newly developed big data indentation technique and pertinent data analytics enable more accurate, multi-parameter, and cross-scale characterization of highly heterogeneous materials and explicitly uncover the multi-staged, progressively-homogenized, depth-dependent upscaling of elasticity from individual constituents at the nanoscale to merged virtual interface phases at the mesoscale and to bulk material at the macroscale.