Simultaneously investigating strain partitioning and the underlying deformation mechanisms for both the grain interior and the grain boundary (GB) is essential for understanding the complex plastic deformation of hexagonal close-packed metals. To this end, an automated analysis framework based on high-resolution digital image correlation (HRDIC) and electron backscatter diffraction (EBSD) data fusion and computer vision, integrating nanoscale resolution and a large field of view, is proposed. This framework consists of: (1) HRDIC-EBSD data fusion; (2) Segmenting the strain field into individual grains each with a core and a mantle; (3) Data clustering of the Matrix and slip bands (SBs) for each grain; (4) Full slip system (SS) identification and SS assignment to the SBs. The capabilities of this framework were demonstrated on Mg-10Y during compression. The strain field data, which was segmented into different clusters, including grain mantle, grain core, Matrix, and SBs, was analyzed statistically and quantitatively. The pixel-based slip activity, which considers the SB morphology, was obtained from a statistical perspective. Inter-granular accommodating mechanisms, including GB strain, slip transfer, and GB sliding, were quantitatively analyzed. Overall, this analysis framework, which can be applied to other materials, can automatically and statistically evaluate both nanoscale strain fields and underlying intra- and inter-granular deformation mechanisms grain-by-grain. This work provides valuable experimental insights into plastic deformation and accommodation mechanisms for polycrystals.