Spatio–temporal analysis of large strain plastic flow at or near interfaces and free surfaces is important for understanding practical problems in the cutting and sliding of metals. In this context, the use of direct in situ imaging, coupled with digital image correlation (DIC), has gained popularity in the past couple of decades since it does not require a priori assumptions about the nature of the deformation field. Moreover, the application of DIC to dynamically evolving interfaces remains challenging. Common techniques such as hierarchical grid refinement or post analysis interpolation are either spatially restrictive or can lead to significant data loss. In this work, we present an alternative experimental method -termed ensemble averaged DIC- that circumvents both these limitations by resorting to ensemble averaging of deformation fields over a number of related, yet independent, unstructured grids. The resulting fields are accurate to second order and are benchmarked against standard 1D and 2D test cases, before being applied to two plastic flow fields arising in deformation processing—frictional sliding and orthogonal machining. We benchmark our scheme against commercially available packages to demonstrate its enhanced ability to resolve plastic flow near interface and free surface. The scheme is shown to accurately estimate residual surface strains on the cut/processed material surface without any a priori information about the flow field.