Naturally occurring granular flows, such as landslides, debris flows and avalanches typically have size ratios of up to $$10^{6}$$ between the smallest and largest constituent particles. For the purposes of modelling, however, it is generally assumed that a single representative size can adequately describe the grains. Polydisperse flows are not described more completely primarily because of two reasons: The first is a lack of understanding of the physical mechanisms which affect polydisperse flows. The second is a lack of models with which to describe such systems. Here, we present a heterarchical multiscale model which accounts for both the microstructural evolution within representative elementary volumes, and also the associated changes in bulk flow properties. Three key mechanisms are addressed; segregation, comminution and mixing. Granular segregation is an important mechanism for industrial processes aiming at mixing grains. Additionally, it plays a pivotal role in determining the kinematics of geophysical flows. Because of segregation, the grainsize distribution in a granular medium varies in space and time during flow. Additional complications arise from the presence of comminution, where new particles are created, potentially enhancing segregation. This has a feedback on the comminution process, as particles change their local neighbourhood. Simultaneously, particles are generally undergoing remixing, further complicating the segregation and comminution processes. The interaction between these mechanisms is explored using a stochastic lattice model with three rules: one for each of segregation, comminution and mixing. The interplay between these rules creates complex patterns, as seen in segregating systems, and depth dependent log-normal grading curves, which have been observed in avalanche runout.