Recent studies have shown that the subgrid-scale (SGS) mixture fraction has different structures and probability distributions for different SGS scalar variances. We study the effects of these structures on the scalar diffusion and temperature diffusion in the context of large-eddy simulation (LES) of turbulent combustion. Line images obtained in turbulent partially premixed (Sandia) flames are used to analyze the scalar and temperature diffusion filtered conditionally on the scalar and temperature, which must be correctly modeled in LES. The results show that for small SGS variance the scalar and temperature diffusion have a relatively simple structure. For large SGS variance the diffusion is much more complex, with the flamelet structure and local extinction playing important roles. The results show that it is important that mixing models for filtered density function methods be able to account for the different SGS mixture fraction and temperature structures for small and large SGS variance.