Mirror symmetry is a type of infrared duality in 3D quantum field theory that relates the low-energy dynamics of two distinct ultraviolet descriptions. Though first discovered in the supersymmetric context, it has far-reaching implications for understanding nonperturbative physics in general 3D quantum field theories. We study mirror symmetry in 3D $\mathcal{N}=4$ supersymmetric field theories whose Higgs or Coulomb branches realize $D$- and $E$-type Kleinian singularities in the $ADE$ classification, generalizing previous work on the $A$-type case. Such theories include the $SU(2)$ gauge theory coupled to fundamental matter in the $D$-type case and non-Lagrangian generalizations thereof in the $E$-type case. In these cases, the mirror description is given by a quiver gauge theory of affine $D$- or $E$-type. We investigate the mirror map at the level of the recently identified 1D protected subsector described by topological quantum mechanics, which implements a deformation quantization of the corresponding $ADE$ singularity. We give an explicit dictionary between the monopole operators and their dual mesonic operators in the $D$-type case. Along the way, we extract various operator product expansion (OPE) coefficients for the quantized Higgs and Coulomb branches. We conclude by offering some perspectives on how the topological subsectors of the $E$-type quivers might shed light on their non-Lagrangian duals.