We study nonreciprocal wave transmission across the interface of two dissimilar granular media separated by an elastic solid medium. Specifically, a left, larger-scale and a right smaller-scale granular media composed of two-dimensional, initially uncompressed hexagonally packed granules are interfacing with an intermediate linearly elastic solid, modeled either as a thin elastic plate or a linear Euler-Bernoulli beam. The granular media are modeled by discrete elements and the elastic solid by finite elements assuming a plane stress approximation for the thin plate. Accounting for the combined effects of Hertzian, frictional and rotational interactions in the granular media, as well as the highly discontinuous interfacial effects between the (discrete) granular media and the (continuous) intermediate elastic solid, the nonlinear acoustics of the integrated system is computationally studied subject to a half-sine shock excitation applied to a boundary granule of either the left or right granular medium. The highly discontinuous and nonlinear interaction forces coupling the granular media to the elastic solid are accurately computed through an algorithm with interrelated iteration and interpolation at successive adaptive time steps. Numerical convergence is ensured by monitoring the (linearized) eigenvalues of a nonlinear map of interface forces at each (variable) time step. Due to the strong nonlinearity and hierarchical asymmetry of the left and right granular media, time scale disparity occurs in the response of the interface which breaks acoustic reciprocity. Specifically, depending on the location and intensity of the applied shock, propagating wavefronts are excited in the granular media, which, in turn, excite either (slow) low-frequency vibrations or (fast) high-frequency acoustics in the intermediate elastic medium. This scale disparity is due to the size disparity of the left and right granular media, which yields drastically different wave speeds in the resulting propagating wavefronts. As a result, the continuum part of the interface responds with either low-frequency vibrations-when the shock is applied to the larger-scale granular medium, or high-frequency waves-when the shock is applied to the smaller-scale granular medium. This provides the fundamental mechanism for breaking reciprocity in the interface. The nonreciprocal interfacial acoustics studied here apply to a broad class of asymmetric hybrid (discrete-continuum) nonlinear systems and can inform predictive designs of highly effective granular shock protectors or granular acoustic diodes.