We present a study on the double longitudinal-spin asymmetry of dihadron production in semi-inclusive deep inelastic scattering, in which the total transverse momentum of the final-state hadron pairs is integrated out. In particular, we investigate the origin of the $\cos\phi_R$ azimuthal asymmetry for which we take into account the coupling of the helicity distribution $g_1$ and the twist-3 dihadron fragmentation function $\widetilde{D}^{\sphericalangle}$. We calculate the $s$-wave and $p$-wave interference term $\widetilde{D}^{\sphericalangle}_{ot}$ in a spectator model. We estimate the $\cos\phi_R$ asymmetry at the kinematics of COMPASS which is collecting data on dihadron production in polarized deep inelastic scattering. The prediction of the same asymmetry at JLab 12GeV and a future EIC are also presented. Our study indicates that measuring the $\cos\phi_R$ asymmetry in SIDIS may be a ideal way to probe the dihadron fragmentation function $\widetilde{D}^{\sphericalangle}$.