Quantum interfaces (QIs) that generate entanglement between a multimode atomic memory and a photon forms a multiplexed repeater node and hold promise to greatly improve quantum repeater rates. Recently, the temporal multimode spin-wave memory that is entangled with a photon has been demonstrated with cold atoms. However, due to additional noise generated in multimode operation, the fidelity of spin-wave-photon entanglement significantly decreases with the mode number. So far, the improvement on temporal-multimode entanglement fidelity via suppressing the additional noise remains unexplored. Here, we propose and experimentally demonstrate a scheme that can suppress the additional noise of a temporally-multiplexed QI. The scheme uses an asymmetric channel to collect the photons coming and retrieving from the temporally-multiplexed QI. For making comparisons, we also set up a QI that uses symmetric channel for the photon collections. When the QIs store 14 modes, the measured Bell parameter S for the QIs using the asymmetric and the symmetric photon-collection channels are 2.36+/-0.03 and 2.24+/-0.04, respectively, showing that the QI using the asymmetric channel gives rise to a 3% increase in entanglement fidelity, i.e., a 1.7-fold decrease in the additional noise, compared with the QI using the symmetric one. On the other hand, the 14-mode entanglement QIs that use the asymmetric and symmetric collections preserve the violation of a Bell inequality for storage times up to 25 us and 20 us, respectively, showing that the asymmetric QI has a higher entanglement storage performance.