Sidechain-based Cross-chain exchange protocols enable payers to exchange cryptocurrencies among different blockchains via a sidechain. Many efforts, such as P2DEX (ACNS' 21), have been proposed to enhance cross-chain exchange privacy protection. However, existing sidechain-based cross-chain solutions for Monero on privacy concerns have limitations: requiring multiple pairs of parties paying simultaneously or fixed transaction amounts. This paper proposes ZeroCross, a novel privacy-preserving sidechain-based scheme that guarantees transaction unlinkability, exchanging fairness, and value confidentiality. ZeroCross designs: (i) a key exchange mechanism that guarantees exchanging fairness and (ii) a verification mechanism that utilizes CP-SNARK to ensure the transaction is confirmed without revealing the details of transactions. In addition, we discuss the influence of the remote side-channel attack in cross-chain exchange and the defence strategy. Finally, we prove the privacy and security of ZeroCross under the Universal Composability (UC) framework and evaluate the practical performance on computation and communication costs.