The presence of horizontal pleiotropy in Mendelian randomization (MR) analysis has long been a concern due to its potential to induce substantial bias. In recent years, many robust MR methods have been proposed to address this by relaxing the ‘no horizontal pleiotropy’ assumption. Here, we propose a novel two-stage framework called CMR, which integrates a conditional analysis of multiple genetic variants to remove pleiotropy induced by linkage disequilibrium, followed by the application of robust MR methods to model the conditional genetic effect estimates. We demonstrate how the conditional analysis can reduce horizontal pleiotropy and improve the performance of existing MR methods. Extensive simulation studies covering a wide range of scenarios of horizontal pleiotropy showcased the superior performance of the proposed CMR framework over the standard MR framework in which marginal genetic effects are modeled. Moreover, the application of CMR in a negative control outcome analysis and investigation into the causal role of body mass index across various diseases highlighted its potential to deliver more reliable results in real-world applications.