Pharmacovigilance is vital for drug safety. The process typically involves two key steps: initial signal generation from spontaneous reporting systems (SRSs) and subsequent expert review to assess the signals' (potential) causality and decide on the appropriate action. We propose a novel discovery and verification approach to pharmacovigilance based on electronic healthcare data. We enhance the signal detection phase by introducing an ensemble of methods which generated signals are combined using Borda count ranking; a method designed to emphasize consensus. Ensemble methods tend to perform better when data is noisy and leverage the strengths of individual classifiers, while trying to mitigate some of their limitations. Additionally, we offer the committee of medical experts with the option to perform an in-depth investigation of selected signals through tailored pharmacoepidemiological studies to evaluate their plausibility or spuriousness. To illustrate our approach, we utilize data from the German Pharmacoepidemiological Research Database, focusing on drug reactions to the direct oral anticoagulant rivaroxaban. In this example, the ensemble method is built upon the Bayesian confidence propagation neural network, longitudinal Gamma Poisson shrinker, penalized regression and random forests. We also conduct a pharmacoepidemiological verification study in the form of a nested active comparator case-control study, involving patients diagnosed with atrial fibrillation who initiated anticoagulant treatment between 2011 and 2017. The case study reveals our ability to detect known adverse drug reactions and discover new signals. Importantly, the ensemble method is computationally efficient. Hasty false conclusions can be avoided by a verification study, which is, however, time-consuming to carry out. We provide an online tool for easy application: https://borda.bips.eu.