The main result of this paper is to extend from $\Q$ to each of the nine imaginary quadratic fields of class number one a result of Serre (1987) and Mestre-Oesterl\'e (1989), namely that if $E$ is an elliptic curve of prime conductor then either $E$ or a $2$-, $3$- or $5$-isogenous curve has prime discriminant. For four of the nine fields, the theorem holds with no change, while for the remaining five fields the discriminant of a curve with prime conductor is either prime or the square of a prime. The proof is conditional in two ways: first that the curves are modular, so are associated to suitable Bianchi newforms; and second that a certain level-lowering conjecture holds for Bianchi newforms. We also classify all elliptic curves of prime power conductor and non-trivial torsion over each of the nine fields: in the case of $2$-torsion, we find that such curves either have CM or with a small finite number of exceptions arise from a family analogous to the Setzer-Neumann family over $\Q$.