We revisit mean-risk portfolio selection in a one-period financial market where risk is quantified by a positively homogeneous risk measure ρ. We first show that under mild assumptions, the set of optimal portfolios for a fixed return is nonempty and compact. However, unlike in classical mean-variance portfolio selection, it can happen that no efficient portfolios exist. We call this situation ρ-arbitrage, and prove that it cannot be excluded -- unless ρ is as conservative as the worst-case risk measure. After providing a primal characterisation of ρ-arbitrage, we focus our attention on coherent risk measures that admit a dual representation and give a necessary and sufficient dual characterisation of ρ-arbitrage. We show that the absence of ρ-arbitrage is intimately linked to the interplay between the set of equivalent martingale measures (EMMs) for the discounted risky assets and the set of absolutely continuous measures in the dual representation of ρ. A special case of our result shows that the market does not admit ρ-arbitrage for Expected Shortfall at level α if and only if there exists an EMM Q ≈ P such that ll dQ/dP ll∞ < 1/α.