We study the variation of the dimension of the Bloch-Kato Selmer group of a p-adic Galois representation of a number field that varies in a refined family. We show that, if one restricts ourselves to representations that are, at every place dividing $p$, crystalline, non-critically refined, and with a fixed number of non-negative Hodge-Tate weights, then the dimension of the Selmer group varies essentially lower-semi-continuously. This allows to prove lower bounds for Selmer groups by continuity, in particular to prove some predictions of the conjecture of Bloch-Kato for modular forms.