We study the Selmer group and the fine Selmer group of $p$-adic Galois representations defined over a non-commutative $p$-adic Lie extension and their Hida deformations. For the fine Selmer group, we generalize the pseudonullity conjecture of J. Coates and R. Sujatha, Fine Selmer group of elliptic curves over $p$-adic Lie extensions, in this context and discuss its invariance in a branch of a Hida family. We relate the structure of the ‘big’ Selmer (resp. fine Selmer) group with the specialized individual Selmer (resp. fine Selmer) groups.