From a clinical point of view, how to force a transition from insomnia brain state to healthy brain state by external driven stimulation is of great interest. This needs to define brain state of insomnia disorder as metastable substates. The current study was to identify recurrent substates of insomnia disorder in terms of probability of occurrence, lifetime, and alternation profiles by using leading eigenvector dynamics analysis (LEiDA) method. We enrolled 32 patients with insomnia disorder and 30 healthy subjects. We firstly obtained the BOLD phase coherence matrix from Hilbert transform of BOLD signals and then extracted all the leading eigenvectors from the BOLD phase coherence matrix for all subjects across all time points. Lastly, we clustered the leading eigenvectors using a k-means clustering algorithm to find the probabilistic metastable substates (PMS) and calculate the probability of occurrence and associated lifetime for substates. The resulting 3 clusters were optimal for brain state of insomnia disorder and healthy brain state, respectively. The occurred probabilities of the PMS were significantly different between the patients with insomnia disorder and healthy subjects, with 0.51 versus 0.44 for PMS-1 (p < 0.001), 0.25 versus 0.27 for PMS-2 (p = 0.051), and 0.24 versus 0.29 for PMS-3 (p < 0.001), as well as the lifetime (in TR) of 36.65 versus 33.15 for PMS-1 (p = 0.068), 14.36 versus 15.43 for PMS-2 (p = 0.117), and 14.80 versus 16.34 for PMS-3 (p = 0.042). The values of the diagonal of the transition matrix were much higher than the probabilities of switching states, indicating the metastable nature of substates. The resulted probabilistic metastable substates hint the characteristic brain dynamics of insomnia disorder. The results may lay a foundation to help determine how to force a transition from insomnia brain state to healthy brain state by external driven stimulation.