Battery impedance spectroscopy models are given by fractional-order (FO) differential equations. In the discrete-time domain, they give rise to state-space models where the latent process is not Markovian. Parameter estimation for these models is, therefore, challenging, especially for noncommensurate FO models. In this paper, we propose a Bayesian approach to identify the parameters of generic FO systems. The computational challenge is tackled with particle Markov chain Monte Carlo methods, with an implementation specifically designed for the non-Markovian setting. Two examples are provided. In a first example, the approach is applied to identify a battery commensurate FO model with a single constant phase element (CPE) by using real data. We compare the proposed approach to an instrumental variable method. Then, we consider a noncommensurate FO model with more than one CPE and synthetic data sets, investigating how the proposed method enables the study of various effects on parameter identification, such as the data length, the magnitude of the input signal, the choice of prior, and the measurement noise.