Modeling non-Markovian time series is a recent topic of research in many fields such as climate modeling, biophysics, molecular dynamics, or finance. The generalized Langevin equation (GLE), given naturally by the Mori-Zwanzig projection formalism, is a frequently used model including memory effects. In applications, a specific form of the GLE is most often obtained on a data-driven basis. Here, Bayesian estimation has the advantage of providing both suitable model parameters and their credibility in a straightforward way. It can be implemented in the approximating case of white noise, which, far from thermodynamic equilibrium, is consistent with the fluctuation-dissipation theorem. However, the exploration of the posterior, which is done via Markov chain Monte Carlo sampling, is numerically expensive, which makes the analysis of large data sets unfeasible. In this work, we discuss an efficient implementation of Bayesian estimation of the GLE based on a piecewise constant approximation of the drift and diffusion functions of the model. In this case, the characteristics of the data are represented by only a few coefficients, so that the numerical cost of the procedure is significantly reduced and independent of the length of the data set. Further, we propose a modification of the memory term of the GLE, leading to an equivalent model with an emphasis on the impact of trends, which ensures that an estimate of the standard Langevin equation provides an effective initial guess for the GLE. We illustrate the capabilities of both the method and the model by an example from turbulence.