Continuous-time time series are widely used for modeling the realizations of those phenomena where it is theoretically possible to have observation at any point of the sampling domain. However, technical restrictions cause to see the realizations of such processes as discrete-time sample paths. We project time-domain observations into frequency-domain periodograms and employ the Whittle’s likelihood approximation to make the inference about the parameters of CARMA processes. The inference is given under the Bayesian paradigm and some scenarios about the prior distributions are discussed. We show that our proposed estimator is robust for a large family of Lévy-driven distributed noises under some mild conditions. The Bayes estimates of parameters are obtained using a fast MCMC algorithm and they are examined numerically for the non-informative priors. The mean squared errors of the presented estimates are also compared to two other likelihood-based estimators, numerically. We also examine the performance of this Bayes estimate in prediction procedure of exchange rate data as a real dataset.