Price forecasting in the financial market is one of the most important and challenging tasks in the field of time series forecasting since it is noisy, non-linear and non-stationary. In this paper, we first develop a kernel-free support vector regression model which not only has a strong flexibility to capture the nonlinear structure of the data but also maintains the high efficiency to avoid choosing a suitable kernel and its related parameters. Then a novel hybrid method is proposed combining empirical mode decomposition algorithm, quadratic surface support vector regression and autoregressive integrated moving average method for the stock indices and future price forecasting. This ensemble scheme fully takes the advantages of these individual methods to efficiently produce accurate time series forecasts. Finally, to compare our proposed method with other benchmark forecasting methods, three stock indices and three future prices are selected as the forecasting targets. The numerical results and statistical test strongly demonstrate the promising performance of our proposed hybrid method in terms of forecasting accuracy, efficiency and robustness.