The forecasting research literature has developed greatly in recent years as a result of advances in information technology. Financial time-series tasks have made substantial use of machine learning and deep neural networks, but building a prediction model from scratch takes time and computational resources. Transfer learning is growing popular in tackling these constraints of training time and computational resources in several disciplines. This study proposes a hybrid base model for the financial time series prediction employing the recurrent neural network (RNN) and long-short term memory (LSTM) called RNN-LSTM. We used random search to fine-tune the hyperparameters and compared our proposed model to the RNN and LSTM base models and evaluate using the RMSE, MAE, and MAPE metrics. When forecasting Forex currency pairs GBP/USD, USD/ZAR, and AUD/NZD our proposed base model for transfer learning outperforms RNN and LSTM base model with root mean squared errors of 0.007656, 0.165250, and 0.001730 respectively.