The ocean, one of the five major components of the Earth’s climate system, plays a key role in climate-forming processes, affecting its change and variability. The ocean influences climate over a wide range of time–space scales. To explore the climate, its components, interactions between them and, in particular, the effect of the ocean on weather and climate, researchers commonly use extremely complex mathematical models of the climate system that describe the atmospheric and ocean general circulations. However, this class of climate models requires enormous human and computing resources to simulate the climate system itself and to analyze the output results. For simple climate models, such as energy balance and similar models, the computational cost is insignificant, which is why these models represent a test tool to mimic a complex climate system and obtaining preliminary estimates of the influence of various internal and external factors on climate, its change and variability. The global mean surface temperature (GMST) and its fluctuations in time serve as critical indicators of changes in the climate system state. We apply a simple two-box ocean model to explore the effect of mixed and deep ocean layers on climate-forming processes and especially on climate change and variability. The effect of mixed and deep ocean layers on GMST is parameterized via the layers’ effective heat capacities and heat exchange between layers. For the listed parameters, the sensitivity functions were derived numerically and analytically, allowing one to obtain an idea of how the mixed and deep ocean layers affect climate change and variability. To study climate change, a deterministic version of the model was used with radiative forcing parameterized by both stepwise and linear functions. In climate variability experiments, a stochastic version of the model was applied in which the radiative forcing is considered as a delta-correlated random process.