We study the influence of system-bath correlations in the initial state on the dynamics of biased two-state systems. The formal solution for the equilibrium correlation function in the form of a power series in the number of transitions between the system states is determined. For Ohmic damping and a special value of the coupling strength (K=1/2), the exact solution for all times and temperatures is presented. We find that at low temperatures the long-time tails of the decay into equilibrium are strongly affected by the initial preparation. Correspondingly, the low-frequency behavior of the neutron-scattering function is qualitatively changed by the presence or absence of correlations in the initial state.