The smaller the scales on which complex fluids are studied, the more fluctuations become relevant, e.g. in microrheology and nanofluidics. In this paper, a general approach is presented for including fluctuations in conformation-tensor based models for viscoelasticity, in accordance with the fluctuation-dissipation theorem. It is advocated to do this not for the conformation tensor itself, but rather for its so-called contravariant decomposition, in order to circumvent two major numerical complications. These are potential violation of the positive semi-definiteness of the conformation tensor, and numerical instabilities that occur even in the absence of fluctuations. Using the general procedure, fluctuating versions are derived for the upper-convected Maxwell model, the FENE-P model, and the Giesekus model. Finally, it is shown that the fluctuating viscoelasticity proposed here naturally reduces to the fluctuating Newtonian fluid dynamics of Landau and Lifshitz [L. D. Landau and E. M. Lifshitz, Fluid Mechanics, Vol. 6 of Course of Theoretical Physics, Pergamon Press, Oxford, 1959], in the limit of vanishingly small relaxation time.