Here we theoretically study Gaussian beams with arbitrarily located polarization singularities (PSs). Under PSs, we mean here an isolated intensity null with radial, azimuthal, or radial-azimuthal polarization around it. An expression is obtained for the complex amplitude of such beams. We study in detail cases in which there is one off-axis PS, two opposite PSs, or more than two PSs located in the vertices of a regular polygon. If such a beam has one or two opposite PSs, these PSs are the centers of radial polarization. If there are three PSs, then one of them has radial polarization, and the other two have mixed radial-azimuthal polarization. If the beam has four PSs, then there are two PSs with radial polarization and two PSs with azimuthal polarization. When propagating in space, PSs are shown to appear in a discrete set of planes, in contrast to the phase singularities existing in any plane. If the beam has two PSs, their polarization is shown to transform from the radial in the initial plane to the azimuthal in the far field. The results can find application in optical communications by using non-uniform polarization.