Let $X$ be an irreducible smooth projective curve of genus $g\ge3$ defined over the complex numbers and let ${\mathcal M}_\xi$ denote the moduli space of stable vector bundles on $X$ of rank $n$ and determinant $\xi$, where $\xi$ is a fixed line bundle of degree $d$. If $n$ and $d$ have a common divisor, there is no universal vector bundle on $X\times {\mathcal M}_\xi$. We prove that there is a projective bundle on $X\times {\mathcal M}_\xi$ with the property that its restriction to $X\times\{E\}$ is isomorphic to $P(E)$ for all $E\in\mathcal{M}_\xi$ and that this bundle (called the projective Poincare bundle) is stable with respect to any polarization; moreover its restriction to $\{x\}\times\mathcal{M}_\xi$ is also stable for any $x\in X$. We prove also stability results for bundles induced from the projective Poincare bundle by homomorphisms $\text{PGL}(n)\to H$ for any reductive $H$. We show further that there is a projective Picard bundle on a certain open subset $\mathcal{M}'$ of $\mathcal{M}_\xi$ for any $d>n(g-1)$ and that this bundle is also stable. We obtain new results on the stability of the Picard bundle even when $n$ and $d$ are coprime.