Let X be a nonsingular algebraic curve of genus g ⩾ 3 , and let M ξ denote the moduli space of stable vector bundles of rank n ⩾ 2 and degree d with fixed determinant ξ over X such that n and d are coprime. We assume that if g = 3 then n ⩾ 4 and if g = 4 then n ⩾ 3 , and suppose further that n 0 , d 0 are integers such that n 0 ⩾ 1 and nd 0 + n 0 d > nn 0 ( 2 g - 2 ) . Let E be a semistable vector bundle over X of rank n 0 and degree d 0 . The generalised Picard bundle W ξ ( E ) is by definition the vector bundle over M ξ defined by the direct image p M ξ * ( U ξ ⊗ p X * E ) where U ξ is a universal vector bundle over X × M ξ . We obtain an inversion formula allowing us to recover E from W ξ ( E ) and show that the space of infinitesimal deformations of W ξ ( E ) is isomorphic to H 1 ( X , End ( E ) ) . This construction gives a locally complete family of vector bundles over M ξ parametrised by the moduli space M ( n 0 , d 0 ) of stable bundles of rank n 0 and degree d 0 over X. If ( n 0 , d 0 ) = 1 and W ξ ( E ) is stable for all E ∈ M ( n 0 , d 0 ) , the construction determines an isomorphism from M ( n 0 , d 0 ) to a connected component M 0 of a moduli space of stable sheaves over M ξ . This applies in particular when n 0 = 1 , in which case M 0 is isomorphic to the Jacobian J of X as a polarised variety. The paper as a whole is a generalisation of results of Kempf and Mukai on Picard bundles over J, and is also related to a paper of Tyurin on the geometry of moduli of vector bundles.