A fully spectral numerical scheme is presented for the unsteady, high Reynolds number, incompressible Navier-Stokes equations, in domains which are infinite or semi-infinite in one dimension. The domain is not mapped, and standard Fourier or Chebyshev expansions can be used. The handling of the infinite domain does not introduce any significant overhead. The scheme assumes that the vorticity in the flow is essentially concentrated in a finite region, which is represented numerically by standard spectral collocation methods. To accommodate the slow exponential decay of the velocities at infinity, extra expansion functions are introduced, which are handled analytically. A detailed error analysis is presented, and two applications to direct numerical simulation of turbulent flows are discussed in relation with the numerical performance of the scheme.