The retrieval of a unique crystal potential from the scattering matrix S in high-energy transmission electron diffraction is discussed. It is shown that, in general, data taken at a single orientation are not sufficient to determine all the elements of S. Additional measurements with tilted incident beam are required for the determination of the whole S matrix. An algorithm for the extraction of the crystal potential from the S matrix measured at a single energy and thickness is presented. The limiting case of thin crystals is discussed. Several examples with simulated data are considered.