We discuss the results of cross-correlating catalogs of bright X-ray binaries with the Yale Southern Proper Motion catalog (version 4.0). Several objects already known to have large proper motions from Hipparcos are recovered. Two additional objects are found which show substantial proper motions, both of which are unusual in their X-ray properties. One is IGR J17544-2619, one of the supergiant fast X-ray transients. Assuming the quoted distances in the literature for this source of about 3 kpc are correct, this system has a peculiar velocity of about 275 km/sec -- greater than the velocity of a Keplerian orbit at its location of the Galaxy, and in line with the expectations formed from suggestions that the supergiant fast X-ray transients should be highly eccentric. We discuss the possibility that these objects may help explain the existence of short gamma-ray bursts outside the central regions of galaxies. The other is the source 2A~1822-371, which is a member of the small class of objects which are low mass X-ray binaries and long (i.e. $>$ 100 millisecond) X-ray pulsars. This system also shows both an anomalously high X-ray luminosity and a large orbital period derivative for a system with its orbital period, and some possible indications of an eccentric orbit. A coherent picture can be developed by adding in the proper motion information in which this system formed in the Perseus spiral arm of the Galaxy about 3 Myr ago, and retains a slightly eccentric orbit which leads to enhanced mass transfer.