We compare the X-ray (1-20 keV) and hard X-ray (20-200 keV) luminosities of black hole binaries (BHBs; i.e., binaries for which the mass of the compact object is known to exceed 3 M☉) and X-ray bursters (neutron star binaries, NSBs). We discuss two ways of distinguishing a BHB from a NSB: (1) If the X-ray luminosity exceeds ~ 1037 ergs s–1, the hard X-ray luminosity of BHBs is relatively unaffected, whereas the hard X-ray luminosity of NSBs decreases drastically; and (2) the hard X-ray luminosity of BHBs is commonly in the range 1037 -6 × 1037 ergs s–1, whereas for NSBs it is 1037 ergs s–1. We show that late in their decays transient BHBs (e.g., GRS 1124-68) have X-ray and hard X-ray luminosities comparable to those observed for NSBs. Thus BHBs can be distinguished from NSBs only at relatively high luminosities.We also compare NSBs with the so-called black hole candidates (BHCs; i.e., systems with similar spectral/temporal properties to BHBs). The X-ray and hard X-ray luminosities of LMC X-1, GRO J0422 + 32, GRS 1915 + 105, 4U 1543-47, and 4U 1630-47 are much larger than the maximum luminosities observed from NSBs, which supports the idea that they contain black holes. Three other BHCs, namely GRS 1716-249, 1E 1740.7-2942, and GRS 1758-258 (which all lack an ultrasoft spectral component), have hard X-ray luminosities at least a factor of ~ 2-3 above the maximum observed from NSBs, which suggests that these objects also contain black hole primaries. The case of GX 339-4 remains very uncertain because of the large uncertainty in its distance estimates (from 1.3 to 4 kpc). Assuming the larger distance, the X-ray and hard X-ray luminosities of the source, and its luminosity-related spectral changes which are similar to transient BHBs (e.g., GRS 1124-68), support the idea that it contains a black hole. Finally, the X-ray and hard X-ray luminosities of the puzzling X-ray source 4U 1957 + 11 are in the range of those observed for NSBs, consistent with the idea that it might contain a neutron star. If 4U 1957 + 11 is, in fact, a neutron star system, this would establish that the combination of a power-law tail and an ultrasoft component (which is present in the spectrum of 4U 1957+ 11) is not a unique spectral signature of an accreting black hole.