High configurational entropies have been hypothesized to stabilize solid solutions in equiatomic, multi-element alloys which have attracted much attention recently as “high-entropy” alloys with potentially interesting properties. To evaluate the usefulness of configurational entropy as a predictor of single-phase (solid solution) stability, we prepared five new equiatomic, quinary alloys by replacing individual elements one at a time in a CoCrFeMnNi alloy that was previously shown to be single-phase [1]. An implicit assumption here is that, if any one element is replaced by another, while keeping the total number of elements constant, the configurational entropy of the alloy is unchanged; therefore, the new alloys should also be single-phase. Additionally, the substitute elements that we chose, Ti for Co, Mo or V for Cr, V for Fe, and Cu for Ni, had the same room temperature crystal structure and comparable size/electronegativity as the elements being replaced to maximize solid solubility consistent with the Hume–Rothery rules. For comparison, the base CoCrFeMnNi alloy was also prepared. After three-day anneals at elevated temperatures, multiple phases were observed in all but the base CoCrFeMnNi alloy, suggesting that, by itself, configurational entropy is generally not able to override the competing driving forces that also govern phase stability. Thermodynamic analyses were carried out for each of the constituent binaries in the investigated alloys (Co–Cr, Fe–Ni, Mo–Mn, etc.). Our experimental results combined with the thermodynamic analyses suggest that, in general, enthalpy and non-configurational entropy have greater influences on phase stability in equiatomic, multi-component alloys. Only when the alloy microstructure is a single-phase, approximately ideal solid solution does the contribution of configurational entropy to the total Gibbs free energy become dominant. Thus, high configurational entropy provides a way to rationalize, after the fact, why a solid solution forms (if it forms), but it is not a useful a priori predictor of which of the so-called high-entropy alloys will form thermodynamically stable single-phase solid solutions.