The effects of Ca2+ and Mg2+ on the fluorescence behavior of the phospholipid analogues 1-acyl-2-[6-[(7-nitro-2,1,3-benzoxadiazol-4-yl)amino]caproyl]phosphatidylcholin e and N-(7-nitro-2,1,3-benzoxadiazol-4-yl)phosphatidylethanolamine in small unilamellar vesicles consisting of phosphatidylserine, mixtures of phosphaitdylserine/phosphatidylcholine, and mixtures of phosphatidylserine/cholesterol were studied. Fluorescence quenching was observed when Ca2+, but not Mg2+, was added to phosphatidylserine vesicles containing 5 mol% fluorescent lipid. The quenching process, which could be monitored continuously, was virtually complete within 5-6 min at Ca2+ concentrations greater than or equal to 1.5 mM and resulted in a decrease of fluorescence intensity of approximately 60%. Fluorescence quenching did not occur in the presence of 0.5 mM Ca2+; however, simultaneous addition of 6 mM Mg2+ initiated a quenching process similar in rate and extent to that observed at higher concentrations of Ca2+ alone. This quenching of 4-nitro-2,1,3-benzoxadiazole (NBD) fluorescence is best explained in terms of Ca2+-induced separation of lipid phases that leads to an increase in local concentration of NBD-lipid in the bilayer and hence to self-quenching of NBD fluorescence. The kinetics of Ca2+-induced phase separation were also studied in several mixed lipid systems containing phosphatidylserine. In the case of mixtures of phosphatidylserine/cholesterol, the results indicate the presence of phase-separated regions as an intrinsic property of the vesicles in the absence of Ca2+. Finally, results are presented indicating that the kinetics of phase separation is slow compared to vesicle-vesicle fusion.