By performing 1-norm operation for phase-shifting interferograms, a rapid phase shift extraction and phase retrieval algorithm is proposed in random phase-shifting interferometry (PSI). First, by performing the 1-norm operation for a sequence of phase-shifting interferograms, we can construct a group of new interference patterns, possessing less data information but the same phase shifts relative to the original interferograms. Second, based on the least-square iteration (LSI) algorithm, accurate phase shifts can be extracted from these construct interference patterns. Finally, the measured phase can be achieved from the above extracted phase-shifts using the LSI algorithm. The obtained results show that as long as the pixel number of the constructed interference pattern through performing 1-norm operation is about 0.4% pixel number of the original interferogram, the proposed algorithm can achieve accurate phase shifts and be measured. Compared with other iteration algorithms of PSI, in addition to maintaining the advantage of high accuracy, the proposed algorithm reveals outstanding performance in processing speed.