Current studies show that binary fringe patterns can be optimized in the phase or intensity domain. The problem with the former is the poor sensitivity to defocusing levels and the problem with the latter is that phase errors can not be restricted efficiently. In this paper, a binary fringe pattern optimization technology, combining intensity similarity and phase similarity, is proposed to improve fringe pattern structure similarity and defocusing robustness. Moreover, a binary patch optimization framework is also proposed to solve the time-consuming problem. Simulation and experiment results show that the proposed optimization can produce binary fringe patterns with high phase quality and defocusing robustness.