Attribute binding enhancement improvement for diffusion model based zero-shot image segmentation

Abstract

As diffusion model's potential ability to accomplish perception tasks being discovered, many researchers have tried to apply diffusion model in segmentation tasks and achieved good results. However, there are not many methods to optimize the diffusion model for segmentation tasks by improving the text side. Even if some research have pointed out that diffusion model sometimes 'misunderstand' prompt and bind attributes to wrong objects. With the development of artificial intelligence and the gradual entry of large models into people's daily lives, in addition to the performance of the model, the model's interactivity and ability to understand natural language are also more important. Based on existing zero-shot diffusion model based segmentation method, this work introduces a new method to enhance attribute binding in the embedding of prompt improve the performance of the model. Through this method, more descriptive text will get better segmentation results, which to some extent, improves the segmentation performance of the model for inputs that are more in line with natural language description habits.