In recent years, 3D model analysis has made a revolutionary development. Point cloud contains rich 3D object geometry information, which is an important 3D object data format widely used in many applications. However, the irregularity and disorder of the point cloud also cause its vulnerability to environmental impact, which may bring security risks to safety-critical 3D applications such as self-driving tasks. Recently, there are only a few methods engaged to attack the point cloud models to improve the robustness of point cloud analysis models. Most of them only focus on the attack by adjusting the points but ignore learning the perturbation’s distribution characteristics. In this work, we propose a novel framework to attack point cloud models. By introducing the GAN structure, we train a generator to produce slight point-to-point perturbations according to the sample’s raw classification, which can effectively boost the attack performance. Meanwhile, we propose an outlier removal module to constrain the magnitude of the generated perturbation. The goal is to guarantee the visual quality of generation samples to improve the difficulty of training and further improve the robustness of 3D analysis models. Finally, we carry out extensive attack experiments, and the related results demonstrate the effectiveness of our proposed method.