Indium phosphide (InP) crystals exhibit a propensity for brittle damage during machining, categorizing them as challenging materials due to their inherent brittleness and pronounced anisotropy. Atomic force microscopy (AFM) machining serves as a crucial approach for achieving low-damage removal of such brittle crystals. Molecular dynamics simulations were conducted to investigate the vibration-assisted AFM machining of InP crystals, systematically exploring the effects of scratching force, stress distribution, and subsurface damage. The findings revealed that plastic deformation of InP crystals during vibration-assisted AFM scratching was primarily governed by mechanisms including stress-induced amorphization transitions, phase transformations, dislocations, stacking faults, and lattice distortions. Compared to conventional AFM machining, vibration-assisted AFM machining had a smaller average contact area and maintained a periodic movement pattern of contact-separation. This induced the change of chip removal from piling up in front of the cutting tool to uniform flow to both sides. Employing appropriate vibration parameters in vibration-assisted AFM significantly diminished the scratching force, stress, dislocation length, and subsurface damage depth. These results not only elucidate the material removal and damage mechanisms of InP crystals at atomic and close-to-atomic scales, but also offer theoretical guidance for parameter optimization in the vibration-assisted AFM machining of other soft and brittle crystals.