We report the discovery of four kpc-scale binary AGNs. These objects were originally selected from the Sloan Digital Sky Survey based on double-peaked [O III] 4959,5007 emission lines in their fiber spectra. The double peaks could result from pairing active supermassive black holes (SMBHs) in a galaxy merger, or could be due to bulk motions of narrow-line region gas around a single SMBH. Deep near-infrared (NIR) images and optical slit spectra obtained from the Magellan 6.5 m and the APO 3.5 m telescopes strongly support the binary SMBH scenario for the four objects. In each system, the NIR images reveal tidal features and double stellar components with a projected separation of several kpc, while optical slit spectra show two Seyfert 2 nuclei spatially coincident with the stellar components, with line-of-sight velocity offsets of a few hundred km/s. These objects were drawn from a sample of only 43 objects, demonstrating the efficiency of this technique to find kpc-scale binary AGNs.