We herein report the synthesis, characterization and anticancer activity of BTPIP (2-(4-(benzo[b]thiophen-2-yl)phenyl)-1H-imidazo[4,5-f][1,10]phenanthroline) and its four ruthenium(II) polypyridyl complexes [Ru(NN)2(BTPIP)](ClO4)2 (N-N = bpy = 2,2'-bipyridine, Ru(II)-1; phen = 1,10-phenanthroline, Ru(II)-2; dmb = 4,4'-dimethyl-2,2'-bipyridine, Ru(II)-3; dmp = 2,9-dimethyl-1,10-phenanthroline, Ru(II)-4). The DNA binding behaviors reveal that the complexes bind to calf thymus DNA by intercalation. Cytotoxicity of the complexes against A549, HepG-2, SGC-7901 and Hela cells were evaluated in vitro. Complexes Ru(II)-1, Ru(II)-2, Ru(II)-3, Ru(II)-4 show moderate activity on the cell proliferation in A549 cells with IC50 values of 9.3 ± 1.2, 12.1 ± 1.6, 10.3 ± 1.6, 8.9 ± 1.2 μM, respectively. Apoptosis assessment, intracellular mitochondrial membrane potential (MMP), location in mitochondria, reactive oxygen species (ROS), cell invasion assay and cell cycle arrest were also performed to explore the mechanism of this action. When the concentration of the ruthenium(II) complexes is increased, the amount of reactive oxygen species increases obviously and the mitochondrial membrane potential decreases dramatically in A549 cells. Most importantly, the ruthenium(II) polypyridyl complexes could arrive the cytoplasm through the cell membrane and accumulate in the mitochondria. These results showed that the ruthenium(II) complexes could induce apoptosis in A549 cells through an ROS-mediated mitochondrial dysfunction pathway.