The Co-administration of multiple drugs can enhance the efficacy of disease treatment by reducing drug resistance and side effects. However, it also raises the risk of adverse drug interactions, presenting a challenging problem in healthcare. Various approaches have been developed to predict drug-drug interactions (DDIs) by leveraging both knowledge graphs and drug attribute information. While these methods have shown promise, they often fail to effectively capture correlations between biomedical information in the knowledge graph and drug properties. This work introduces a novel end-to-end DDI predictor framework based on generative adversarial networks. This framework utilizes a message-passing neural network to capture molecular structure information while employing the knowledge-aware graph attention network to capture the representation of drugs in the knowledge graph through considering the importance of different multihop neighbor nodes and relationships. The dual generative adversarial networks employ two generators and two discriminators in knowledge graph embedding and molecular topology embedding for adversarial training to capture the interrelations and complementary knowledge between molecular structure information and semantic information from the knowledge graph. comprehensive experiments have demonstrated that the proposed method outperforms state-of-the-art algorithms in binary classification, with improvements of 1.0% in accuracy, 0.45% in area under the receiver operating characteristic curve (AUC), 0.24% in area under the precision-recall curve (AUPR), and 0.98% in F1 score. Furthermore, for multiclass classification tasks, improvements were observed across various evaluation metrics, including 0.9% in accuracy, 0.25% in macro precision, 0.2% in macro recall, and 0.28% in macro F1. Additionally, ablation studies were conducted to showcase the effectiveness and robustness of our method in DDI prediction tasks.