Identification of drug-target interactions (DTI) is a crucial step in drug development with high specificity and low toxicity. To accelerate the process, computer-aided DTI prediction algorithms have been used to screen compounds or targets rapidly. Furthermore, DTI prediction can be used to identify potential targets for existing drugs, thus uncovering new indications and repositioning them. Therefore, it is of great importance to develop efficient and accurate DTI prediction algorithms. Current algorithms usually represent drugs as extracted features, which are learned by convolutional neural networks (CNNs) from its linear representation, or utilize graph neural networks (GNNs) to learn its graph representation. However, these methods either lose information or fail to capture the structural information of the drug. To address this issue, a novel molecule secondary structure representation network (MSSRN) is proposed to learn drug characterization more accurately. Firstly, the network performs relational graph convolutional networks (R-GCNs) on the drug's molecular graph and integrates drug sequence convolutions to learn the sequential information. Secondly, inspired by the attention mechanism, spatial importance weights of the drug sequence are calculated to guide R-GCNs to learn the topological information of the drug. A drug-target affinity model, called MSSRN-DTA, was then constructed by using MSSRN to learn drug structure and CNN to learn protein sequence. The effectiveness of the proposed method is verified by comparing it with other alternative methods and baseline models on two benchmark datasets.