Direct oral anticoagulants (DOACs) are increasingly recommended over warfarin in stroke prevention for patients with non-valvular atrial fibrillation (AF). However, there is an important evidence gap in choosing the most appropriate DOAC for Chinese patients in clinical practice. A multi-criteria decision analysis (MCDA) was adopted to build a scoring framework. Attributes and criteria were identified and determined by a scoping literature review, two rounds of Delphi surveys, and a consensus meeting. Weights of each attribute and criterion in the framework were determined using analytic hierarchy process (AHP). Evidence was collected based on the domestic or at least Asian data. Scoring methods for each criterion were developed depended on their characteristics and determined with an expert consensus meeting. Comprehensive scores of each DOAC were calculated based on the utility scores of each criterion and their corresponding weights. A total of 5 attributes, including safety, efficacy, costs/cost-effectiveness, suitability, and accessibility, were determined, and 16 criteria were under the 5 attributes. The safety and efficacy were ranked as the top two important attributes with the weights of 38.8% and 35.9%, respectively, while the suitability received the lowest weight of 7.9%. The comprehensive score for edoxaban was the highest (72.3), followed by dabigatran (49.7), rivaroxaban (37.9), and apixaban (35.8). This study provided a scoring framework developed for comprehensive evaluation of DOACs in China. The ranking of DOACs could help to support the decision-making in clinical practice. The framework could provide a reference for comprehensive evaluation of other drugs.