Clinical outcome analysis using patient medical data facilitates clinical decision-making and increases prognostic accuracy. Recently, deep learning (DL) with learning big data features has shown expert-level accuracy in predicting clinical outcomes. Many of these sophisticated machine learning models, however, lack interpretability, creating significant trust-related healthcare issues. This necessarily requires the need for interpretable AI systems capable of explaining their decisions. In this respect, the paper proposes an interpretable classifier of the adaptive neuro-fuzzy inference method (iANFIS), which combines the fuzzy inference system with critical rule selection by attention mechanism. The rule-based processing of ANFIS helps the user to understand the behavior of the proposed model. The essential activated rule and the most important input features that predict the outcome are identified by the attention-based rule selector. We conduct two experiments with two cancer diagnostic datasets for verifying the performance of the proposed iANFIS. By using recursive rule elimination (RRE) to prune fuzzy rules, the model’s complexity is significantly reduced while preserving system efficiency that makes it more interpretable.