Understanding the reason why a prediction has been made by a machine is crucial to grant trust to a human decision-maker. However, data mining based decision support systems are, in general, not designed to promote interpretability; instead, they are developed to improve accuracy. Interpretability becomes a more challenging issue in the context of data stream mining. Where the prediction model has to deal with enormous volumes of data gathered continuously at a fast rate and whose underlying distribution may change over time. On the one hand, the majority of the methods that address classification in a data stream are black-box models or white-box models into ensembles. Either do not provide a clear view of why a particular decision has been made. On the other hand, white-box models, such as rule-based models, do not provide acceptable accuracy to be considered in many applications. This paper proposes modeling the data as a special graph, which is built over the attribute space, and from which interpretable rules can be extracted. To overcome concept drift and enhance model accuracy, different variants of such graphs are considered within an ensemble that is updated over time. The proposed approach has shown the best overall classification results when compared to six rule-based algorithms in twelve streaming domains.