As a report of a working group at ITiCSE 2005, this paper represents a vision of the use of XML specifications and tools in algorithm visualization, particularly with regard to supporting user interaction. A detailed description is given of how an interesting event to be visualized is decomposed, combined with interactive questions, narratives, control flow code and metadata, and finally rendered into graphical primitive and transformation specifications. The heart of the paper is our discussion of XML specifications for content generation (the object being visualized), interactive questions, and graphical primitives and transformations, with briefer discussions of narratives and metadata. Examples are provided for each in an appendix, with fuller details to be published on an associated website that we hope will become a source of future standards in this area. In conclusion, the approach of the working group is discussed, and important remaining challenges are identified.