AbstractWeb Services (WS) can be considered as the most influent enabling technology for the next generation of web applications. WS‐based application providers will face challenging features related to nonfunctional properties in general and to performance and QoS in particular. Moreover, WS‐based developers have to provide solutions to extend such applications with self‐healing (SH) mechanisms as required for autonomic computing to face the complexity of interactions and to improve availability. Such solutions should be applicable when the components implementing SH mechanisms are deployed on both or only one platform on the WS providers and requesters sides depending on the deployment constraints. Associating application‐specific performance requirements and monitoring‐specific constraints will lead to complex configurations where fine tuning is needed to provide SH solutions. To contribute to enhancing the design and the assessment of such solutions for WS technology, we designed and implemented a monitoring and measurement framework, which is part of a larger Self‐Healing Architectures (SHA) developed during the European WS‐DIAMOND project. We implemented the Conference Management System (CMS), a real WS‐based complex application. We achieved a large‐scale experimentation campaign by deploying CMS on top of SHA on the French grid Grid5000. We experienced the problem as if we were a service provider who has to tune reconfiguration strategies. Our results are available on the web in a structured database for external use by the WS community. Copyright © 2010 John Wiley & Sons, Ltd.