The power of evaluations: one year review of FAO's surveillance evaluation tool (SET) and tracking future impacts

Abstract

Purpose: With funding from the United States Agency for International Development, the Food and Agriculture Organization of the United Nations developed a standardized Surveillance Evaluation Tool (SET) to assess national animal disease surveillance systems and provide targeted recommendations in the form of a feasible and time-bound action plan. After completing the first year of missions in African countries, work is well underway to implement these plans. FAO is piloting a tracking system to measure the impact of SET recommendations at country-level and define success. Methods & Materials: SET was adapted from the Oasis toolkit by French agency ANSES, and includes components of Joint External Evaluations related to animal health surveillance. Official documentation is reviewed and interviews are conducted with actors at all levels. Outputs highlighting the strengths and weaknesses are generated and a participatory approach used to develop locally-relevant recommendations. Some SET recommendations have already been incorporated into FAO country work plans to ensure continuous support and capacity building. A tracking system is now being piloted to monitor countries’ progress towards completing SET-derived action plans. Results: A total of 13 evaluations were conducted and individual mission reports provide national veterinary services with an in-depth assessment of their disease surveillance system along with a detailed action plan for improvement. Assessments also provide a baseline to which future assessments can be compared. In addition, evaluation results can contribute to the development of national and regional strategies for animal disease surveillance and help technical and financial partners identify key priorities for capacity building. Thus far, several countries have already begun implementing specific SET recommendations, such as the development of an epidemiology unit and standardized disease reporting materials in Liberia. In future, common strengths and weaknesses across regions will be monitored, indicators created to assess impacts on local health systems, and success stories on recommendation implementation shared between countries. Conclusion: While there are known benefits from conducting evaluations, there is a growing push from the international community to develop methods that result in policy support and allow for sustainable capacity building. With a tracking strategy for implementation and impact of animal health surveillance capacity building, tangible change can be achieved.