Abstract
Background The National Institutes of Health (NIH), US Department of Health and Human Services (HHS), realized the need to better understand its Web users in order to help assure that websites are user friendly and well designed for effective information dissemination. A trans-NIH group proposed a trans-NIH project to implement an online customer survey, known as the American Customer Satisfaction Index (ACSI) survey, on a large number of NIH websites—the first “enterprise-wide” ACSI application, and probably the largest enterprise Web evaluation of any kind, in the US government. The proposal was funded by the NIH Evaluation Set-Aside Program for two years at a cost of US $1.5 million (US $1.275 million for survey licenses for 60 websites at US $18,000 per website; US $225,000 for a project evaluation contractor).Objective The overall project objectives were to assess the value added to the participating NIH websites of using the ACSI online survey, identify any NIH-wide benefits (and limitations) of the ACSI, ascertain any new understanding about the NIH Web presence based on ACSI survey results, and evaluate the effectiveness of a trans-NIH approach to Web evaluation. This was not an experimental study and was not intended to evaluate the ACSI survey methodology, per se, or the impacts of its use on customer satisfaction with NIH websites.Methods The evaluation methodology included baseline pre-project websites profiles; before and after email surveys of participating website teams; interviews with a representative cross-section of website staff; observations of debriefing meetings with website teams; observations at quarterly trans-NIH Web staff meetings and biweekly trans-NIH leadership team meetings; and review and analysis of secondary data.Results Of the original 60 NIH websites signed up, 55 implemented the ACSI survey, 42 generated sufficient data for formal reporting of survey results for their sites, and 51 completed the final project survey. A broad cross-section of websites participated, and a majority reported significant benefits and new knowledge gained from the ACSI survey results. NIH websites as a group scored consistently higher on overall customer satisfaction relative to US government-wide and private sector benchmarks.Conclusions Overall, the enterprise-wide experiment was successful. On the level of individual websites, the project confirmed the value of online customer surveys as a Web evaluation method. The evaluation results indicated that successful use of the ACSI, whether site-by-site or enterprise-wide, depends in large part on strong staff and management support and adequate funding and time for the use of such evaluative methods. In the age of Web-based e-government, a broad commitment to Web evaluation may well be needed. This commitment would help assure that the potential of the Web and other information technologies to improve customer and citizen satisfaction is fully realized.
Highlights
Project BackgroundThe Internet plays an increasingly important role for the Institutes, Centers, and Offices of the Office of the NIH Director of the National Institutes of Health (NIH)
For participating web sites: - Did the American Customer Satisfaction Index (ACSI) meet the needs and expectations of participating teams? - What were the benefits of using a continuous online customer satisfaction survey for site evaluation? - What value did the ACSI results provide? Were teams able to turn the results into action steps? What effects did any site changes have on subsequent ACSI scores?
The majority of teams indicated that they derived significant value from the data provided by the custom questions they added to their ACSI survey
Summary
Project BackgroundThe Internet plays an increasingly important role for the Institutes, Centers, and Offices of the Office of the NIH Director (subsequently referred to as ICs and ODOs) of the National Institutes of Health (NIH). The ICs and ODOs make extensive use of the Internet to accomplish their missions, including disseminating information, providing access to research information, conducting business transactions related to the IC/ODO function, and supporting administrative processes for NIH programs. Many of the ICs and ODOs have multiple web sites to support their internal functions as well as their variety of individual programs. Before 2004, web site evaluation across NIH was broadly focused on measures of web site usage (e.g., tracking reports). A minority of teams managing web sites were using web evaluation methods such as usability testing and heuristic review. Typical obstacles to broadening evaluation objectives and methods included time and resource constraints such as staff time required to develop and conduct an online survey, calendar time required for obtaining Office of Management and Budget (OMB) approval for collecting survey data, and lack of funding for web site evaluation
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
