Abstract
Comprehensive and objective evaluation of all observing assets, tools, and services within an ocean observing system is essential to maximize effectiveness and efficiency; yet, it often eludes programs due to the complexity of such robust evaluation. In order to address this need, the Pacific Islands Ocean Observing System (PacIOOS) transformed an evaluation matrix developed for the energy sector to one suitable for ocean observing. The resulting innovation is a decision analysis methodology that factors in multiple attributes (market, risk, and performance factors) and allows for selective weighting of attributes based on system maturity, external forcing, and consumer demand. This evaluation process is coupled with an annual review of priorities with respect to stakeholder needs and the program’s five-year strategic framework in order to assess the system’s components. The results provide information needed to assess the effectiveness, efficiency, and impact of each component within the system, and informs a decision-making process that determines additional investment, refinement, sustainment, or retirement of individual observing assets, services, or component groups. Regularly evaluating, and taking action to improve, modify, or terminate weak system components allows for the continuous improvement of PacIOOS services by ensuring resources are directed to the priorities of the stakeholder community. The methodology described herein is presented as an innovative opportunity for others looking for a systematic approach to evaluate their observing systems to inform program-level decision-making as they develop, refine, and distribute data and information products.
Highlights
Program evaluation, defined as “systematic investigation to determine the success of a specific program,” (Barker, 2003) is an important tool used by organizational managers to inform decision-making with respect to program effectiveness
This paper describes an innovative approach that Pacific Islands Ocean Observing System (PacIOOS)’ leadership and management refined and adopted in order to account for market, risk, economic, and performance factors as they collaborate to sustain and enhance a regional coastal and ocean observing system that addresses stakeholder needs
All authors contributed to manuscript revision and have read and approved the submitted version
Summary
Program evaluation, defined as “systematic investigation to determine the success of a specific program,” (Barker, 2003) is an important tool used by organizational managers to inform decision-making with respect to program effectiveness. A sample of the attributes gleaned from systems examined (Conley, 1987; Keeney and Raiffa, 1993; Waganer, 1998; Wholey et al, 2010) but not selected by PacIOOS for evaluation purposes include the following: liquidity; time to market; leverage (debt); profit; policy landscape; public perception; revenue diversity; prestige; competitiveness; return on investment; depletion of valued resource; supply chain dependence; environmental impact As evidenced by this list, even in commercial models, revenue and other economic factors are not the only attributes of interest when evaluating success of a company or a system. Uniqueness Do any other providers deliver a similar or identical service to stakeholders in the PacIOOS region? If so, does the similar service provide a greater, lesser, or same level of utility?
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
