Abstract
Digital catalogs of ocean data have been available for decades, but advances in standardized services and software for catalog searches and data access now make it possible to create catalog-driven workflows that automate—end-to-end—data search, analysis, and visualization of data from multiple distributed sources. Further, these workflows may be shared, reused, and adapted with ease. Here we describe a workflow developed within the US Integrated Ocean Observing System (IOOS) which automates the skill assessment of water temperature forecasts from multiple ocean forecast models, allowing improved forecast products to be delivered for an open water swim event. A series of Jupyter Notebooks are used to capture and document the end-to-end workflow using a collection of Python tools that facilitate working with standardized catalog and data services. The workflow first searches a catalog of metadata using the Open Geospatial Consortium (OGC) Catalog Service for the Web (CSW), then accesses data service endpoints found in the metadata records using the OGC Sensor Observation Service (SOS) for in situ sensor data and OPeNDAP services for remotely-sensed and model data. Skill metrics are computed and time series comparisons of forecast model and observed data are displayed interactively, leveraging the capabilities of modern web browsers. The resulting workflow not only solves a challenging specific problem, but highlights the benefits of dynamic, reusable workflows in general. These workflows adapt as new data enter the data system, facilitate reproducible science, provide templates from which new scientific workflows can be developed, and encourage data providers to use standardized services. As applied to the ocean swim event, the workflow exposed problems with two of the ocean forecast products which led to improved regional forecasts once errors were corrected. While the example is specific, the approach is general, and we hope to see increased use of dynamic notebooks across geoscience domains.
Highlights
This notebook searches for data by querying a Catalog Service for the Web (CSW) service to extract metadata records from the Integrated Ocean Observing System (IOOS) that meet the user‐specified criteria, extracts data from the Sensor Observation Service (SOS) and OPeNDAP+CF data service endpoints discovered in the returned records
This notebook searches for data by querying a CSW service to extract metadata records from the IOOS that meet the user-specified criteria, extracts data from the SOS and OPeNDAP+CF
There are a number of benefits to developing dynamic, reusable workflows
Summary
When tackling a scientific or engineering problem that requires integrating information from a large number of data providers, a challenging issue can be finding and acquiring the data. HHoowweevveerr,, wwiitthh tthhee iinnccrreeaassiinngg aavvaaiillaabbiilliittyy ooff ssttaannddaarrddiizzeedd,, vvaarriieedd,, aanndd ppoowweerrffuull wweebb-‐bbaasseedd ddaattaa sseerrvviicceess,, uusseerrssccaannnnoowwemempplolyoycactaatlaolgogsesrevrivceicsetsotnoont ootnloynfliyndfinddatdaaatcaroascsromssulmtipulletipprleovpidroevrsid, berust, bthuetnthtoenactoquaicrqeutihreestehdesaetadiantaaipnraogprraomgrmamatmicawtiacyw, saoy,tshoatthbaotthbosteharsceharacnhdanacdceascsceasrseaaruetaoumtoamtedat.ed. Soinncoet water temperatures can fluctuate substantially with wind events, such as upwelling, race organizers. S3ionfc1e2 water temperatures can fluctuate substantially with wind events, such as upwelling, race organizers aree iinntteerreesstteeddiinnffoorreeccaassttwwaateterrtetmemppereartauturersesosothtehyeycacnaninifnofromrmswsiwmimemrserasnadntdhethiresirupsuppoprtotretatmeasmosf hoof whohwazhaarzdaorudsoucosncdointidointisoanrsealriekelilkyetloybtoe.be. SScreeenshot from aa wweebb bbrroowwsseerr,, ddiissppllaayyiinngg tthhee oorriiggiinnaall pprreeddiiccttiion ooff ssuurrffaaccee ccuurreents aannd forecast water temperatures in the Boston Harbor region from the IOOS NECOFS-‐MassBay model. TThhe sswwimmers start offshore in colder water, and swim into the harbor following the incoming tide. TThhee mmooddeell iiss pprreeddiiccttiinngg 5522––5533 ◦°F water temperatures at the race start, dangerously cold for a nnoo-‐wwet suit swwimm
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
