Abstract
Data sharing is required for research collaborations, but effective data transfer performance continues to be difficult to achieve. The NetSage Measurement and Analysis Framework can assist in understanding research data movement. It collects a broad set of monitoring data and builds performance Dashboards to visualize the data. Each Dashboard is specifically designed to address a well-defined analysis need of the stakeholders. This paper describes the design methodology, the resulting architecture, the development approach and lessons learned, and a set of discoveries that NetSage Dashboards made possible.
Highlights
Scientific investigation is highly collaborative and requires the ability to seamlessly share data between institutions to enable scientific discovery
It is not uncommon for researchers to resort to shipping disks instead of using the network for data delivery due to network performance issues, such as is done for the Event Horizon Telescope black hole analysis [19]
A significant change in behavior was experienced by this link in early 2018 because the US Department of Energy network operators added the NEAAR circuit to the set of network resources that support data transfers related to the Large Hadron Collider (LHC)
Summary
Scientific investigation is highly collaborative and requires the ability to seamlessly share data between institutions to enable scientific discovery. We walk through several use cases to show the types of analyses and discoveries that the NetSage Framework enables This paper describes both the international and US domestic deployments of NetSage Portals as of April 2021. Previous publications about the NetSage Framework, for example [93], only addressed the International Portal and a minimal set of components and use cases. Other publications, such as [29, 30], focused on how the project worked with large-scale data and some very preliminary deployments prior to wide-scale collection of flow data. Presentations by the NetSage Project at numerous venues are available on the NetSage Project Website [56]
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
