Abstract
As the capabilities of modern X-ray detectors and acquisition technologies increase, so do the data rates and volumes produced at synchrotron beamlines. This brings into focus a number of challenges related to managing data at such facilities, including data transfer, near real-time data processing, automated processing pipelines, data storage, handling metadata and remote user access to data. The Advanced Photon Source Data Management System software is designed to help beamlines deal with these issues. This paper presents the system architecture and describes its components and functionality; the system's current usage is discussed, examples of its use have been provided and future development plans are outlined.
Highlights
Specific data management requirements typically vary from beamline to beamline, generally dependent upon the type of detectors, X-ray techniques and data processing tools in use
The Data Management System (DM) web portal provides administrative interfaces for managing user roles, experiments and experiment types, as well as for enabling and supporting beamline deployments. It is implemented as a Java EE application running in a GlassFish application server and built using Java Persistence API (JPA), Java Server Faces (JSF) and PrimeFaces component suite
The Advanced Photon Source (APS) 8-ID-I beamline may have tens of active DM processing jobs running at any given time during a user’s experiment. Those jobs are running on the APS high-performance computing (HPC) cluster and are monitored by staff and users via static web pages generated by a cron job running DM utilities
Summary
Specific data management requirements typically vary from beamline to beamline, generally dependent upon the type of detectors, X-ray techniques and data processing tools in use. Most of these requirements are related to a set of tasks that are common to all synchrotrons and other large data-generating scientific user facilities:. (ii) Enabling users and applications to find and access data This includes the use of metadata, replica catalogs and remote data access tools. Very few beamlines provided their users with any kind of remote access to their data, no beamlines had a framework for automated raw data processing and no beamlines kept track of user data or utilized searchable metadata catalogs. We discuss the system architecture and features, describe its usage, and present plans for future enhancements
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Free) 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
