Abstract
We have developed a highly scalable application, called Shoal, for tracking and utilizing a distributed set of HTTP web caches. Our application uses the Squid HTTP cache. Squid servers advertise their existence to the Shoal server via AMQP messaging by running Shoal Agent. The Shoal server provides a simple REST interface that allows clients to determine their closest Squid cache. Our goal is to dynamically instantiate Squid caches on IaaS clouds in response to client demand. Shoal provides the VMs on IaaS clouds with the location of the nearest dynamically instantiated Squid Cache.
Highlights
The CERN Virtual Machine File System (CVMFS) [1] is widely adopted by the High Energy Physics (HEP) community for the distribution of project software
When CVMFS is used on a cluster of worker nodes, a HTTP web proxy can be used to cache the file system contents, so that all subsequent requests for that file will be delivered from the local HTTP proxy server
The use of IaaS cloud resources is becoming a realistic solution for HEP workloads [3, 4], and CVMFS is an effective means of providing the software to the virtual machines (VMs)
Summary
The CERN Virtual Machine File System (CVMFS) [1] is widely adopted by the High Energy Physics (HEP) community for the distribution of project software. A HEP computing site has a local or regional Squid HTTP web proxy [2], with the central CVMFS servers located at the main laboratory, such as CERN for the LHC experiments. The use of IaaS cloud resources is becoming a realistic solution for HEP workloads [3, 4], and CVMFS is an effective means of providing the software to the virtual machines (VMs). One can imagine dynamically instantiating Squid servers in an opportunistic cloud environment to meet application demand. There is currently no mechanism for locating the optimal Squid server. We have developed Shoal as a new service that can dynamically publish and advertise the available Squid servers. Shoal is ideal for an environment using both static and dynamic Squid servers
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
