Amazon Simple Storage Research Articles

ONLINE BACKUP AND RECOVERY USING AMAZON S3

The project on File recovery and backup involves using of the emerging cloud based technology Amazon Simple Storage Service (S3).This project caters the purpose of safe and secure storage of the data as well as backup of that data for an Educational Institute such as a school or a college. The process in this project involves registering the client using Facebook or Google Accounts or simple sign up form. Then client gets his own manageable storage space for storing the data on the Amazon servers. The project involves creating server side webpage which eases the Signup process. For Backup and recovery of the customer data, project uses the java platform to create client side software. The software involves the simple UI for uploading the data onto the Amazon servers. A person can share the particular data by making that data public, data is stored safely on the cloud as they use different encryption methods. And it involves a feature known as time URL which makes selected data on the cloud available to person or group of persons for a specified interval of the time. The project uses Amazon S3 Technology because Amazon servers provide Easy robust and durable storage of the data and they provide complete (99.9999 percent) availability. And also they are inexpensive and reliable than the physical storage options.

Results in the Cloud: Using Web Storage for Auction House Price Lists

The authors describe an effort led by The Metropolitan Museum of Art’s Thomas J. Watson Library, in collaboration with the Frick Art Reference Library, the National Gallery of Art Library, the Getty Research Institute Library, and the Ingalls Library at the Cleveland Museum of Art, to archive thousands of auction catalog price list files. Previous locally developed solutions to save and link this data to auction catalog bibliographic records meant the time-consuming effort was duplicated across several libraries. In 2011, The Metropolitan Museum opened an Amazon Simple Storage (S3) account to store its price list, developed a simple convention for linking price list files with bibliographic records based on OCLC numbers, and made the account available to other libraries so they could divide the responsibility for archiving the price lists.

SU-E-T-18: GATE Monte Carlo Simulation in a Cloud Computing Environment

Purpose: The GEANT4-based GATE is a unique and powerful Monte Carlo (MC) platform, which provides a single code library allowing the simulation of several specific applications, e.g. PET, SPECT, CT, internal and external radiotherapy, and hadron therapy. However, its lengthy computing time hinders its routine use in the clinic. Reducing its computing time is therefore of great importance. Thus, a commercial cloud compute service is well suited for GATE MC simulation, both in terms of cost and efficiency. This study achieves a reliable and efficient execution of GATE MC simulation and provides execution frameworks to end-users. Methods: The GATE software was ported on a commercial compute cloud environment — Amazon Elastic Compute Cloud (EC2). Simulation data was initially loaded onto the master node, and then distributed among independent worker nodes. The filed output from EC2 was sent down to the Amazon Simple Storage Service (S3). The results were finally aggregated on the local computer for display and data analysis. The distributed implementation was executed using a photon beam interacting in a 40 cm by 40 cm by 40 cm water phantom and a four-head SPECT imaging as benchmarks. Results: A cloud computing environment led to increased calculation speed for the cases implemented in this study. The speed increase scaled approximately linearly with the number of nodes used for computing. The output of the cloud-based GATE MC simulation was identical to that produced by the single-threaded implementation, and was resilient to hardware failure, indicating the reliability of the cloud computing platform. The user-friendliness offered by the workflow implementation does not introduce significant overhead. Conclusion: A cloud computing infrastructure has been established for GATE MC simulation. It substantially improves the speed of simulation, and makes rapid MC simulation for imaging/radiotherapy application possible.

Large Distributed Arabic Handwriting Recognition System based on the Combination of FastDTW Algorithm and Map-reduce Programming Model via Cloud Computing Technologies

This paper proposes a robust, efficient and scalable distributed Arabic handwriting OCR system based on a parallel FastDTW algorithm via cloud computing technologies. The three techniques Hadoop, MapReduce and Cascading are used to implement the parallel FastDTW algorithm. The experiments were deployed on Amazon EC2 Elastic Map Reduce and Amazon Simple Storage Service (S3) using a large scaled dataset built from the IFN/ENIT database.

Menu-driven cloud computing and resource sharing for R and Bioconductor

We report CRdata.org, a cloud-based, free, open-source web server for running analyses and sharing data and R scripts with others. In addition to using the free, public service, CRdata users can launch their own private Amazon Elastic Computing Cloud (EC2) nodes and store private data and scripts on Amazon's Simple Storage Service (S3) with user-controlled access rights. All CRdata services are provided via point-and-click menus. CRdata is open-source and free under the permissive MIT License (opensource.org/licenses/mit-license.php). The source code is in Ruby (ruby-lang.org/en/) and available at: github.com/seerdata/crdata. hbolouri@fhcrc.org.

ATLAS@AWS

We show how the ATLAS offline software is ported on the Amazon ElasticCompute Cloud (EC2). We prepare an Amazon Machine Image (AMI) on the basisof the standard ATLAS platform Scientific Linux 4 (SL4). Then an instance ofthe SLC4 AMI is started on EC2 and we install and validate a recent releaseof the ATLAS offline software distribution kit. The installed software isarchived as an image on the Amazon Simple Storage Service (S3) and can bequickly retrieved and connected to new SL4 AMI instances using the AmazonElastic Block Store (EBS). ATLAS jobs can then configure against the releasekit using the ATLAS configuration management tool (cmt) in the standard way.The output of jobs is exported to S3 before the SL4 AMI is terminated. Jobstatus information is transferred to the Amazon SimpleDB service. The wholeprocess of launching instances of our AMI, starting, monitoring and stoppingjobs and retrieving job output from S3 is controlled from a client machineusing python scripts implementing the Amazon EC2/S3 API via the boto libraryworking together with small scripts embedded in the SL4 AMI. We report ourexperience with setting up and operating the system using standard ATLAS jobtransforms.

Towards the integration of StoRM on Amazon Simple Storage Service (S3)

In Grid systems, a core resource being shared among geographically-dispersed communities of users is the storage. For this resource, a standard interface specification (Storage Resource Management or SRM) was defined and is being evolved in the context of the Open Grid Forum. By implementing this interface, all storage resources part of a Grid could be managed in an homogenous fashion. In this work, we consider the extension of StoRM (STOrage Resource Manager, an implementation of SRM v2.2) in order to integrate a new type of storage resource: the Amazon Simple Storage Service (Amazon S3). Amazon S3 is a simple Web services interface offering access to the same highly scalable, reliable, fast, inexpensive data storage infrastructure that Amazon uses to run its own global network of Web sites. By performing this integration, we offer to the Grid community the capability to manage and access an incredible amount of storage resources freeing them from considering the costs associated with server maintenance, or whether they have enough storage available. The characteristics of StoRM are suitable for a smooth integration with Amazon S3. In particular, StoRM is designed to be easily adapted to the underlying storage resource via a plug-in mechanism, therefore a new plugin for integration with the Amazon S3 Web Service will be written. As regards the access policies, StoRM translates the Grid authorization rules into the Amazon S3 ones and applies them to the Amazon Web Services identity.