Backup mainframe data to object storage systems

Abstract

Mainframe systems backup and data archival is performed via proprietary data movement protocols to FICON-attached tape devices or virtual tape libraries using specialized backup software running on the mainframe operating system. These proprietary tape storage systems do not offer support for TCP/IP-attached storage solutions and for the standard protocols used to access these systems. The mainframe backup and archive software is tightly coupled with the tape architecture and by design writes data sets as a sequential, serial stream of data to tape. As the amounts of data in mainframe systems keep growing, the resource intensive backup and archive process become more expensive. The team developed software-defined patented[1] technology that enables backup, archive and restore of mainframe systems directly to TCP/IP-attached object storage systems. Moreover, the technology provides relief in CPU resource utilization by offloading the backup archive and restore processes from standard Central Processors (CP) to zIIP-processors [2]. The software engine invokes the built-in operating system data backup program, DFDSS, to read data sets from disk. DFDSS is invoked using multiple call-back functions that receive control during its operation. The call-back functions are used to receive mainframe disk data as it is read by DFDSS and move it over a shared memory buffer to the software engine. The software engine implements, among others, an S3-compatible client, that can write the received mainframe data directly to object storage systems over TCP/IP. The engine buffers data received from DFDSS in chunks. When a chunk is filled, it is taken by the next available thread to be sent to the object storage. Before sending, the thread may compress and encrypt the chunk data. Each chunk is written as its own object in the object storage. Multiple chunks may be handled and sent in parallel, improving performance over serial write to tape. Finally, a mapping object is written that includes an index of all chunks comprising a backed up mainframe data set. Metadata is assigned to the mapping object to document the logical attributes of the data set such as its block size and record format. This metadata is used during restore to reconstruct the data set. The metadata may also be searched or queried by the object storage management interface. Mapping objects follow a naming convention that allows locating them using direct access upon a restore request, without the need for searching or querying the object storage. This also saves the need to maintain an additional backup catalog on the mainframe side.