Unique Chunks Research Articles

InDe: An Inline Data Deduplication Approach via Adaptive Detection of Valid Container Utilization

Inline deduplication removes redundant data in real-time as data is being sent to the storage system. However, it causes data fragmentation: logically consecutive chunks are physically scattered across various containers after data deduplication. Many rewrite algorithms aim to alleviate the performance degradation due to fragmentation by rewriting fragmented duplicate chunks as unique chunks into new containers. Unfortunately, these algorithms determine whether a chunk is fragmented based on a simple pre-set fixed value, ignoring the variance of data characteristics between data segments. Accordingly, when backups are restored, they often fail to select an appropriate set of old containers for rewrite, generating a substantial number of invalid chunks in retrieved containers. To address this issue, we propose an inline deduplication approach for storage systems, called InDe , which uses a greedy algorithm to detect valid container utilization and dynamically adjusts the number of old container references in each segment. InDe fully leverages the distribution of duplicated chunks to improve the restore performance while maintaining high backup performance. We define an effectiveness metric, valid container referenced counts (VCRC) , to identify appropriate containers for the rewrite. We design a rewrite algorithm F-greedy that detects valid container utilization to rewrite low-VCRC containers. According to the VCRC distribution of containers, F-greedy dynamically adjusts the number of old container references to only share duplicate chunks with high-utilization containers for each segment, thereby improving the restore speed. To take full advantage of the above features, we further propose another rewrite algorithm called F-greedy+ based on adaptive interval detection of valid container utilization. F-greedy+ makes a more accurate estimation of the valid utilization of old containers by detecting trends of VCRC’s change in two directions and selecting referenced containers in the global scope. We quantitatively evaluate InDe using three real-world backup workloads. The experimental results show that compared with two state-of-the-art algorithms (Capping and SMR), our scheme improves the restore speed by 1.3×–2.4× while achieving almost the same backup performance.

Improving the Performance of Deduplication-Based Backup Systems via Container Utilization Based Hot Fingerprint Entry Distilling

Data deduplication techniques construct an index consisting of fingerprint entries to identify and eliminate duplicated copies of repeating data. The bottleneck of disk-based index lookup and data fragmentation caused by eliminating duplicated chunks are two challenging issues in data deduplication. Deduplication-based backup systems generally employ containers storing contiguous chunks together with their fingerprints to preserve data locality for alleviating the two issues, which is still inadequate. To address these two issues, we propose a container utilization based hot fingerprint entry distilling strategy to improve the performance of deduplication-based backup systems. We divide the index into three parts: hot fingerprint entries, fragmented fingerprint entries, and useless fingerprint entries. A container with utilization smaller than a given threshold is called a sparse container . Fingerprint entries that point to non-sparse containers are hot fingerprint entries. For the remaining fingerprint entries, if a fingerprint entry matches any fingerprint of forthcoming backup chunks, it is classified as a fragmented fingerprint entry. Otherwise, it is classified as a useless fingerprint entry. We observe that hot fingerprint entries account for a small part of the index, whereas the remaining fingerprint entries account for the majority of the index. This intriguing observation inspires us to develop a hot fingerprint entry distilling approach named HID . HID segregates useless fingerprint entries from the index to improve memory utilization and bypass disk accesses. In addition, HID separates fragmented fingerprint entries to make a deduplication-based backup system directly rewrite fragmented chunks, thereby alleviating adverse fragmentation. Moreover, HID introduces a feature to treat fragmented chunks as unique chunks. This feature compensates for the shortcoming that a Bloom filter cannot directly identify certain duplicated chunks (i.e., the fragmented chunks). To take full advantage of the preceding feature, we propose an evolved HID strategy called EHID . EHID incorporates a Bloom filter, to which only hot fingerprints are mapped. In doing so, EHID exhibits two salient features: (i) EHID avoids disk accesses to identify unique chunks and the fragmented chunks; (ii) EHID slashes the false positive rate of the integrated Bloom filter. These salient features push EHID into the high-efficiency mode. Our experimental results show our approach reduces the average memory overhead of the index by 34.11% and 25.13% when using the Linux dataset and the FSL dataset, respectively. Furthermore, compared with the state-of-the-art method HAR, EHID boosts the average backup throughput by up to a factor of 2.25 with the Linux dataset, and EHID reduces the average disk I/O traffic by up to 66.21% when it comes to the FSL dataset. EHID also marginally improves the system's restore performance.

Mise en Scène: A Film Scholarship Augmented Reality Mobile Application

Introduction The presentation of film scholarship in the United States, like the majority of scholarship in academia, has remained relatively unchanged for decades; it was, and is, primarily textual. However, in addition to printed text, film scholarship has often employed the still frame or screen capture as a way of integrating film's visual dimension. And while the screen capture continues to be a valuable tool in the presentation of film analysis, it is only one of many possible forms of multimedia. Mise en Scene, an augmented reality (AR) mobile application, is a tool that has been designed to bring cinema scholarship into a kinetic and auditory realm more closely in line with the actual medium being studied: film. By embracing digital mobile technology, specifically AR, Mise en Scene is able to augment printed articles and books with short video examples, audio clips, and image galleries. Mise en Scene can augment both existing scholarship and works that are crafted with the technology specifically in mind. It should be noted that some attempts have been made to implement audio and video into film scholarship, but these attempts have been limited in both the technology used and the exposure achieved. Journals like Mediascape, Brighlights Film Journal, Refractory, and Underground Film Journal do embed videos into their online articles using Flash players like YouTube or JW Player, but in a rudimentary way that resemble blog articles. Additionally, the PDF versions of their articles that are available to print do not contain any ability to access the selected content. This project was a result of frustration with the current limitations in traditional scholarly exhibition. The impetus was an unpublished article on Alfred Hitchcock's Strangers on a Train that included nearly 30 still images in a protracted appendix. Due largely in part to physical space constraints, the inclusion of more than only a few still images in a single article on film is largely discouraged within the community of publishers and editors and thus the Hitchcock article was untenable from a publishing perspective. It became apparent that the number of still images in the Hitchcock article could have been reduced had the exhibition format of scholarly findings allowed for the inclusion of video and audio clips instead of just screen captures. Augmented reality offered two unique opportunities for film scholarship: a seamless merging of digital content and physical paper and the ability to more easily enhance archived works of nondigitized film scholarship. In the original conception of Mise en Scene as an enhanced PDF reader, film scholarship would either have to be created digitally with the application in mind or already-printed, pre-digital texts would have to be digitized and then enhanced. With augmented reality, physical books and articles could be enhanced/augmented without the need for digitization of the original work--digital and analogue would no longer have to be mutually exclusive. It is easily within the realm of possibility that a book or article from the early 20th century could, without digitization, be enhanced with digital multimedia. While this cross-generation augmentation was purely conceptual in the early stages of the project, the concept became reality when the Mise en Scene development teams were not only able to isolate specific articles to ensure there wouldn't be any overlaps between documents, but also figured out a way to instruct the AR software to recognize unique chunks of text, not just images. These breakthroughs allow any book or article from any time and any printer to be uniquely augmented with digital multimedia content without having to digitize the original source. Mise en Scene not only addresses an enormous technological gap in the film-scholarly exchange, it accurately reflects the complexity of the medium of film; cinema scholarship, for the first time in its history, can be presented in a way that utilizes the aural and kinetic nature of film itself. …