Pair Encoding Research Articles

Open Vocabulary Arabic Diacritics Restoration

Diacritics restoration is a necessary component in order to develop Arabic text to speech systems. When diacritics are present, the phonetic transcription algorithm can be implemented based on a few rules. Restoring Arabic diacritics based on language model scoring is the dominant approach. A fixed vocabulary is usually used to build the language model used for scoring. Since Arabic is a morphologically rich language, the number of the Out-of-vocabulary (OOV) words is large and the diacritization algorithm fails to restore diacritics for these words. In this letter, we present a novel approach to support open vocabulary diacritics restoration based on the Byte Pair Encoding (BPE) method. The BPE method segments the words into variable length sub-word units and allows open vocabulary from fixed sub-word units dictionary. On the Tashkeela diacritization task, this open vocabulary approach outperforms the word and character based methods commonly used in the literature.

Predicate signatures from pair encodings via dual system proof technique

Abstract Recently, Attrapadung (Eurocrypt 2014) proposed a generic framework for fully (adaptively) secure predicate encryption (PE) based on a new primitive, called pair encodings. The author shows that if the underlying pair encoding scheme is either perfectly secure or computationally (doubly-selectively) secure, then the PE scheme will be fully secure. Although the pair encodings were solely introduced for PE, we show that these can also be used to construct predicate signatures, a signature analogue of PE. More precisely, we propose a generic construction of predicate signature (PS) from pair encoding schemes. Our construction provides unconditional signer privacy, and unforgeability in the adaptive model. Thereafter, we instantiate many PS schemes with new results, e.g., the first practical PS schemes for regular languages, the first attribute-based signature (ABS) scheme with constant-size signatures in the adaptive model, unbounded ABS with large universes in key-policy flavor, etc.

Arabic–Chinese Neural Machine Translation: Romanized Arabic as Subword Unit for Arabic-sourced Translation

Morphologically rich and complex languages such as Arabic, pose a major challenge to neural machine translation (NMT) due to the large number of rare words and the inability of NMT to translate them. Unknown word (UNK) symbols are used to represent out-of-vocabulary words because NMT typically operates with a fixed vocabulary size. These rare words can be effectively encoded as sequences of subword units by using algorithms, such as byte pair encoding (BPE), to tackle the UNK problem. However, for languages with highly inflected and morphological variations, such as Arabic, the aforementioned method has its own limitations that make it not effective enough for translation quality. To alleviate the UNK problem and address the inconvenient behavior of BPE when translating the Arabic language, we propose to utilize a romanization system that converts Arabic scripts to subword units. We investigate the effect of our approach on NMT performance under various segmentation scenarios and compare the results with systems trained on original Arabic form. In addition, we integrate Romanized Arabic as an input factor for Arabic-sourced NMT compared with well-known factors, namely, lemma, part-of-speech tags, and morph features. Extensive experiments on Arabic-Chinese translation demonstrate that the proposed approaches can effectively tackle the UNK problem and significantly improve the translation quality for Arabic-sourced translation. Additional experiments in this study focus on developing the NMT system on Chinese-Arabic translation. Before implementing our experiments, we first propose standard criteria for the data filtering of a parallel corpus, which helps in filtering out its noise.

Hierarchical Transfer Learning Architecture for Low-Resource Neural Machine Translation

Neural Machine Translation(NMT) has achieved notable results in high-resource languages, but still works poorly on low-resource languages. As times goes on, It is widely recognized that transfer learning methods are effective for low-resource language problems. However, existing transfer learning methods are typically based on the parent-child architecture, which does not adequately take advantages of helpful languages. In this paper, inspired by human transitive inference and learning ability, we handle this issue by proposing a new hierarchical transfer learning architecture for low-resource languages. In the architecture, the NMT model is trained in the unrelated high-resource language pair, the similar intermediate language pair and the low-resource language pair in turn. Correspondingly, the parameters are transferred and fine-tuned layer by layer for initialization. In this way, our hierarchical transfer learning architecture simultaneously combines the data volume advantages of high-resource languages and the syntactic similarity advantages of cognate languages. Specially, we utilize Byte Pair Encoding(BPE) and character-level embedding for data pre-processing, which effectively solve the problem of out of vocabulary(OOV). Experimental results on Uygur-Chinese and Turkish-English translation demonstrate the superiorities of the proposed architecture over the NMT model with parent-child architecture.

Four-level phase pair encoding and decoding with single interferometric phase retrieval for holographic data storage

We propose four-level phase pair encoding and decoding with single interferometric phase retrieval for holographic data storage. Inherent with phase pair encoding, phase shifting is generated by assigning a certain phase difference between two pixels of the phase pair. Multiple phase shifting operations are not required. In addition, a phase-readout reference beam can be a plane beam with an arbitrary phase in our method because phase shifting can be encoded on the phase-only spatial light modulator easily and accurately. Therefore, our method can not only increase the data transfer rate, but also improve the robustness of the holographic data storage system. Although the code rate of our method needs to be sacrificed by half, the code rate is still twice that of amplitude code when four-level phase encoding is used. We demonstrated experimentally that there is only a 1×10−2 order of bit error rate before error correcting, which is acceptable. We believe our method will further advance the phase-modulated holographic data storage technique.

Towards leakage-resilient fine-grained access control in fog computing

Fog Computing, a technology that takes advantage of both the paradigms of Cloud Computing and the Internet of Things, has a great advantage in reducing the communication cost. Since its introduction, fog computing has found a lot of applications, including, for instance, connected vehicles, wireless sensors, smart cities and etc. One prominent problem in fog computing is how fine-grained access control can be imposed. Functional encryption, a new cryptographic primitive, is known to support fine-grained access control. However, when it comes to some new threats in the fog computing scenario, such as side channel attacks, functional encryption cannot maintain its security. Therefore, we need new cryptographic primitives that not only provide a way to securely share data with a fine-grained access control but also are able to resist those new threats.In this paper, we consider how to construct functional encryption schemes (FEs) adaptively secure in continual memory leakage model (CML), which is one of the strongest models that allows continuous leakage on both user and master secret keys. Besides providing privacy and fine-grained access control in fog computing, our scheme can also guarantee security against side channel attacks. More concretely, we propose a generic framework for constructing fully secure leakage-resilient FEs (LR-FEs) in the CML model results from leakage-resilient pair encoding, which is an extension of pair encoding presented in the recent work of Attrapadung. In this way, our framework simplifies the design and analysis of LR-FEs into the design and analysis of predicate encodings. Moreover, we discover new adaptively secure LR-FEs, including FE for regular languages, attribute-based encryption (ABE) for large universe and ABE with short ciphertext. Above all, leakage-resilient adaptively secure functional encryption schemes can equip fog computing with higher security and fine-grained access control.

Functional encryption for computational hiding in prime order groups via pair encodings

Lewko and Waters introduced the computational hiding technique in Crypto'12. In their technique, two computational assumptions that achieve selective and co-selective security proofs lead to adaptive security of an encryption scheme. Later, pair encoding framework was introduced by Attrapadung in Eurocrypt'14. The pair encoding framework generalises the computational hiding technique for functional encryption (FE). It has been used to achieve a number of new FE schemes such as FE for regular languages and unbounded attribute based encryption allowing multi-use of attributes. Nevertheless, the generalised construction of Attrapadung's pair encoding for those schemes is adaptively secure only in composite order groups, which leads to efficiency loss. It remains a challenging task to explore constructions in prime order groups for gaining efficiency improvement, which leaves the research gap in the existing literature. In this work, we aim to address this drawback by proposing a new generalised construction for pair encodings in prime order groups. Our construction will lead to a number of new FE schemes in prime order groups, which have been previously introduced only in composite order groups by Attrapadung.

Byte Pair Transformation using Zero-Frequency Bytes with Varying Number of Passes

pair encoding (BPE) algorithm was suggested by P. Gage is to achieve data compression. It encodes all instances of most frequent byte-pair using zero-frequency byte in the source data. This process is repeated for maximum m possible number of passes until no further compression is possible, either because there are no more frequently occurring byte pairs or there are no more unused zero-frequency bytes to represent pairs. It writes out substitution information before the encoded data in each pass. This algorithm is very time consuming as it requires to determine most frequent byte-pair in each pass before starting substitution. We have proposed k- pass byte-pair transformation algorithm where k may be very very small as compared to maximum possible passes m. Our aim is to minimize the compression time and achieve equvivalent compression rate. Proposed algorithm transforms half of the possible most-frequent byte pairs in each pass except the last. In the last pass, it transforms all remaining possible byte-pairs. This reduced number of passes save the time taken in computing frequency of byte-pairs in maximum m passes. Experimental results have shown that proposed algorithm had taken 3.213, 9.794, 13.324, 16.323, 22.388 seconds with 1, 2, 3, 4 and 6 passes respectively as compared to 295.642 seconds of m-passes. Compression rate achieved due to transformation is 14.72%, 20.12%, 21.89%, 22.67% and 22.96% with 1, 2, 3, 4 and 6 passes respectively as compared to 25.55% using maximum m-passes. As the number of passes increases, compression is better with increased execution time. Our aim of achieving speed is achieved with little loss in compression rate.

A new method of fast compression of program code for ota updates in consumer devices

This paper presents a technology for over the air (OTA) updating that allows software updates in consumer devices such as mobile phones and car-navigation systems that are connected to networks, and it adopts NAND flash memories and demand paging technologies. Software updates for these kinds of consumer devices are carried out using the binary difference that minimizes the amount of update data, software construction technologies that minimize the rewriting of flash memories, and fast compression technologies that enables rapid rewriting. This paper focuses on byte pair encoding (BPE), which is one of the most efficient methods of compressing program code in consumer devices that adopts NAND flash memories and demand paging technologies. We describe a fast method of compressing BPE and evaluating it.

Transcriptional Analysis and Functional Characterization of a Gene Pair Encoding Iron-Regulated Xenocin and Immunity Proteins of Xenorhabdus nematophila

We describe a two-gene cluster encoding a bacteriocin, xenocin, and the cognate immunity protein in the insect-pathogenic bacterium Xenorhabdus nematophila, which infects and kills larval stages of the common crop pest Helicoverpa armigera. The two genes, xcinA and ximB, are present in the genome as a single transcriptional unit, which is regulated under SOS conditions. The stress-inducible promoter was activated by mitomycin C, glucose, and Fe(3+) depletion and at an elevated temperature when it was tested in Escherichia coli cells. Expression of the xenocin protein alone in E. coli inhibited the growth of this organism. The growth inhibition was abolished when the immunity protein was also present. A recombinant xenocin-immunity protein complex inhibited the growth of E. coli indicator cells when it was added exogenously to a growing culture. Xenocin is an endoribonuclease with an enzymatically active C-terminal domain. Six resident bacterial species (i.e., Bacillus, Enterobacter, Enterococcus, Citrobacter, Serratia, and Stenotrophomonas species) from the H. armigera gut exhibited sensitivity to recombinant xenocin when the organisms were grown under iron-depleted conditions and at a high temperature. Xenocin also inhibited the growth of two Xenorhabdus isolates. This study demonstrates that Fe(3+) depletion acts as a common cue for synthesis of xenocin by X. nematophila and sensitization of the target strains to the bacteriocin.

Associative and strategic components of episodic memory: A life-span dissociation.

The authors investigated the strategic component (i.e., elaboration and organization of episodic features) and the associative component (i.e., binding processes) of episodic memory and their interactions in 4 age groups (10-12, 13-15, 20-25, and 70-75 years of age). On the basis of behavioral and neural evidence, the authors hypothesized that the two components are functionally related but follow different life-span gradients. In a fully crossed design, age differences in recognition memory for single words versus word pairs (associative demand manipulation) were examined under instructions that emphasized item, pair, or elaborative-pair encoding (strategy manipulation). As predicted, the results showed that the strategic and associative components follow different life-span trajectories. Relative to younger adults, children's difficulties in episodic memory primarily reflected lower levels of strategic functioning. In contrast, older adults showed impairments in both strategic and associative components. The authors conclude that the comparison of strategic and associative components of episodic memory across the life span helps to delineate the two components' unique and interactive contributions to episodic memory performance.

Speeding Up HMM Decoding and Training by Exploiting Sequence Repetitions

We present a method to speed up the dynamic program algorithms used for solving the HMM decoding and training problems for discrete time-independent HMMs. We discuss the application of our method to Viterbi’s decoding and training algorithms (IEEE Trans. Inform. Theory IT-13:260–269, 1967), as well as to the forward-backward and Baum-Welch (Inequalities 3:1–8, 1972) algorithms. Our approach is based on identifying repeated substrings in the observed input sequence. Initially, we show how to exploit repetitions of all sufficiently small substrings (this is similar to the Four Russians method). Then, we describe four algorithms based alternatively on run length encoding (RLE), Lempel-Ziv (LZ78) parsing, grammar-based compression (SLP), and byte pair encoding (BPE). Compared to Viterbi’s algorithm, we achieve speedups of Θ(log n) using the Four Russians method, \(\Omega(\frac{r}{\log r})\) using RLE, \(\Omega(\frac{\log n}{k})\) using LZ78, \(\Omega(\frac{r}{k})\) using SLP, and Ω(r) using BPE, where k is the number of hidden states, n is the length of the observed sequence and r is its compression ratio (under each compression scheme). Our experimental results demonstrate that our new algorithms are indeed faster in practice. We also discuss a parallel implementation of our algorithms.

Associative Memory Encoding and Recognition in Schizophrenia: An Event-Related fMRI Study

We used an event-related functional Magnetic Resonance Imaging (fMRI) approach to examine the neural basis of the selective associative memory deficit in schizophrenia. Fifteen people with schizophrenia and 18 controls were scanned during a pair and item memory encoding and recognition task. During encoding, subjects studied items and pairs of visual objects. In a subsequent retrieval task, participants performed an item recognition memory test (old/new decisions) and an associative recognition test (intact/rearranged decisions). The fMRI analysis of the recognition data was restricted to correct items only and a random effects model was used. At the behavioral level, both groups performed equally well on item recognition, whereas people with schizophrenia demonstrated lower performance on associative recognition relative to the control group. At the brain level, the comparison between associative and item encoding revealed greater activity in the control group in the left prefrontal cortex and cingulate gyrus relative to the schizophrenia group. During recognition, greater left dorsolateral prefrontal and right inferior prefrontal activations were observed in the control group relative to the schizophrenia group. This fMRI study implicates the prefrontal cortex among other brain regions as the basis for the selective associative memory encoding and recognition deficit seen in schizophrenia.

String Matching Over Compressed Text on Handheld Devices Using Tagged Sub-Optimal Code (TSC)

This paper presents Tagged Sub-optimal code (TSC), a new coding technique to speed up string matching over compressed databases on personal digital assistants (PDA). TSC is a variable-length sub-optimal code that supports minimal prefix property. It always determines its codeword boundary without traversing a tree or lookup table. TSC technique may be beneficial in many types of applications: speeding up string matching over compressed text, and speeding decoding process. This paper also presents two algorithms for string matching over compressed text using TSC (SCTT) and the Byte Pair Encoding (BPE) technique (SCTB). indent Several experiments were conducted to compare the performance of TSC, Byte Pair Encoding (BPE), and Huffman code. Several PDA databases with different record sizes were used: the well-known Calgary dataset and a set of small-sized PDA databases. Experimental results show that SCTT is almost twice as fast as the Huffman-based algorithm. SCTT has also the same performance in search time as the search in uncompressed databases and is faster than the SCTB algorithm. For frequently updated PDA databases such as phone books, to-do list, and memos, SCTT is the recommended method regardless of the size of the average record length, since the time required to compress the updated records using BPE poses significant delays compared to TSC.

Vector quantization of speech line spectrum pair parameters and reflection coefficients

Methods are presented for intraframe encoding of line spectrum pairs. A two-stage vector quantizer-lattice vector quantizer (VQ-LVQ) has a 2-3 b/vector performance advantage over the split vector quantizer, and can save 1-3 b/vector, compared to multistage vector quantization, depending on the number of stages and the number of survivors retained at each stage. The vector quantizer-lattice vector quantizer has a moderate computational complexity. Among three trellis codes considered, trellis-coded vector quantization with nonlinear prediction provides line spectrum pair encoding performance similar to the split vector quantizer. Hybrid transformations are developed for encoding the reflection coefficients using vector quantization, and offer a small encoding advantage over the commonly used arcsine transformation and log area ratios.