Free Bits Research Articles

Collaborative filtering recommendation algorithm based on variational inference

Purpose The purpose of this paper is to alleviate the problem of poor robustness and over-fitting caused by large-scale data in collaborative filtering recommendation algorithms. Design/methodology/approach Interpreting user behavior from the probabilistic perspective of hidden variables is helpful to improve robustness and over-fitting problems. Constructing a recommendation network by variational inference can effectively solve the complex distribution calculation in the probabilistic recommendation model. Based on the aforementioned analysis, this paper uses variational auto-encoder to construct a generating network, which can restore user-rating data to solve the problem of poor robustness and over-fitting caused by large-scale data. Meanwhile, for the existing KL-vanishing problem in the variational inference deep learning model, this paper optimizes the model by the KL annealing and Free Bits methods. Findings The effect of the basic model is considerably improved after using the KL annealing or Free Bits method to solve KL vanishing. The proposed models evidently perform worse than competitors on small data sets, such as MovieLens 1 M. By contrast, they have better effects on large data sets such as MovieLens 10 M and MovieLens 20 M. Originality/value This paper presents the usage of the variational inference model for collaborative filtering recommendation and introduces the KL annealing and Free Bits methods to improve the basic model effect. Because the variational inference training denotes the probability distribution of the hidden vector, the problem of poor robustness and overfitting is alleviated. When the amount of data is relatively large in the actual application scenario, the probability distribution of the fitted actual data can better represent the user and the item. Therefore, using variational inference for collaborative filtering recommendation is of practical value.

Towards a Metrology class ADC based on Josephson junction devices

Precision measurement of DC and AC voltages requires an analogue to digital converter specification to have two key parameter values: the number of effective or noise free bits and the signal bandwidth. Commercial ADCs may offer the potential for use in a metrology class voltmeter but there is no easy way to compare the dynamic specification of a DMM with that of an ADC, especially when the number of bits or bandwidth changes. An architecture independent ADC/DMM figure of merit (FOM), is proposed to facilitate this comparison. At National Measurement Institutes, it is necessary to be able to measure with significantly greater precision than the device or instrument under test. Analysis shows that the FOM required of a metrology class ADC is beyond the state of the art for both ADCs and instruments. Fortunately, the use of superconducting Josephson junction based circuits offers the possibility to achieve the required precision. This paper gives the background to and reviews the progress of an EU funded research project for a quantum analogue to digital converter which is aiming to achieve the required FOM, beyond the state of the art.

Devising a method for the active coordination of video cameras in optical navigation based on the multi-agent approach

Here we present results of research into multi-agent approach to solving the problem of constructing a wireless Ad Hoc network to transmit video stream during optical navigation. When obtaining data from video cameras in real time, there is a problem of realization of effective coordination of video data. To improve the algorithms of capturing an object, it is necessary to improve the architecture of devices that utilize network metadata for the analysis of scenes from video surveillance. As a result of exploring the ways of solving the specified problems, we proposed a method for active coordination of video cameras based on the multi-agent approach (MAC-on-MAA). The application of this method increases efficiency of network cameras through their reconfiguration. We examined the problem of determining an adequate level of illuminance normalization of a video frame based on analysis of the modules throughput of multi-agent system for encoding transmitted video data. It was revealed that the time of analysis of video stream from a network decreases when ignoring the decoding of certain domains of graphic elements in the received video frames. The devised MAC-on-MAA method allows effective updating of routing tables in the reactive protocol AODV. We obtained analytic expressions for the assessment of quantitative characteristics of energy consumption by mobile agents. Protection of communication channels of video surveillance network is improved through the integration of steganocontainers in the free bits of RGB characteristics of video frames pixels.

FreeRider: Non-Local Adaptive Network-on-Chip Routing with Packet-Carried Propagation of Congestion Information

Non-local adaptive routing techniques, which utilize statuses of both local and distant links to make routing decisions, have recently been shown to be effective solutions for promoting the performance of Network-on-Chip (NoC). The essence of non-local adaptive routing was an additional network dedicated to propagate congestion information of distant links on the NoC. While the dedicated Congestion Propagation Network (CPN) helps routers to make promising routing decisions, it incurs additional wiring and power costs and becomes an unnecessary decoration when the load of NoC is light. Moreover, the CPN has to be extended if one would utilize more sophisticated congestion information to enhance the performance of NoC, bringing in even larger wiring and power costs. This paper proposes an innovative non-local adaptive routing technique called FreeRider, which does not use a dedicated CPN but instead leverages free bits in head flits of existing packets to carry and propagate rich congestion information without introducing additional wires or flits. In order to balance the network load, FreeRider adopts a novel three-stage strategy of output link selection, which adequately utilizes the propagated information to make routing decisions. Experimental results on both synthetic traffic patterns and application traces show that FreeRider achieves better throughput, shorter latency, and smaller power consumption than a state-of-the-art adaptive routing technique with dedicated CPN.

Energy-efficient register caching with compiler assistance

The register file is a critical component in a modern superscalar processor. It must be large enough to accommodate the results of all in-flight instructions. It must also have enough ports to allow simultaneous issue and writeback of many values each cycle. However, this makes it one of the most energy-consuming structures within the processor with a high access latency. As technology scales, there comes a point where register accesses are the bottleneck to performance and so must be pipelined over several cycles. This increases the pipeline depth, lowering performance. To overcome these challenges, we propose a novel use of compiler analysis to aid register caching. Adding a register cache allows us to preserve single-cycle register accesses, maintaining performance and reducing energy consumption. We do this by passing information to the processor using free bits in a real ISA, allowing us to cache only the most important registers. Evaluating the register cache over a variety of sizes and associativities and varying the read ports into the cache, our best scheme achieves an energy-delay-squared (EDD) product of 0.81, with a performance increase of 11%. Another configuration saves 13% of register system energy. Using four register cache read ports brings both performance gains and energy savings, consistently outperforming two state-of-the-art hardware approaches.

Enhanced Message Passing in Turbo Product Code

Turbo product code with single parity check (TPC/SPC) has demonstrated superior error correction capability over the conventional PRML channel. However, the performance usually suffers at higher code rate. A scheme to improve BER of TPC/SPC at higher code rate is proposed. By adding cyclic redundancy check (CRC) to TPC/SPC, a sub-block can be ensured error free after passing both the parity check and CRC. At the completion of each iteration, the messages from the error free sub-blocks are modified to greatly increase their confidence levels. The artificially high confidence level from those error free sub-block bits results in much more efficient iterations subsequently and greatly lowers the ultimate BER. It is found that enhanced message passing combined with noise prediction maximum likelihood (NPML) detection in the soft output Viterbi algorithm (SOVA) gives the best performance. Spinstand head/media signals passed through software channel are used to validate the channel performance.

A modified UTF-8 transformation format of ISO 10646 for storage optimization

Abstract ISO 10646 Universal Character Set (UCS) covers symbols in most of the world's written languages. There are various UCS transformation formats (UTF), but UTF-8 is the most important one because of its compatibility with both software systems and communication systems that assume 8-bit characters. At first, three properties an UTF-8-like transformation format should satisfy are defined to preserve the main characteristics of UTF-8. Then, a derived 5-byte sequence with 31 free bits is illustrated to construct an UTF-8-like transformation format, which is capable of resolving the dummy byte sequences locally. After that, we try to reveal if the last byte patterns of the 3-byte and 4-byte sequences in the UTF-8-like transformation format are replaced with byte pattern 1xxxxxxx, two more free bits for the 3-byte and 4-byte sequences can be increased. The final version of the derived UTF-8-like transformation format, UTF-8M, is proved to have the minimal average storage of encoding an UCS-4 character, 16.3% less than what UTF-8 requires.

Inequalities for quantum channels assisted by limited resources

The information capacities and ``distillability'' of a quantum channel are studied in the presence of auxiliary resources. These include prior entanglement shared between the sender and receiver and free classical bits of forward and backward communication. Inequalities and trade-off curves are derived. In particular an alternative proof is given that in the absence of feedback and shared entanglement, forward classical communication does not increase the quantum capacity of a channel.

Approximating Satisfiable Satisfiability Problems

We study the approximability of the Maximum Satisfiability Problem (MAX SAT) and of the boolean k -ary Constraint Satisfaction Problem (MAX k CSP) restricted to satisfiable instances. For both problems we improve on the performance ratios of known algorithms for the unrestricted case. Our approximation for satisfiable MAX 3CSP instances is better than any possible approximation for the unrestricted version of the problem (unless P=NP). This result implies that the requirement of perfect completeness weakens the acceptance power of non-adaptive PCP verifiers that read 3 bits. We also present the first non-trivial results about PCP classes defined in terms of free bits that collapse to P.

Introduction to Special Section on Probabilistic Proof Systems

The study of probabilistically verifiable proofs originated in the mid 1980s with the introduction of Interactive Proof Systems (IPs). The primary focus of research in this area in the '80s has been twofold: the role of zero-knowledge interactive proofs within cryptographic protocols, and characterizing which languages are efficiently interactively provable. In the 1990s, the focus of research on the topic shifted. Extensions of the interactive proof model, such as Multiprover Interactive Proofs (MIPs) and Probabilistically Checkable Proofs (PCPs), were considered with the intention of expanding our notion of what should be considered efficiently verifiable. In addition, researchers have taken a closer look at the exact resources (and tradeoffs amongst them) needed to verify a proof using various proof systems. This culminated in the important discovery that it is possible to verify NP statements (with a constant error probability) by only examining a constant number of bits of a PCP and using logarithmic amount of randomness. Perhaps, however, the most dramatic development has been the connection which was found between probabilistically verifiable proofs and proving hardness of approximation for optimization problems. It has been shown that a large variety of optimization versions of NP-hard problems (e.g., the maximum size of a clique in a graph, the minimum number of colors necessary to color a graph, and the maximum number of clauses satisfiable in a CNF formula) are not only NP-hard to solve exactly but also NP-hard to approximate in a very strong sense. The tools to establish hardness of approximation came directly from results on MIPs and PCPs. Indeed, almost every improvement in the efficiency of these proof systems translates directly into showing larger factors within which these optimization problems are hard to approximate. In 1994--1995 two exciting workshops were held at the Weizmann Institute in Israel on the new developments in probabilistically verifiable proofs and their applications to approximation problems, cryptography, program checking, and complexity theory at large. Over 60 papers were presented in the workshop series, and we are proud to include three of them in this special section. "On the Power of Finite Automata with Both Nondeterministic and Probabilistic States" by Anne Condon, Lisa Hellerstein, Samuel Pottle, and Avi Wigderson, considers constant round interactive proof systems where the verifier is restricted to use constant space and public coins. An equivalent characterization is finite automata with both nondeterministic and random states (npfa's), which accept their languages with a small probability of error. The paper shows that npfa's restricted to run in polynomial expected time accept only the regular languages in the case of npfa with 1-way input head, and that if Lis a nonregular language, then either L or its complement is not accepted by any npfa with a 2-way input head. "A Parallel Repetition Theorem" by Ran Raz, addresses and resolves the Parallel Repetition Conjecture which has eluded researchers for some time. The broader topic is what happens to the error probability of proof systems when they are composed. It has been known for awhile that sequential composition of proof systems (both single and multiprover interactive proofs) reduces the error exponentially, but this increases the number of rounds. For interactive proof systems, parallel repetition is known to reduce the error exponentially, and the Parallel Repetition Conjecture asserts that the same holds in a one-round two-prover proof system. Raz proves a constructive bound on the probability of error which indeed reduces at an exponential rate. The constant in the exponent is logarithmic in the total number of possible answers of the two provers, which means one can achieve two-prover one-round MIPs for NP statements with arbitrarily small constant error probability. This, in turn, has played a crucial role in further developments in the area and in particular in those reported in the next paper. "Free Bits, PCPs, and Nonapproximability---Towards Tight Results" by Mihir Bellare, Oded Goldreich, and Madhu Sudan, continues the investigation of PCPs and nonapproximability with emphasis on trying to get closer to tight results. The work consists of three parts. The first part presents several PCP proof systems for NP, based on a new error-correcting code called the Long Code. The second part shows that the connection between PCPs and hardness of approximation is not accidental. In particular, it shows that the transformation of a PCP for NP into hardness results for MaxClique can be reversed. Finally, the third part initiates a systematic investigation of the properties of PCPs as a function of the various parameters: randomness, query complexity, free-bit complexity, amortized free-bit complexity, proof size, etc. Two more papers submitted for this special section were not ready at this time for publication. They will appear in future issues of the SIAM Journal on Computing.

Free Bits, PCPs, and Nonapproximability---Towards Tight Results

This paper continues the investigation of the connection between probabilistically checkable proofs (PCPs) and the approximability of NP-optimization problems. The emphasis is on proving tight nonapproximability results via consideration of measures such as the "free-bit complexity" and the "amortized free-bit complexity" of proof systems. The first part of the paper presents a collection of new proof systems based on a new error-correcting code called the long code. We provide a proof system that has amortized free-bit complexity of $2 + \epsilon$, implying that approximating MaxClique within $N^{\frac13-\e}$, and approximating the Chromatic Number within $N^{\frac15-\e}$, are hard, assuming $\NP\neq\coRP$, for any e > 0. We also derive the first explicit and reasonable constant hardness factors for Min Vertex Cover, $\MSAT{2}$, and Max Cut, and we improve the hardness factor for $\MSAT{3}$. We note that our nonapproximability factors for $\maxsnp$ problems are appreciably close to the values known to be achievable by polynomial-time algorithms. Finally, we note a general approach to the derivation of strong nonapproximability results under which the problem reduces to the construction of certain "gadgets." The increasing strength of nonapproximability results obtained via the PCP connection motivates us to ask how far this can go and whether PCPs are inherent in any way. The second part of the paper addresses this. The main result is a "reversal" of the connection due to Feige et al. (FGLSS connection) [J. ACM, 43 (1996), pp. 268--292]: where the latter had shown how to translate proof systems for NP into NP-hardness of approximation results for MaxClique, we show how any NP-hardness of approximation result for MaxClique yields a proof system for NP. Roughly, our result says that for any constant f, if MaxClique is NP-hard to approximate within N1(1+f), then $\NP\subseteq \overline{\fpcp}[\log,f]$, the latter being the class of languages possessing proofs of logarithmic randomness and amortized free-bit complexity f. This suggests that PCPs are inherent to obtaining nonapproximability results. Furthermore, the tight relation suggests that reducing the amortized free-bit complexity is necessary for improving the nonapproximability results for MaxClique. The third part of our paper initiates a systematic investigation of the properties of PCP and FPCP (free PCP) as a function of the following various parameters: randomness, query complexity, free-bit complexity, amortized free-bit complexity, proof size, etc. We are particularly interested in triviality results, which indicate which classes are not powerful enough to capture NP. We also distill the role of randomized reductions in this area and provide a variety of useful transformations between proof checking complexity classes.