Code Parameters Research Articles

CPM-2: Large-scale cost-effective pre-trained language models

In recent years, the size of pre-trained language models (PLMs) has grown by leaps and bounds. However, efficiency issues of these large-scale PLMs limit their utilization in real-world scenarios. We present a suite of cost-effective techniques for the use of PLMs to deal with the efficiency issues of pre-training, fine-tuning, and inference. (1) We introduce knowledge inheritance to accelerate the pre-training process by exploiting existing PLMs instead of training models from scratch. (2) We explore the best practice of prompt tuning with large-scale PLMs. Compared with conventional fine-tuning, prompt tuning significantly reduces the number of task-specific parameters. (3) We implement a new inference toolkit, namely infmoe, for using large-scale PLMs with limited computational resources. Based on our cost-effective pipeline, we pre-train two models: an encoder-decoder bilingual model with 11 billion parameters (CPM-2) and its corresponding MoE version with 198 billion parameters. In our experiments, we compare CPM-2 with mT5 on downstream tasks. Experimental results show that CPM-2 has excellent general language intelligence. Moreover, we validate the efficiency of infmoe when conducting inference of large-scale models having tens of billions of parameters on a single GPU. All source code and model parameters are available at https://github.com/TsinghuaAI/CPM.

Non-Degraded Adaptive HEVC Steganography by Advanced Motion Vector Prediction

Current video steganography operates with either the decoded frame images or the compression coding parameters, which could cause quality degradation of the reconstructed frames. In this letter, by exploiting the advanced motion vector prediction (AMVP) technique of High Efficiency Video Coding (HEVC) standard, we propose a non-degraded adaptive steganographic approach for H.265/HEVC videos. The index value in the candidate list of the prediction unit (PU) is used for embedding. Experimental results demonstrate the superiority of the proposed steganographic approach against both hand-crafted feature-based and deep learning network-based steganalytic detectors. Our work explores a new embedding space that is not previously studied. It is a significant development in finding new ways to escape from video quality change-based steganalysis.

Modeling of basic image-lossy compression procedures using transformation coding methods, as an example

Using the set of transformation coding methods as the ground to synthesize of universal video compression algorithms is based on using the features of the human visual system: - exposure time; - observation distance; - the value of the interframe difference; - dimension and ratio of main observed objects; - type of images, etc. Transform coding allows transiting from spatial to spectral representation of the processed images and provides ample opportunities for their subsequent compression and/or steganographic processing. The main component of the research algorithm is the discrete cosine transform (DCT). It is universal for almost all types of images. DCT provides a good ability to concentrate most of the relevant information in a smaller (compared to other transformations) number of coefficients. This DCT property creates good initial conditions for all the following image compression procedures. By influencing the spectral representation of the processed images, it is possible to balance between the reproduction quality and the degree of compression of the original image, and by influencing the size of the sub-blocks and the parameters of their preprocessing, it is possible to reduce the total time of operations proceed. When using transform coding methods, the main parameters affecting the efficiency of the compression process are: - the used size of the sub-blocks into which the original image is divided; - the method of selecting significant coefficients; - the used principle of quantizing important coefficients. The size of the sub-blocks is decisive, since the number of such blocks and the total computation time for each element of the DCT matrices depend on it. An important step is the rounding process of the values obtained after DCT. The study uses a coarsening factor for this. The procedure for normalizing matrix elements is implemented by dividing the DCT values by a given coarsening constant, followed by rounding the result to the integer numbers. The aim of the work is to simulate the main steps of the compression of static images and study the influence of the main coding parameters on the qualitative and temporal characteristics of the reproduced images (processing time, computational complexity, reproduction quality, distortion characteristics, etc.).

A Nearly Optimal Method of Polar Code Constructions for the AWGN Channel

Polar code is a kind of capacity-approaching code with explicit structure as part of the next generation wireless communication standard. The performance of the polar code is directly determined by the construction method, in which there are three important parameters: code rate $(R)$ , block-length $(N)$ , and designSNR ( DSNR ). It is very difficult to obtain an optimal construction because all these three parameters should be optimized in a recursive structure model. In this article, an efficient method, which constructs the desired optimal polar code by simultaneously selecting the three parameters in terms of the error probability, is presented. In the proposed construction, a dimensionality reduction search method is used to solve a fore mentioned optimization problem and to find the relation between the performance of polar code and design parameters. At the same time, the optimal parameters $N$ and $R$ can be obtained under any DSNR by taking the average group accuracy rate as the evaluation criterion. Through this method, the appropriate solution of three parameters in the construction of the polar code can be determined according to the required error probability. The proposed construction method is verified by numerical simulation, and the optimal performance to $N$ , $R $ and DSNR is compared and discussed.

An Optimality Summary: Secret Key Agreement with Physical Unclonable Functions.

We address security and privacy problems for digital devices and biometrics from an information-theoretic optimality perspective to conduct authentication, message encryption/decryption, identification or secure and private computations by using a secret key. A physical unclonable function (PUF) provides local security to digital devices and this review gives the most relevant summary for information theorists, coding theorists, and signal processing community members who are interested in optimal PUF constructions. Low-complexity signal processing methods are applied to simplify information-theoretic analyses. The best trade-offs between the privacy-leakage, secret-key, and storage rates are discussed. Proposed optimal constructions that jointly design the vector quantizer and error-correction code parameters are listed. These constructions include modern and algebraic codes such as polar codes and convolutional codes, both of which can achieve small block-error probabilities at short block lengths, corresponding to a small number of PUF circuits. Open problems in the PUF literature from signal processing, information theory, coding theory, and hardware complexity perspectives and their combinations are listed to stimulate further advancements in the research on local privacy and security.

Low-Complexity Error Resilient HEVC Video Coding: A Deep Learning Approach.

Intra/inter switching-based error resilient video coding effectively enhances the robustness of video streaming when transmitting over error-prone networks. But it has a high computation complexity, due to the detailed end-to-end distortion prediction and brute-force search for rate-distortion optimization. In this article, a Low Complexity Mode Switching based Error Resilient Encoding (LC-MSERE) method is proposed to reduce the complexity of the encoder through a deep learning approach. By designing and training multi-scale information fusion-based convolutional neural networks (CNN), intra and inter mode coding unit (CU) partitions can be predicted by the networks rapidly and accurately, instead of using brute-force search and a large number of end-to-end distortion estimations. In the intra CU partition prediction, we propose a spatial multi-scale information fusion based CNN (SMIF-Intra). In this network a shortcut convolution architecture is designed to learn the multi-scale and multi-grained image information, which is correlated with the CU partition. In the inter CU partition, we propose a spatial-temporal multi-scale information fusion-based CNN (STMIF-Inter), in which a two-stream convolution architecture is designed to learn the spatial-temporal image texture and the distortion propagation among frames. With information from the image, and coding and transmission parameters, the networks are able to accurately predict CU partitions for both intra and inter coding tree units (CTUs). Experiments show that our approach significantly reduces computation time for error resilient video encoding with acceptable quality decrement.

Low-rank parity-check codes over Galois rings

Low-rank parity-check (LRPC) codes are rank-metric codes over finite fields, which have been proposed by Gaborit et al. (Proceedings of the workshop on coding and cryptography WCC, vol 2013, 2013) for cryptographic applications. Inspired by a recent adaption of Gabidulin codes to certain finite rings by Kamche et al. (IEEE Trans Inf Theory 65(12):7718–7735, 2019), we define and study LRPC codes over Galois rings—a wide class of finite commutative rings. We give a decoding algorithm similar to Gaborit et al.’s decoder, based on simple linear-algebraic operations. We derive an upper bound on the failure probability of the decoder, which is significantly more involved than in the case of finite fields. The bound depends only on the rank of an error, i.e., is independent of its free rank. Further, we analyze the complexity of the decoder. We obtain that there is a class of LRPC codes over a Galois ring that can decode roughly the same number of errors as a Gabidulin code with the same code parameters, but faster than the currently best decoder for Gabidulin codes. However, the price that one needs to pay is a small failure probability, which we can bound from above.

Steiner systems and configurations of points

The aim of this paper is to make a connection between design theory and algebraic geometry/commutative algebra. In particular, given any Steiner System S(t, n, v) we associate two ideals, in a suitable polynomial ring, defining a Steiner configuration of points and its Complement. We focus on the latter, studying its homological invariants, such as Hilbert Function and Betti numbers. We also study symbolic and regular powers associated to the ideal defining a Complement of a Steiner configuration of points, finding its Waldschmidt constant, regularity, bounds on its resurgence and asymptotic resurgence. We also compute the parameters of linear codes associated to any Steiner configuration of points and its Complement.

ESTIMATION OF DISPLACEMENT CAPACITY OF RECTANGULAR RC SHEAR WALLS USING EXPERIMENTAL AND ANALYTICAL DATABASE

This study is focused on the evaluation of the displacement capacity of RC shear walls using both experimental and analytical results. The first observation of the study is that few experimental results for slender RC shear walls having thicknesses larger than 150 mm are available in the literature. From the experimental database, it was observed that the mean and the median ultimate drift of squat RC shear walls is about half of that obtained for slender RC shear walls. Considering the limitation of the experimental database, the simple empirical model for the ultimate drift ratio of slender RC shear walls proposed in this study is also based on available analytical results from the literature. The model provides a good fit with the observed results and besides, due to the fact that it does not require sectional analysis of the element, it allows a rapid assessment of the displacement capacity of slender RC shear walls as a function of the seismic design code parameters. The proposed formula can be inserted in future revisions of the seismic assessment guidelines for RC structures for rapid seismic evaluation purposes.

STH/CFD coupled calculations of postulated transients from mixed to natural circulation conditions in the NACIE-UP facility

In the present paper, coupled STH/CFD calculations are performed addressing two operating conditions in the frame of the NACIE-UP experimental campaign. In order to simulate mixed and natural circulation phenomena related to liquid metals thermal hydraulics, the RELAP5/Mod3.3 STH code and the CFD code ANSYS Fluent are adopted in an integrated manner. The combined use of STH and CFD codes aims to help the user overcoming the limits of the individual techniques and thus achieving a better predicting capability with respect to each stand-alone application.Measured values and calculated results are compared. In particular, the coupled application provides the best results in terms of predicted mass flow rate and comprehensiveness of information giving the possibility to analyse the local and cross-sectional distributions. Furthermore, as an additional beneficial aspect, CFD results can be used as a tuning tool for some of the STH code parameters. The aim of the present work is highlighting the advantages and drawbacks of the adopted techniques, also reporting the margins of error that may presently be expected from these applications. Though perfectly matching results were not obtained, coupled STH/CFD application report promising modelling capabilities and may be consequently considered as a sound candidate for addressing liquid metal thermal hydraulics in complex geometries and large systems.

Faster Multiscale Dictionary Learning Method With Adaptive Parameter Estimation for Fault Diagnosis of Traction Motor Bearings

Traction motor bearings are crucial components to guarantee stable operation. Thus, it is significant to monitor the bearing condition. Dictionary learning is a powerful method to extract the characteristic condition. Compared with ordinary dictionary learning, the multiscale dictionary learning method applied to transform coefficients performs well in extracting the impact signals and takes less learning time. However, it spends more time tuning parameters to make the algorithm more efficient, especially in practical industrial applications. Hence, a faster adaptive parameter multiscale dictionary learning (Faster AP-MSDL) method is proposed in this article which adaptively chooses the scales of learning and estimates the parameters of sparse coding in dictionary learning simultaneously, and it possesses two core traits including less learning time and adaptive parameter estimation, which make this method more suitable to practical industrial application. Finally, a simulation experiment and two practical industrial experiments including a bearing run-to-failure test and a fault experiment of traction motor rolling bearing are conducted using our proposed method to verify adaptive ability and superiority. The superiority of the proposed method is also verified by comparing with other state-of-the-art methods.

LDPC codes constructed from cubic symmetric graphs

Low-density parity-check (LDPC) codes have been the subject of much interest due to the fact that they can perform near the Shannon limit. In this paper we present a construction of LDPC codes from cubic symmetric graphs. The codes constructed are (3, 3)-regular and the vast majority of the corresponding Tanner graphs have girth greater than four. We analyse properties of the codes obtained and present bounds for the code parameters, the dimension and the minimum distance. Furthermore, we give an expression for the variance of the syndrome weight of the codes constructed. Information on the LDPC codes constructed from bipartite cubic symmetric graphs with less than 200 vertices is presented as well. Some of the codes constructed are optimal, and some have an additional property of being self-orthogonal or linear codes with complementary dual (LCD codes).

Cascade Codes for Distributed Storage Systems

A novel coding scheme for exact repair-regenerating codes is presented in this paper. The codes proposed in this work can trade between the repair bandwidth of nodes (number of downloaded symbols from each surviving node in a repair process) and the required storage overhead of the system. These codes work for general system parameters $(n,k,d)$, which are the total number of nodes, the number of nodes suffice for data recovery, and the number of helper nodes in a repair process, respectively. The proposed construction offers a unified scheme to develop exact-repair regenerating codes for the entire trade-off, including the MBR and MSR points. We conjecture that the new storage-vs.-bandwidth trade-off achieved by the proposed codes is optimum. Some other key features of this code include: the construction is linear; the required field size is only $\Theta(n)$; and the code parameters and in particular sub-packetization level is at most $(d-k+1)^k$; which is independent of the number of the parity nodes. Moreover, the proposed repair mechanism is \emph{helper-independent}, that is the data sent from each helper only depends on the identity of the helper and failed nodes, but independent of the identity of other helper nodes participating in the repair process.

Dynamic Sparse Coded Multi-Hop Transmissions Using Reinforcement Learning

Packet transmissions over multi-hop lossy links are important in the future space-air-ground integrated networks. This letter considers sparse coded transmissions with reduced complexity compared to the well-known random linear network coding, which is known to be efficient in multi-hop lossy links. We propose a reinforcement learning framework for dynamically adjusting coding parameters on-line to improve the performance based on decoder feedback. A key advantage of the dynamic scheme is that it does not require a prior knowledge of the environment nor an analysis model. Extensive evaluation shows that it adapts well in scenarios where link conditions are unknown and/or changing, and achieves performance close to that of the optimal fixed schemes found by exhaustive search.

Parameter Checking Method of Large Scale Particle Model

As one of the popular discrete element numerical software, the selection of parameters of particle flow code directly affects the numerical results, especially for large-scale models. In the past, the calibration of parameters was based on the scale of laboratory test, and the scale effect of rock was ignored. In this paper, based on the trial and error method, the parameter checking method of large-scale models was proposed firstly, and then the method are checked through a large tunnel model tests. The research result show that the mechanical strength of the research large models will be increased by directly using the parameters checked by the small model, and the impact on the failure mode is also greater, but the impact on the elastic model is not great. It is of great significance to obtain the parameters of the large model by using the step-by-step checking method.

Binary primitive LCD BCH codes

Linear complementary dual (LCD) codes have attracted much attention in recent years due to their applications in implementations against side-channel attacks and fault injection attacks. Comparing coset leaders, we introduce the new concept of absolute coset leaders, which provides advantages for constructing LCD BCH codes. We then give explicit presentations for the largest, second largest and third largest absolute coset leaders and use them to construct binary LCD BCH codes. Lastly, we determine the parameters of these LCD BCH codes.

Design of Flexural Members Reinforced with GFRP Bars

Reinforced concrete has emerged as a primary construction material since the nineteenth century. A passive combination of concrete and steel reinforcing bar (rebar), this composite material is widely used in various types of structures. However, corrosion of steel reinforcement has been identified as a significant problem affecting the structural integrity of such concrete structures. In this context, the use of GFRP bars is gaining prominence due to their non-corrosive nature. This paper presents behaviour of flexural members reinforced with GFRP bars. The strength reduction factors and guidelines available with different codes for flexural members are studied. The design of flexural members reinforced with GFRP bars has been done using Indian Standard code parameters. The flexural capacity of the beams is estimated from the strain compatibility and equilibrium of forces. The theoretical and experimental results are compared. It is seen that designing the beams as an over-reinforced section will increase the stiffness of the beams.

Constructions of several classes of linear codes with a few weights

Recently, linear codes with a few weights have been constructed and extensively studied due to their applications in secret sharing, authentication codes, association schemes, and strongly regular graphs. In this paper, we construct several classes of linear codes with a few weights over Fp, where p is an odd prime. The weight distributions of these constructed codes are also settled by applications of the theory of quadratic forms and Gauss sums over finite fields. Some of the linear codes obtained are optimal or almost optimal. The parameters of these linear codes are new in most cases. Moreover, two classes of MDS codes are obtained.

Construction of quantum codes from $$\lambda $$-constacyclic codes over the ring $$\frac{\mathbb {F}_p[u,v]}{\langle v^3-v , u^3-u , uv-vu\rangle }$$

In this paper we construct quantum codes over $$\mathbb F_p$$ by using cyclic and $$\lambda $$ -cyclic codes over the ring $$R=\mathbb {F}_{p}+u\mathbb {F}_{p}+u^2\mathbb {F}_{p}+v\mathbb {F}_{p} +uv\mathbb {F}_{p}+u^2v\mathbb {F}_{p}+v^2\mathbb {F}_{p}+uv^2\mathbb {F}_{p} +u^2v^2\mathbb {F}_{p}$$ where $$v^3=v , u^3=u , uv=vu$$ and p is an odd prime integer. Using the idempotent decomposition method, we have given the parameters of the quantum code. Moreover, the structure of cyclic and $$\lambda $$ -constacyclic codes over R is studied. Some quantum codes over $$\mathbb {F}_p$$ are given.

Magneto-fluid dynamic and second law analysis in a hot porous cavity filled by nanofluid and nano-encapsulated phase change material suspension with different layout of cooling channels

Heat transfer, free convective flow and entropy generation of water, Al2O3-water, and nano encapsulated phase change material (NEPCM) diluted in water as NEPCM-water suspension in a hot enclosure is studied. The enclosure is a square porous cavity which heated from below, and the remained walls are thermally insulated. The cavity is cooled by four cooling channels with three different configurations (CONF 1–3). By considering internally heat generation, the impacts of the magnetic field (Ha = 0 and 100), Darcy number (Da=0.001 and 100) and laminar Rayleigh number (Ra=104, 105, and 106) at an equal concentration of nanoparticle and NEPCM (ϕ=2%) are evaluated by finite volume method (FVM) using ANSYS Fluent. The dimensionless form of the governing equations and entropy generation are used to tune the primitive parameters of the CFD code. The validation test and grid verification check are performed to guarantee the accuracy of the results. The results show that the average Nusselt number decreases from CONF1 to CONF3. The buoyancy effect amplified in higher Rayleigh number and porous medium while the magnetic field controls the buoyancy-driven flow. In addition, at low Rayleigh number, the cavity that contains the NEPCM-water suspension is desirable due to its heat transfer capability and low irreversibility.