JPEG Encoder Research Articles

A coded knowledge distillation framework for image classification based on adaptive JPEG encoding

In knowledge distillation (KD), a lightweight student model yields enhanced test accuracy by mimicking the behavior of a pre-trained large model (teacher). However, the cumbersome teacher model often makes over-confident responses, resulting in poor generalization when presented with unseen data. Consequently, a student trained by such a teacher also inherits this problem. To mitigate this issue, in this paper, we present a new framework of KD dubbed coded knowledge distillation (CKD) in which the student is trained to mimic instead the behavior of a coded teacher. Compared to the teacher in KD, the coded teacher in CKD has an additional adaptive encoding layer in the front, which adaptively encodes an input image into a compressed version (using JPEG encoding for instance) and then feeds the compressed input image to the pre-trained teacher. Comprehensive experimental results show the effectiveness of CKD over KD. In addition, we extend the deployment of a coded teacher to other knowledge transfer methods, showcasing its ability to enhance test accuracy across these methods.

A Methodology for Fault-tolerant Pareto-optimal Approximate Designs of FPGA-based Accelerators

Approximate Computing Techniques (ACTs) take advantage of resilience computing applications to trade off among output precision, area, power, and performance. ACTs can lead to significant gains at affordable costs when efficiently implemented on Field Programmable Gate Array– (FPGA) based accelerators. Although several novel ACTs works have been proposed for FPGA accelerators, their applicability to high-assurance systems has not been explored as much. ACTs are becoming necessary in many critical Edge computing systems, such as self-driving cars and Earth observation satellites, to increase computational efficiency. However, an important question comes to mind when targeting critical systems: Does ACT optimization negatively affect the reliability of the system and how can one find optimal design architectures that blend classic mitigation techniques like Triple Modular Redundancy with approximation- and precise-based arithmetic hardware units to achieve the best possible computational efficiency without compromising dependability? This work aims to solve this research problem by introducing a Design Space Exploration (DSE) methodology that employs ACTs in arithmetic units of the design and identifies Pareto-optimal microarchitectures that balance all relevant gains of ACTs, such as area, speed, power, failure rate, and precision, by inserting the correct amount of approximation in the design. In a nutshell, our DSE methodology has formulated the DSE with a Multi-Objective Optimization Problem (MOP). Each Pareto-optimal solution of our tool finds which arithmetic units of the design to implement with precise and approximate circuits and which units to selectively triplicate to remove single points of failure that compromise system reliability below acceptable thresholds. We also suggest another formulation of the DSE into a Single-Objective constraint Optimization Problem (ScOP) producing a single optimal point, and that the user may demand, as a less time-consuming alternative to the MOP if a complete Pareto-front is not needed. Our methodology generates fault-tolerant versions of the Pareto-optimal approximate designs (or simple optimized approximate designs if the ScOP choice is picked) by selectively applying mitigation techniques in a way that the overheads of redundant resources for fault-tolerance do not negate the gains of approximation in comparison to the fault-tolerant versions of the precise design. We evaluate our method on two FPGA-based accelerators: a JPEG encoder and an H.264/Advanced Video Coding decoder. Our experimental results show significant gains in area, frequency, and power consumption without compromising output quality and system reliability compared to classic solutions that replicate all or a part of the resources of the precise design to increase dependability metrics.

Low-complexity JPEG quantization table requiring only bit-shift operations

ABSTRACT In this paper, a low-complexity quantization table is proposed for the baseline JPEG encoder. The proposed scheme does not require any multiplications or additions; only bit-shift operations are involved. The computational complexity should be drastically reduced without almost incurring any significant performance loss. The resulting approximate quantization table has extremely low arithmetic cost and is compared to state-of-the-art schemes in terms of computational complexity. Comparisons showed that our proposal exhibits better or similar results. The proposed low-complexity quantization table is also investigated in image compression context. It could outperforms other related schemes in low and high-compression scenarios. Comparisons with respect to average PSNR and SSIM are conducted. Experiment outcomes show up to 4.03 dB improvement in average PSNR over the state-of-the-art competitor. The proposed quantization table has a considerably lower computational cost while maintaining much higher image quality.

Implementation of Fast Huffman Encoding Based on FPGA

In order to improve the efficiency of JPEG encoding architecture, some imagine compression systems use pipeline technology. Different modules will spend different cycles in the image compression, so the pipeline is prone to blockage. This paper proposes uses doublebyte splicing output Huffman code architecture and double Huffman encoding units on FPGA, so that Huffman encoding process spent less clock cycles in JPEG encoding, and pipelined JPEG encoding algorithm can be encoded in real time without blocking. According to the FPGA verification, although the number of logic units used will increase, Huffman encoding takes less clock cycles.

A fast and concise parallel implementation of the 8x8 2D forward and inverse DCTs using halide

The Discrete Cosine Transform (DCT) is commonly used for image and video coding and very efficient implementations of the forward and inverse transforms are of great importance. The popular libjpeg-turbo library contains handwritten, highly-optimised assembly language DCT implementations utilizing SIMD instruction sets for a variety of architectures. We present an alternative approach, implementing the 8x8 IDCT and FDCT written in the functional image processing language Halide. We show how less than 200 lines of Halide can replace over 20,000 lines of code the libjpeg-turbo library to perform JPEG encoding and decoding. The Halide implementation is compared for ARMv8 NEON and x86-64 SIMD extensions and shows a 5–25 percent performance improvement over the SIMD code in libjpeg-turbo for decoding and a 10–40 percent improvement for encoding. The Halide code is significantly easier to maintain and port to new architectures than the existing code.

Joint compressed sensing and JPEG coding based secure compression scheme in OFDM-PON

This paper proposes a secure compressed communication scheme in orthogonal frequency division multiplexing passive optical network (OFDM-PON) based on joint compressed sensing and JPEG coding (JCS-JPEG) and multilevel encryption, with a focus to its application in secure storage and transmission. Specifically, working with the under-sampling data of compressed sensing (CS), we combine the JPEG coding standard to deepen compression into the storage and transmission link while inserting random factors into Deoxyribonucleic Acid (DNA) encryption to improve the security of the physical layer. The rationale behind the joint compression is that the data after CS is suitable for block quantization and entropy encoding methods to compress further, similar to the JPEG encoding. For the compression of multimedia data, we demonstrate that it can achieve a satisfactory reconstruction effect and provide an additional 5 to 8 times compression ratio on images. Moreover, we show that the DCT coefficient encryption and random DNA encryption can be seamlessly incorporated into the encryption in CS through the two dimensions of a 2D-chaotic map, thus realizing secure compression and transmission when under-sampling. The proposed scheme is expected to reduce the computational complexity and storage pressure of light resource-constrained sensors, simultaneously improve bandwidth utilization and sensitive information protection in OFDM-PON.

The Seventh International Olympiad in Cryptography: Problems and solutions

The International Olympiad in Cryptography NSUCRYPTO is the unique Olympiad containing scientific mathematical problems for professionals, school and university students from any country. Its aim is to involve young researchers in solving curious and tough scientific problems of modern cryptography. In 2020, it was held for the seventh time. Prizes and diplomas were awarded to 84 participants in the first round and 49 teams in the second round from 32 countries. In this paper, problems and their solutions of NSUCRYPTO'2020 are presented. We consider problems related to attacks on ciphers and hash functions, protocols, permutations, primality tests, etc. We discuss several open problems on JPEG encoding, Miller -- Rabin primality test, special bases in the vector space, AES-GCM. The problem of a modified Miller -- Rabin primality test was solved during the Olympiad. The problem for finding special bases was partially solved.

Semi-fragile watermarking scheme based on perceptual hash function (PHF) for image tampering detection

The increasing performances of personal computers, as well as software of image processing, enable the easy manipulation of digital media content. Unfortunately, this easy manipulation makes the detection of changes in the multimedia content a very difficult task. Digital watermarking is among the most appropriate techniques to verify the integrity of multimedia content. In this paper, we propose a semi-fragile watermarking scheme for JPEG2000 image self-authentication that ensuring the robustness of the watermark against the compression attacks generated by the JPEG2000 encoder itself. The scheme is combined with the JPEG2000 encoder by embedding the generated watermark into the host image during the JPEG2000 compression process. To generate the watermark we suppose to use a perceptual hash function (PHF) operating on discrete wavelet coefficients of the host image. The proposed watermark generation process leads the system to verify the integrity of the image without the need to any file except for the watermarked image. The watermark is embedded during the JPEG2000 compression process after Discrete Wavelet Transform (DWT) step into the approximation sub-band coefficients of the five wavelet decomposition using Index Modulation Quantification (QIM), and can be extracted during image decoding. To prove the authentication of the image, the system compares the extracted watermark with the new watermark generated from the received image. Experimental results show that our proposed approach has not only an extremely high accuracy of tampering detection but also a relatively very high resistance against JPEG2000 compression attacks.

Region of Interest Coding Based on Convolutional Neural Network

The traditional region of interest coding method mainly uses low-level features to detect the Region of Interest (ROI). The ROI detected by it is poor in stability and is not easily interfered by noise. In this paper, ROI detection is performed on the image through a deep convolutional network to obtain a stable ROI based on the high-level feature extraction of the image, and then the discrete cosine transform (DCT) is performed on the image and divided into coding units. According to whether the coding unit is an ROI, To determine the quantization matrix used when encoding it. This article uses fine quantization for coding units that belong to ROI, and coarse quantization for non-ROI coding units. In this way, it can be ensured that the compression rate is greatly reduced without affecting the subjective perception of the image. Experiments show that the compression rate of this method can reach about 84%, and the weighted peak signal-to-noise ratio is improved by about 0. 99dB on average compared with JPEG encoding.

Glossiness-aware Image Coding in JPEG Framework

In images, the representation of glossiness, translucency, and roughness of material objects (Shitsukan) is essential for realistic image reproduction. To date, image coding has been developed considering various indices of the quality of the encoded image, for example, the peak signal-to-noise ratio. Consequently, image coding methods that preserve subjective impressions of qualities such as Shitsukan have not been studied. In this study, the authors focus on the property of glossiness and propose a method of glossiness-aware image coding. Their purpose is to develop an encoding algorithm that produces images that can be decoded by standard JPEG decoders, which are commonly used worldwide. The proposed method consists of three procedures: block classification, glossiness enhancement, and non-glossiness information reduction. In block classification, the types of glossiness in a target image are classified using block units. In glossiness enhancement, the glossiness in each type of block is emphasized to reduce the amount of degradation of glossiness during JPEG encoding. The third procedure, non-glossiness information reduction, further compresses the information while maintaining the glossiness by reducing the information in each block that does not represent the glossiness in the image. To test the effectiveness of the proposed method, the authors conducted a subjective evaluation experiment using paired comparison of images coded by the proposed method and JPEG images with the same data size. The glossiness was found to be better preserved in images coded by the proposed method than in the JPEG images.

Scrubbing-Aware Placement for Reliable FPGA Systems

Field Programmable Gate Arrays (FPGAs) provide complex embedded blocks to ease the development of high-performance computing systems for diverse area applications, including among others space, avionics and health. Although the rich set of features is ever expanding, there is one significant shortcoming of the SRAM-based FPGAs which concerns system designers for applications demanding high reliability: their vulnerability to Single Event Upsets (SEUs) which can cause system malfunction. In this work, we propose a placement approach to improve system reliability by reducing the execution time of configuration memory scrubbing, which can be used in conjunction with other reliability mechanisms proposed in the literature. The proposed placement approach is based on i) an automated floorplanning process to shape and locate the design region(s) and ii) a modified version of the Simulated Annealing placement algorithm aiming to reduce the scrubbing time. First, we performed a set of experiments with three QUIP benchmarks to demonstrate the efficiency of the proposed approach at different device utilization levels. Moreover, we illustrated its effectiveness for three different fault tolerance schemes, where scrubbing plays a different role in each one: i) a TMR microcontroller combined with scrubbing, ii) a soft processor protected by a low-cost mitigation scheme including scrubbing and checkpointing, iii) a JPEG encoder protected by a prioritized scrubbing scheduling scheme based on module criticality levels. The experimental results showed that the proposed approach improves system reliability in all the above schemes by reducing critical timing parameters, such as mean-time-to-detect and mean-time-to-repair. This reduction leads to a modest or high reliability improvement depending on the role of scrubbing in the adopted fault tolerance (FT) scheme.

Visibility of quantization errors in reversible JPEG2000

Image compression systems that exploit the properties of the human visual system have been studied extensively over the past few decades. For the JPEG2000 image compression standard, all previous methods that aim to optimize perceptual quality have considered the irreversible pipeline of the standard. In this work, we propose an approach for the reversible pipeline of the JPEG2000 standard. We introduce a new methodology to measure visibility of quantization errors when reversible color and wavelet transforms are employed. Incorporation of the visibility thresholds using this methodology into a JPEG2000 encoder enables creation of scalable codestreams that can provide both near-threshold and numerically lossless representations, which is desirable in applications where restoration of original image samples is required. Most importantly, this is the first work that quantifies the bitrate penalty incurred by the reversible transforms in near-threshold image compression compared to the irreversible transforms.

Quality controlled ECG data compression based on 2D discrete cosine coefficient filtering and iterative JPEG2000 encoding

This paper presents a quality controlled reconstruction of ECG signal by the formulation of 2D Discrete Cosine Transform (DCT) coefficient and iterative JPEG2000 encoding scheme for Electrocardiogram (ECG) data compression. It employs variable blockwise (L = 32 to 2048 samples) DCT to exploit inter and intra-beat correlation inherent in ECG. The Regula-Falsi method is applied to obtain the optimal threshold for the DCT coefficients, which are stacked row-wise in order to form a 2D DCT coefficient array for the correlation analysis. Afterwards, window-based filtering is applied to retain the significant DCT coefficients with varying window size (ws). Differential pulse code modulation based quantization scheme is then applied to 2D DCT coefficients array in order to make it conformal to JPEG2000 encoding. Finally, iterative JPEG2000 encoding is utilized for achieving the overall predefined reconstruction error. The experimentations were performed on all the 48 records of Massachusetts Institute of Technology-Beth Israel Hospital (MIT-BIH) arrhythmia database. The proposed method provides compression ratio = 39.48 & 40.18; percentage root mean square difference = 2.02 & 2.06 and the quality score = 19.45 & 19.56 at ws = 50% & 20% respectively for L = 256. The proposed technique results were also validated on all records of four standard ECG databases including i) MIT-BIH Normal sinus rhythm ii) CU ventricular tachyarrhythmia iii) CSE and iv) PTB Database. Further, a comparative analysis between the proposed method and some recent existing works was also performed. The results clearly demonstrate the superiority of proposed method.

TOSAM: An Energy-Efficient Truncation- and Rounding-Based Scalable Approximate Multiplier

A scalable approximate multiplier, called truncation- and rounding-based scalable approximate multiplier (TOSAM) is presented, which reduces the number of partial products by truncating each of the input operands based on their leading one-bit position. In the proposed design, multiplication is performed by shift, add, and small fixed-width multiplication operations resulting in large improvements in the energy consumption and area occupation compared to those of the exact multiplier. To improve the total accuracy, input operands of the multiplication part are rounded to the nearest odd number. Because input operands are truncated based on their leading one-bit positions, the accuracy becomes weakly dependent on the width of the input operands and the multiplier becomes scalable. Higher improvements in design parameters (e.g., area and energy consumption) can be achieved as the input operand widths increase. To evaluate the efficiency of the proposed approximate multiplier, its design parameters are compared with those of an exact multiplier and some other recently proposed approximate multipliers. Results reveal that the proposed approximate multiplier with a mean absolute relative error in the range of 11%–0.3% improves delay, area, and energy consumption up to 41%, 90%, and 98%, respectively, compared to those of the exact multiplier. It also outperforms other approximate multipliers in terms of speed, area, and energy consumption. The proposed approximate multiplier has an almost Gaussian error distribution with a near-zero mean value. We exploit it in the structure of a JPEG encoder, sharpening, and classification applications. The results indicate that the quality degradation of the output is negligible. In addition, we suggest an accuracy configurable TOSAM where the energy consumption of the multiplication operation can be adjusted based on the minimum required accuracy.

Design and Analysis of a Neural Network Inference Engine Based on Adaptive Weight Compression

Neural networks generally require significant memory capacity/bandwidth to store/access a large number of synaptic weights. This paper presents design of an energy-efficient neural network inference engine based on adaptive weight compression using a JPEG image encoding algorithm. To maximize compression ratio with minimum accuracy loss, the quality factor of the JPEG encoder is adaptively controlled depending on the accuracy impact of each block. With 1% accuracy loss, the proposed approach achieves $63.4{\times }$ compression for multilayer perceptron (MLP) and $31.3 {\times }$ for LeNet-5 with the MNIST dataset, and $15.3 {\times }$ for AlexNet and $10.2 {\times }$ for ResNet-50 with ImageNet. The reduced memory requirement leads to higher throughput and lower energy for neural network inference ( $3 {\times }$ effective memory bandwidth and $22 {\times }$ lower system energy for MLP).

Fine-Grained Quality Assessment for Compressed Images.

Image quality assessment (IQA) has attracted more and more attention due to the urgent demand in image services. The perceptual-based image compression is one of the most prominent applications that require IQA metrics to be highly correlated with human vision. To explore IQA algorithms that are more consistent with human vision, several calibrated databases have been constructed. However, the distorted images in the existing databases are usually generated by corrupting the pristine images with various distortions in coarse levels, such that the IQA algorithms validated on them may be inefficient to optimize the perceptual-based image compression with fine-grained quality differences. In this paper, we construct a large-scale image database which can be used for fine-grained quality assessment of compressed images. In the proposed database, reference images are compressed at constant bitrate levels by JPEG encoders with different optimization methods. To distinguish subtle differences, the pair-wise comparison method is utilized to rank them in subjective experiments. We select 100 reference images for the proposed database, and each image is compressed into three target bitrates by four different JPEG optimization methods, such that 1200 distorted images are generated in total. Sixteen well-known IQA algorithms are evaluated and analyzed on the proposed database. With the devised fine-grained IQA database, we expect to further promote image quality assessment by shifting it from a coarse-grained stage to a fine-grained stage. The database is available at: https://sites.google.com/site/zhangxinf07/fg-iqa.

A Low Area Fully Pipelined Implementation of JPEG on FPGA

This paper presents a low-area and high-throughput design and implementation of JPEG encoder on FPGA. The design consists of three main components: (1) 2-D DCT module, based on the row-column decomposition technique, (2) Quantization in zigzag ordering, based on look-up tables, and (3) Entropy coder, transforming the quantized DCT coefficients into JPEG words. All components are fully pipelined and optimized for FPGA resource utilization. The proposed implementation of JPEG encoder is able to encode 143 and 71 SDTV frames per second with 720x480 gray scale and color pixels per frame, respectively, on Xilinx Spartan 6 FPGA. Moreover, the proposed architecture is capable of encoding at least 53 and 26 HD Ready TV frames per second with 1280x720 gray scale and color pixels per frame, respectively, on this FPGA chip. Thus, the proposed JPEG encoder architecture is well-suited to various image and video compressing applications where performance and area are significantly important.

Implementation of an 8x8 Discrete Cosine Transform on Programmable System-on-chip

This paper presents experimental results that compares between a full software (SW) implementation and a software/hardware (SW/HW) co-design implementation of a DSP algorithm on a Xilinx Zynq programmable System-on-chip (SoC). The case study being used is the 8x8 two-dimensional discrete cosine transform (2D DCT), present in the popular JPEG and MPEG4-AVC encoder. The full SW design is implemented on a hardcore ARM processor on the FPGA SoC. The SW/HW co-design utilizes both the ARM processor and the Configurable Logic Blocks (CLB) of the FPGA SoC, where the communication channel is implemented using the Xillybus FIFO buffers, implemented as an external DRAM. In this case, the core 2D DCT operations are executed on the CLB, while data initialization and transfers are implemented on the processor. Results show that SW implementation is faster compared to SW/HW implementation for data inputs of less than 0.48 megapixels. As data inputs get larger, SW/HW implementation shows better performance, with up to 2x faster for 2 megapixels data input size. This study proves the viability of implementing the 2D DCT operations as dedicated hardware accelerator in multimedia encoders.

Image acquisition system using on sensor compressed sampling technique

Advances in CMOS technology have made high resolution image sensors possible. These image sensor pose significant challenges in terms of the amount of raw data generated, energy efficiency and frame rate. This paper presents a new design methodology for an imaging system and a simplified novel image sensor pixel design to be used in such system so that Compressed Sensing (CS) technique can be implemented easily at the sensor level. This results in significant energy savings as it not only cuts the raw data rate but also reduces transistor count per pixel, decreases pixel size, increases fill factor, simplifies ADC, JPEG encoder and JPEG decoder design and decreases wiring as well as address decoder size by half. Thus CS has the potential to increase the resolution of image sensors for a given technology and die size while significantly decreasing the power consumption and design complexity. We show that it has potential to reduce power consumption by about 23%-65%.

Improvement of JPEG XT Floating-Point HDR Image Coding Using Region Adaptive Prediction

Displays capable of representing high-dynamic range (HDR) images were recently released in the digital consumer electronics market, and consumers have become increasingly interested in HDR images. However, the majority of today’s digital imaging devices are still low dynamic range (LDR) using an 8-bit representation for each RGB color components; hence, a backward-compatible HDR image format with an existing LDR-based imaging environment is required. Accordingly, JPEG XT, a new HDR image coding standard providing JPEG backward compatibility was established by the JPEG committee based on the market needs. JPEG XT consists of a base and a residual layer. Both layers are independently encoded by a legacy JPEG encoder. Most importantly, the base layer contains a tone-mapped LDR version of the HDR image for compliance with a legacy JPEG decoder. To date, three profiles have been defined for JPEG XT to encode HDR images, which are represented using floating-point values. This paper proposes a new image-coding scheme that uses a region adaptive prediction method with modified current specifications on JPEG XT Part 7. The proposed method adaptively predicts the HDR pixel value using different prediction information for each block based on the ratio between the LDR and HDR blocks, which is invariant to the tone-mapping operator, while the existing JPEG XT profiles globally predict the HDR values using an inverse transformation of the LDR values. Experimental results using various sample images show the superiority of the proposed method in terms of objective and visual evaluations.