Consumer Electronics Systems Research Articles

Enhanced Security of DSP Circuits Using Multi-Key Based Structural Obfuscation and Physical-Level Watermarking for Consumer Electronics Systems

Security of Digital Signal Processing (DSP) based Intellectual Property (IP) core is very important for the overall security and reliability of Consumer Electronics (CE) systems. These DSP based IP cores are highly vulnerable to threats such as reverse engineering (RE) leading to Trojan insertion, counterfeiting and cloning. Thus developing robust defense mechanisms is highly significant for these DSP circuits used in CE systems, especially to thwart the aforesaid threats. This article presents for the first time a double line of defense mechanism using robust multi-key based structural obfuscation integrated with tamper-tolerant physical level watermarking to counter such hardware threats. The proposed approach introduces novel key-driven partitioning based obfuscation and key-driven folding knob based obfuscation, along with other key-driven transformation based obfuscations. Additionally, the proposed approach presents novel multi-variable signature encoding based physical-level watermarking on key-based structurally obfuscated DSP circuits. The presented double line of defense incurs zero overhead as well yields high tamper tolerance during hardware watermarking. The proposed approach yields stronger strength of obfuscation, lower probability of co-incidence and larger key-space at low design cost against attacks.

IP Core Steganography for Protecting DSP Kernels Used in CE Systems

Intellectual Property (IP) core protection of Digital Signal Processing (DSP) kernels is an important subject of research for Consumer Electronics (CE) systems. An IP core may be prone to piracy, forgery and counterfeiting. The need of the hour is developing effective technique that is robust and incurs low overhead to detect IP core infringement. This paper presents a novel ‘IP core steganography’ methodology for DSP kernels that is capable of detecting IP piracy. The proposed methodology is capable of implanting concealed information into the existing IP core design of DSP datapath without using any external signature, to reflect the IP core ownership. The presented ‘IP core steganography’ methodology is non-intuitive in nature indicating that the intended secret information does not attract attention to itself from an adversary’s perspective. The implanted information incurs almost no design overhead and yields lower design cost than signature-based IP core protection techniques. Further, in the presented approach the amount of concealed information embedded is fully designer controlled through a ‘thresholding’ parameter, unlike signature-based techniques where signature pattern impacts the robustness and overhead. Results of proposed approach yielded lower cost and stronger proof of authorship compared to a signature-based approach.

Embedding Digital Signature Using Encrypted-Hashing for Protection of DSP Cores in CE

Reusable intellectual property (IP) cores from signal processing and multimedia form an integral part of consumer electronics (CE) systems. However, owing to the value it represents, it needs protection against important threats, such as piracy and illegal claim of ownership. This paper presents a novel multi-level encoding and encrypted-hash-based digital signature for protection of complex reusable IP core used in CE systems. For the first time in the literature, a digital signature-based IP core protection is proposed for digital signal processing (DSP) kernels. The proposed methodology is capable of encoding a DSP kernel application, followed by creating a digest of the encoded application to finally form a digital signature using RSA, which is subsequently implanted into the same design during architectural synthesis. The proposed approach on DSP benchmarks is capable of achieving higher robustness in terms of lower probability of coincidence - indicating stronger proof of authorship. The proposed approach achieves stronger robustness (on average by ~24.8%) as well as requires lesser storage hardware (on average reduction of ~14%) compared to similar prior work.

Impact of hardware steganography on DSP core datapath

The protection of Intellectual Property (IP) core for a Digital Signal Processing (DSP) kernel has become an important research area in the last few years for Consumer Electronics systems. The swelling threats of IP piracy, forgery, and counterfeiting and ownership abuse on DSP IP core are the principal reason behind that. This work investigates the impact of hardware steganography on protection of DSP IP core datapath against these threats. As a case study, the complete analysis of its impact on 8-point Discrete Cosine Transformation datapath architecture is presented.

Convolutional Neural Network Implementation for Eye-Gaze Estimation on Low-Quality Consumer Imaging Systems

Accurate and efficient eye gaze estimation is important for emerging consumer electronic systems, such as driver monitoring systems and novel user interfaces. Such systems are required to operate reliably in difficult, unconstrained environments with low power consumption and at minimal cost. In this paper, a new hardware friendly, convolutional neural network (CNN) model with minimal computational requirements is introduced and assessed for efficient appearance-based gaze estimation. The model is tested and compared against existing appearance-based CNN approaches, achieving better eye gaze accuracy with significantly fewer computational requirements.

A Generic Standalone Design Approach to Embedded Hardware Development

Hardware design in an Electronic System refers to a holistic design incorporating the basic schematic, the board layout, routing,fabrication, and testing. Present industry trends use virtual design methodologies to construct one or more printed circuit boards(PCB’s) which implement interconnections between components and are manufactured in bulk for multiple applications in Consumer Electronics, Automobiles, Computers and Entertainment Systems. They even find application is a diverse spectrum ofmicro to mega-scale of electronic products. This paper presents a novel approach to design a proprietary standalone controller card. Hardware Design integrates custom Schematic, Board layout, and manual track routing. The Standalone design technique provides a technique to physically connect microcontrollers to a master device for the sole purpose of firmware upload. The Two fold approach refers to a standard OEP (Original Entry Point) based design juxtaposed with an optional DE (Dual Entry Design) for an additional Proprietary entry point included to serve as an irretrievable secure secondary dumping mechanism. This offers flexibility to upload firmware securely as an extension to the prevalent OEP technique. The final design integrates the PCB’s functionality of interconnected components with accurate placement and feasible dimensions for large-scale production.

Robust Logic locking for Securing Reusable DSP Cores

A System on Chip (SoC) used in Consumer Electronics (CE) systems integrates a number of reusable Intellectual Property (IP) cores from digital signal processing (DSP), multimedia etc. However, these DSP based IP cores are susceptible to various hardware threats such as piracy, Trojan insertion, overbuilding and reverse engineering. Thus, security of DSP cores is very crucial. An IP core can be secured against aforementioned hardware threats by employing logic locking based security mechanisms. This paper presents a novel robust logic locking using hybrid locking cells for securing DSP cores. The proposed logic locking is based on a novel advanced encryption standard (AES) based reconfigurable hybrid locking cell architecture that ensures strong security against key sensitization, removal and SAT attacks. The strength of the proposed approach has been assessed in terms of probability of obtaining correct key of a locked design in exhaustive trials. Results of proposed work on DSP cores yielded higher logic locking strength and lower design overhead compared to recent prior works.

Handling Cell Components in the Production of Multi-Layered Large Format All-Solid-State Batteries with Lithium Anode

All-solid-state batteries (ASSB) are considered as promising energy storage systems for consumer electronics and electric mobility because of their high safety, long life and high energy density. Currently, electrochemistry and materials research attempt to produce ASSB cells on laboratory scale. One future challenge will be to characterize the required manufacturing processes for large format composites of thin ceramic layers used in ASSB and to transfer this knowledge to pilot scale production. The promising technology is still far from being commercially producible on an industrial scale, which is why there is a particular need for extensive research on the manufacturability of large format and stackable sheet type ASSB cells. In this paper we present a concept for handling cell components during cell assembly of ASSB by analyzing extensively the material and process requirements, as well as interdependencies with simultaneous engineering. Furthermore, the possibilities for cell stacking are investigated via SIPOC by appropriately adapting techniques of conventional lithium-ion batteries to the varying requirements of ASSB.

Effect of stand-off height on the shear strength of ball grid array solder joints under varying pad sizes

The solder joints of ball grid array utilized in consumer electronics systems or assemblies degrade and fail overtime. Their degree of degradation is more critical, especially at elevated temperatures and mechanical loading conditions. This study presents the effect of component standoff height (CSH) on the shear strength reliability of ball grid array solder joints under different pad sizes. Investigation of the impact of standoff height on the mechanical reliability of the solder joint of ball grid array components under different pad sizes was conducted in this work. Isothermal ageing of test samples were conducted at 150 °C for 8 days. This study focuses on establishing the relationship between CSH and shear strength of the solder joints under different pad sizes and the corresponding effect of prolonged elevated temperature conditions on the mechanical integrity of the soldered joints. The work also identifies the failure mode and examines the region of the failed joints and surfaces to provide information on the morphological characteristics of the material microstructure. The results of this study demonstrate that the smallest pad size (19 mil) gave the lowest shear strength of 61.08 MPa with a high standoff height of 0.25 mm as compared to the largest pad size (24 mil) with the highest shear strength of 70.43 MPa having a relatively low standoff height of 0.22 mm.

Metasurface eyepiece for augmented reality

Recently, metasurfaces composed of artificially fabricated subwavelength structures have shown remarkable potential for the manipulation of light with unprecedented functionality. Here, we first demonstrate a metasurface application to realize a compact near-eye display system for augmented reality with a wide field of view. A key component is a see-through metalens with an anisotropic response, a high numerical aperture with a large aperture, and broadband characteristics. By virtue of these high-performance features, the metalens can overcome the existing bottleneck imposed by the narrow field of view and bulkiness of current systems, which hinders their usability and further development. Experimental demonstrations with a nanoimprinted large-area see-through metalens are reported, showing full-color imaging with a wide field of view and feasibility of mass production. This work on novel metasurface applications shows great potential for the development of optical display systems for future consumer electronics and computer vision applications.

Audio and Video Technologies: Recent Advances in Consumer Electronics

Our everyday lives are crammed with consumer electronics (CE) systems, ranging from smartphones, smart devices, set-top boxes, video consoles, and tablets to washing machines and kitchen appliances. A CE device is heavily reliant on audio and video technologies for its successful smart functioning. With the evolution of these sophisticated devices and technologies, data (such as images, sensitive information, and so forth) security has become a major concern that requires immediate attention. Another facet of these technologies that needs attention is optimization of data redundancy, coverage area, and energy expenditure of CE systems, such as video devices. This special section aims to publish novel solutions with respect to these aspects of audio and video technologies for CE applications.

An Overview of Regulation Topologies in Resonant Wireless Power Transfer Systems for Consumer Electronics or Bio-Implants

Owing to its relatively high efficiency, extended transmission range, and less exposure to radio frequency radiation, near-field resonant wireless power transfer (R-WPT) has been widely used in consumer electronics and bio-implants. For most applications, a well-regulated output voltage is required against the coupling and loading variations, and thus a regulation scheme should be employed in an R-WPT system. To achieve an optimal receiver (RX) or overall efficiency, together with a reduced cost overhead, several regulation schemes have been proposed in recent years, where the regulation can be implemented at either the RX or transmitter (TX) side, or both. These regulation schemes have been reviewed and comprehensively discussed in this paper. Hence, the main contribution of this paper is to provide a guideline for designing the regulation scheme in R-WPT systems. Moreover, potential new topologies of regulation are investigated here.

Stereo vision architecture for heterogeneous systems-on-chip

Stereo vision is a crucial operation in state-of-the-art computer vision applications. Several efficient algorithms have been recently proposed either to increase the quality of the produced disparity maps or to reach higher computational speed, or both. Among them, hardware-oriented algorithms are desirable when the main objective is including stereo vision in portable consumer electronic and multimedia systems. Modern FPGA-based platforms allow all-programmable heterogeneous embedded systems to be realized as SoCs and they are certainly appropriate also to implement stereo vision. This paper proposes a novel stereo vision algorithm and a specific implementation suitable for heterogeneous SoC FPGA-based embedded systems. A complete embedded system design is also demonstrated using the Xilinx Zynq-7000 SoCs device family. The novel algorithm has been characterized in terms of accuracy by referring to the benchmark sets Middlebury and Kitti. When 640 × 480 8-bit greyscale stereo pairs are processed, the fastest prototype, realized within the XC7Z020 and the XC7Z045 device, respectively, exhibits an 81 and 101 fps frame rate. Conversely, the cheapest implementation occupies only 52691 LUTs, 59715 FFs, 93 DSPs, 40 BRAM18k and 6 BRAM36k memory blocks. In comparison to several counterparts existing in literature, the novel algorithm can achieve higher accuracies, and makes the proposed complete system design able to reach the most favourable accuracy/performance trade-off.

Performance Improvement of Flexible Charge-Trap Memory Transistors Using Conducting Polymer Electrodes and Sacrificial Layer on Plastic Poly(ethylene naphthalate) Substrates

Flexible and wearable electronic systems implemented on the plastic substrates have been energetically researched. Nonvolatile memory device (NVMs) with mechanical flexibility is one of the most important device elements to realize next-generation consumer electronic systems, because the NVMs can be embedded for both functions of an information storage as well as a power saving. In this purpose, we could successfully fabricated the charge-trap memory thin film transistors (CTMs) on poly(ethylene naphthalate) (PEN) substrates. Although good electrical and mechanical bending characteristics could be verified,1) for practical uses of flexible electronic systems, higher device performances are quite demanding even under harsh bending situations. However, there were intrinsic limitation at bending conditions with smaller curvature radius, because the brittle indium-tin-oxide (ITO) was employed as source/drain (S/D) electrodes for the previously demonstrated devices. Therefore, in order to significantly improve the bending performance of CTMs, brittle oxide-based electrodes should be replaced with a material having inherent flexibility, such as conducting polymer. From these backgrounds, poly(3,4-ethylenedioxythiophene)-poly(styrenesulfonate) (PEDOT:PSS), which is one of the most widely used conducting polymers, was introduced for the oxide-semiconductor-based CTMs as flexible S/D electrodes. This approach can be quite promising from the fact that the conducting polymer electrodes with mechanical flexibility and the oxide semiconductor channels with excellent electrical properties can be simultaneously exploited. However, to realize highly-functional CTM devices combined with electrical and mechanical merits, following points should be considered: (1) high electrical conductivity, (2) lithography process compatibility of conducting polymers, and (3) overall TFT-process compatibility. From these viewpoints, H2SO4-treated PEDOT:PSS films were prepared and transferred on PEN substrates, by which the PEN surfaces could be effectively protected from harsh acidic condition. In preliminary experiments, the PEDOT:PSS film was transferred on SA7 (3-µm-thick, synthesized by Dongjin Semichem Co. Ltd.)-treated PEN substrate, which is organic barrier for the surface planarization.2) Even though an inorganic barrier (Al2O3) was desirable to be embedded for improving the water vapor transmission rate of the PEN substrate, in this work, Al2O3 barrier was eliminated so as to secure the lithography compatibility because of the poor adhesion between Al2O3 and PEDOT:PSS films. With this approach, the CTMs were successfully fabricated by using PEDOT:PSS S/D electrode, amorphous In-Ga-Zn-O (a-IGZO) active, and ZnO charge-trap layers. Clockwise hysteresis attributed to the charge-trap mechanism was obtained and memory windows (MWs) were soundly modulated by changing the gate voltage (VGS) sweep range. However, compared to the devices using ITO S/D’s, the CTMs fabricated with PEDOT:PSS S/D electrodes exhibited deteriorated subthreshold swing (SS) and asymmetric memory hysteresis, which may inevitably induce higher power consumption and slower operating speed. These inferior device characteristics were examined to be caused by terribly-rough back-channel region, because organic barrier of SA7 was severely damaged during dry etching patterning process for the PEDOT:PSS S/D electrodes. In order to solve these problems, SA7 thin film was inserted between the organic/inorganic barrier and PEDOT:PSS film as a sacrificial layer, which should be chosen to satisfy following two requirements: (1) the thickness of the sacrificial layer should be sufficiently thin not more than 100 nm so that it can be completely etched during the S/D patterning process and the inorganic barrier can be faced with active channel as a back channel. (2) Sufficient adhesive properties with the PEDOT:PSS should be obtained even with a thinner film thickness. The film thickness and formation conditions of proposed sacrificial layer could be optimized for guaranteeing the smooth back channel and good adhesion with PEDOT:PSS film. The PEDOT:PSS could be transferred and patterned on sacrificial layer and fully compatible with CTM fabrication process. As results, smaller SS value without marked gate leakage current and symmetric MWs with higher memory on/off ratio could be accomplished for the CTM devices fabricated by introducing sacrificial layer. The detailed descriptions on the flexible memory characteristics will be extensively discussed at presentation. Although there remains room for the improvement of device characteristics, the introduction of suitable sacrificial layer was an effective remedy for improving the back-channel properties. This strategy proposed in this work can provide good chance to realize highly-functional and fully-bendable nonvolatile memory devices for flexible consumer electronics. 1) S. J. Kim, M. J. Park, D. J. Yun, W. H. Lee, G. H. Kim and S. M. Yoon, IEEE Trans. Electron Devices vol. 63, no. 4, 1557 (2016) 2) M. J. Park, D. J. Yun, M. K. Ryu, J. H. Yang, J. E. Pi, O. S. Kwon, G. H. Kim, C. S. Hwang, J. Y. Bak and S. M. Yoon, J. Mater. Chem. C, vol. 3, no. 18, 4779 (2015)

Pushing the AI Envelope: Merging Deep Networks to Accelerate Edge Artificial Intelligence in Consumer Electronics Devices and Systems

Deep neural networks (DNNs) are widely used by both academic and industry researchers to solve many long-standing problems in machine learning. There has been such a growth of research in this field, and it has been applied to so many varying problems, that it would be accurate to say that we may be living through the precursor of the singularity [1]. But regardless of one's views on artificial intelligence (AI), there is no doubt that there is a wealth of recent research that leverages the use of various DNNs to solve a broad range of pattern recognition and classification problems. Examples range from the introduction of smart speakers with intelligent assistants to the application of DNNs to solve recalcitrant problems in computer vision for autonomous vehicles. Many of these problems can have very useful applications in the design of smarter consumer electronics (CE) systems and devices. The question for CE engineers is how to leverage this wealth of academic and industry research efforts, turning them into practical DNN solutions suitable for deployment in practical devices and electronic systems.

Pupil response to quality and content transitions in videos

Real-time human eye recognition and tracking systems with human-computer interaction mechanism are being adopted to advance user experience in smart devices and consumer electronic systems. Eye tracking systems measure eye gaze and pupil response non-intrusively. This paper presents an analysis of pupil response to video structure and content. A consumer device that can assess user response to a video can provide better experiences with approaches such as real-time content adaptation. The first set of experiments involved presenting different video content to subjects and measuring eye response with an eye tracker. Results show that pupil constrictions and magnitude of constrictions vary with content. Significant changes in video and scene cuts led to sharp constrictions. User response to videos can provide insights that can improve subjective quality assessment metrics. This paper also presents an analysis of the pupillary response to quality changes in videos with the second set of experiments. The results of the tests show pupil constrictions for noticeable changes in perceived quality. Using real-time eye tracking systems for video analysis and quality evaluation can open a new class of applications for consumer electronic systems.

A Low-Cost Real-Time Embedded Stereo Vision System for Accurate Disparity Estimation Based on Guided Image Filtering

Stereo matching, a key element towards extracting depth information from stereo images, is widely used in several embedded consumer electronic and multimedia systems. Such systems demand high processing performance and accurate depth perception, while their deployment in embedded and mobile environments implies that cost, energy and memory overheads need to be minimized. Hardware acceleration has been demonstrated in efficient embedded stereo vision systems. To this end, this paper presents the design and implementation of a hardware-based stereo matching system able to provide high accuracy and concurrently high performance for embedded vision devices, which are associated with limited hardware and power budget. We first implemented a compact and efficient design of the guided image filter, an edge-preserving filter, which reduces the hardware complexity of the implemented stereo algorithm, while at the same time maintains high-quality results. The guided filter design is used in two parts of the stereo matching pipeline, showing that it can simplify the hardware complexity of the Adaptive Support Weight aggregation step, and efficiently enable a powerful disparity refinement unit, which improves matching accuracy, even though cost aggregation is based on simple, fixed support strategies. We implemented several variants of our design on a Kintex-7 FPGA board, which was able to process HD video (1,280 × 720) in real-time (60 fps), using ∼57.5k and ∼71k of the FPGA's logic (CLB) and register resources, respectively. Additionally, the proposed stereo matching design delivers leading accuracy when compared to state-of-the-art hardware implementations based on the Middlebury evaluation metrics (at least 1.5 percent less bad matching pixels).

Scalable and efficient configuration of time-division multiplexed resources

Consumer-electronics systems are becoming increasingly complex as the number of integrated applications is growing. Some of these applications have real-time requirements, while other non-real-time applications only require good average performance. For cost-efficient design, contemporary platforms feature an increasing number of cores that share resources, such as memories and interconnects. However, resource sharing causes contention that must be resolved by a resource arbiter, such as Time-Division Multiplexing. A key challenge is to configure this arbiter to satisfy the bandwidth and latency requirements of the real-time applications, while maximizing the slack capacity to improve performance of their non-real-time counterparts. As this configuration problem is NP-hard, a sophisticated automated configuration method is required to avoid negatively impacting design time.The main contributions of this article are: (1) an optimal approach that takes an existing integer linear programming (ILP) model addressing the problem and wraps it in a branch-and-price framework to improve scalability. (2) A faster heuristic algorithm that typically provides near-optimal solutions. (3) An experimental evaluation that quantitatively compares the branch-and-price approach to the previously formulated ILP model and the proposed heuristic. (4) A case study of an HD video and graphics processing system that demonstrates the practical applicability of the approach.

Research highlights: translating chips.

Microfluidic and microfabricated systems are providing key functionalities in diagnostic and therapeutic scenarios, translating beyond the research laboratory to pre-clinical animal studies and clinical studies with patients. Here, we highlight a recent study making use of miniaturization and automation in the development of a smartphone-integrated point-of-care diagnostic to detect antibodies to infectious diseases in a global health setting. We also review an intraocular implanted diagnostic system for glaucoma that relies on imaging the location of a fluid meniscus in a microchannel to readout pressure within the eye. Developments in low-cost and highly functional consumer electronic systems (e.g. smartphones in both highlighted works) has led to a continuing trend to incorporate such technologies with microfluidic fluid handling capabilities to achieve complete diagnostic solutions. We conclude with another implanted microdevice that delivers drug locally to tumors through electroosmotic flow and electromigration of charged drug species, which allows high drug concentrations near a tumor or resected tumor site while preventing high systemic levels associated with significant side-effects. The maturity of microsystem components are now allowing integration into fully functional systems that are poised to reach the clinic in a variety of forms - diagnostic to therapeutic.

Patenting Complexity [IP Corner

One of the hallmarks of consumer electronics (CE) products and systems is that, while the product concept is often simple, these products themselves are never simple in their engineering aspects. This short article has only covered three of a much larger number of patents related to the CD drive, and the full engineering details are far more complex and intertwined. But this does serve as a useful example illustrating the complexity and holistic nature of many consumer electronics products.