Mobile Display Research Articles

Effects of different non-driving-related-task display modes on drivers’ eye-movement patterns during take-over in an automated vehicle

Highly automated vehicles relieve drivers from driving tasks, allowing them to engage in non-driving-related-tasks (NDRTs). However, drivers are required to take over control in certain circumstances due to the limitations of highly automated vehicles. This study focused on drivers’ eye-movement patterns during take-overs when an NDRT (watching videos) was presented via a head-up-display (HUD) and a mobile device display (MDD), compared to no NDRT as the baseline. The experiment was conducted in a high-fidelity driving simulator with real-world driving videos scenarios. Forty-six participants took part in the experiment by completing three drives in three counterbalanced conditions (HUD, MDD and baseline). A take-over-request was issued towards the end of automated driving requesting drivers to stop the NDRT and take over control. Eye-movement data including pupil diameter, blinks, glance duration and number of AOI (Area of Interest) were collected and analysed. The results show that during automated driving, drivers were more engaged in the MDD NDRT with smaller pupil diameter and shorter glance duration on the front scenario compared to the HUD and baseline modes. The number of AOI was reduced during automated driving in both MDD and HUD modes. The take-over-request redirected drivers’ visual attention back to the driving task from NDRT by increasing drivers’ pupil diameter, glance duration and number of AOI. However, the effect of MDD NDRT on pupil diameter and glance duration continued even after the take-over-request when the NDRT was terminated. The study demonstrated HUD is a better display to help maintain drivers’ attention on the road.

Personalized Robot Interventions for Autistic Children: An Automated Methodology for Attention Assessment

We propose a robot-mediated therapy and assessment system for children with autism spectrum disorder (ASD) of mild to moderate severity and minimal verbal capabilities. The objectives of the robot interaction sessions is to improve the academic capabilities of ASD patients by increasing the length and the quality of their attention. The system uses a NAO robot and an added mobile display to present emotional cues and solicit appropriate emotional responses. The interaction is semi-autonomous with minimal human intervention. Interaction occurs within an adaptive dynamic scenario composed of 13 sections. The scenario allows adaptive customization based on the attention score history of each patient. The attention score is autonomously generated by the system and depends on face attention and joint attention cues and sound responses. The scoring system allows us to prove that the customized interaction system increases the engagement and attention capabilities of ASD patients. After performing a pilot study, involving 6 ASD children, out of a total of 11 considered in the clinical setup, we conducted a long-term study. This study empirically proves that the proposed assessment system represents the attention state of the patient with 82.4% accuracy.

EarVR: Using Ear Haptics in Virtual Reality for Deaf and Hard-of-Hearing People.

Virtual Reality (VR) has a great potential to improve skills of Deaf and Hard-of-Hearing (DHH) people. Most VR applications and devices are designed for persons without hearing problems. Therefore, DHH persons have many limitations when using VR. Adding special features in a VR environment, such as subtitles, or haptic devices will help them. Previously, it was necessary to design a special VR environment for DHH persons. We introduce and evaluate a new prototype called "EarVR" that can be mounted on any desktop or mobile VR Head-Mounted Display (HMD). EarVR analyzes 3D sounds in a VR environment and locates the direction of the sound source that is closest to a user. It notifies the user about the sound direction using two vibro-motors placed on the user's ears. EarVR helps DHH persons to complete sound-based VR tasks in any VR application with 3D audio and a mute option for background music. Therefore, DHH persons can use all VR applications with 3D audio, not only those applications designed for them. Our user study shows that DHH participants were able to complete a simple VR task significantly faster with EarVR than without. The completion time of DHH participants was very close to participants without hearing problems. Also, it shows that DHH participants were able to finish a complex VR task with EarVR, while without it, they could not finish the task even once. Finally, our qualitative and quantitative evaluation among DHH participants indicates that they preferred to use EarVR and it encouraged them to use VR technology more.

Shock absorption of semi-interpenetrating network acrylic pressure-sensitive adhesive for mobile display impact resistance

With the development of mobile display shapes from flat to flexible, the conventional glass cover window must be replaced by a film-type window. However, film-type cover windows have low impact resistance owing to their poor shock absorption. Therefore, to realize their application to flexible displays, it is essential to characterize and improve the shock-absorbing characteristics of the display film. In this study, a semi-interpenetrating network (semi-IPN) acrylic pressure-sensitive adhesive (PSA) consisting of a crosslinked network structure with a linear acrylic polymer chain was developed. The influence of the multi-functional monomer content and UV dose on the properties of the semi-IPN PSA was studied. The peel strength of the semi-IPN PSA was higher than 10 N/25 mm, which is generally considered as the PSA standard for mobile displays. With an increase in crosslink density, the storage modulus increased, but the damping factor decreased. To evaluate shock absorption, a falling ball impact tester was used to estimate the shock absorption ratio (ΔF) of the film. The semi-IPN PSA incorporating 30 phr of the multi-functional monomer exhibited higher ΔF (31.2%) than that of a conventional UV-curable PSA film (24.6%). This shows that even with the relatively low damping factor of the semi-IPN PSA, its structure could improve shock absorption. Moreover, we suggested two mechanisms for shock absorption of the semi-IPN PSA, energy dissipation into the PSA layer and extended contact time.

Pressure Measurement-Based Method for Battery-Free Food Monitoring Powered by NFC Energy Harvesting.

A novel approach for battery-free food freshness monitoring is proposed and demonstrated in this study. The aim is to track the freshness of different sorts of food such as pork, chicken, and fish during storage. To eliminate the drawbacks of conventional food monitoring methods, which are normally based on measuring gas concentration emitted from food in a container, this approach measures the gradual increase in air pressure caused by the gas emission during storage. Additionally, we aim to design a smart sensor tag that can operate in fully passive mode without an external power source. To achieve this goal, near-field communication (NFC)-based energy harvesting is utilized in this work to achieve a self-powered operation of the sensor tag. To demonstrate the feasibility of the proposed method, experiments with the above-mentioned food were tested at room and refrigerated temperatures in 2 and 4 days, respectively. For each experiment, 200 g of the target food was placed in a 2-L container with the smart sensor tag. The experiments were conducted with both rigid and flexible containers to consider real food packaging environments. The air pressure inside the container was monitored as an indicator of food freshness by a sensitive pressure sensor on the smart sensor tag. The experimental results showed a remarkable increase in air pressure, which was able to be detected with high accuracy by the pressure sensor. The fabricated battery-free smart sensor tag is small (2.5 cm × 2.5 cm) and is capable of less than 1 mW of power consumption, which is ultra-low relative to other ordinary approaches that have a power consumption that normally surpasses 10 mW. The pressure value was used to classify food freshness into different levels on a mobile display to provide food freshness status using an NFC-enabled smartphone.

Illuminant Estimation Through Reverse Calibration of an Auto White-Balanced Image That Contains Displays

This study proposes an illuminant estimation method that reproduces the original illuminant of a scene using a mobile display as a target. The original lighting environment of an auto white-balancing (AWB) photograph is obtained through reverse calibration, using the white point of a display in the photograph. This reproduces the photograph before AWB processed, and we can obtain the illuminant information using Gray World computation. The study consists of two sessions. In Session 1, we measured the display's white points under varying illuminants to prove that display colors show limited changes under any light conditions. Then, in Session 2, we generated the estimations and assessed the performance of display-based illuminant estimation by comparing the result with the optically measured values in the real situation. Overall, the proposed method is a satisfactory way to estimate the less chromatic illuminants under 6300 K that we experience as indoor light in our daily lives.

P‐13.13: The Technology of Narrow Border in Mobile Display

Thanks to new display technology development, mobile phone has been evolving from early analog phone to current smart phone. Most noticeable evidences include display evolution from digital to liquid crystal (LCD), and to more recent active‐matrix organic light‐emitting diode (AMOLED), color advancement from monochromic to full‐color, and display panel size from small‐screen to large‐screen. Besides, full‐screen has also become a new trend in mobile phone market since 2017. To meet the full‐screen display and/or to achieve the high screen‐to‐body ratio, display panel makers have come out a variety of screen design, notch, dew‐drop, hole‐punch, etc. Equally important, narrow border is another key technical requirement that will help increase the screen‐to‐body ratio, ultimately, enhancing visual experience. Technically, screen‐to‐body ratio of 90% or higher can be considered as a full‐screen. In order to meet this full‐screen display, it is necessary to reduce the bottom‐border from 4 mm to 1 mm, and leftand right‐borders from 1.2 mm to 0.5 mm, simultaneously. However, narrow border should be compatible with ESD, De‐mux circuits. This would require a width of about 1 mm in the FANOUT area. In addition, since the upper/left/right‐borders are only 0.5 mm or narrower, frit sealing may have only about 300 um in width, putting panel reliability in a great risk.In this paper, we will describe our technology route that not only can meet narrow border specification in the AMOLED display, but also can maintain the panel stability.

11‐2: Micro LEDs Efficiency Targets for Displays

Micro Light emitting diode (µLED) emissive displays have emerged as promising solution for low power mobile displays. In this work, we discuss the prospects of µLED displays for mobile display applications through physics‐based simulations. Our aim is to assess the possible achievable performance of µLED displays for low power applications. We compare simulated power consumption of µLED displays to experimental data obtained on state‐of‐the‐art mobile devices with OLED displays to determine the minimum required µLED efficiencies to achieve at least 2× lower emitter power. Surprisingly, it is found that, in comparison to state‐of‐the‐art OLED displays, µLED displays can exhibit two times lower emitter power consumption using modest values of µLED power efficacies for red (~10 lm/W), green (~100 lm/W) and blue (~10 lm/W) µLEDs. The simulation results show that µLED displays are a promising candidate for ultralow power mobile displays that are currently being heavily investigated by the industry.

67‐2: 5.5 inch Full Screen Flexible High‐Resolution OLED Display Fabricated by Ink Jet Printing Method

A 5.5 inch full high definition (FHD, 1920 × 1080) printed flexible OLED display (high resolution up to 402ppi) was fabricated with inkjet printing technology. Special sub‐pixel arrangement of Green‐Green‐Blue (GGRB) was employed to show the potential of inkjet printed top‐emission OLED in mobile display application. The electro‐luminance performance of both bottom‐emission and top‐emission printed OLED was investigated.

P‐17: IGZO TFT Gate Driver with Independent both Bootstrapping and Control Units for AMOLED Mobile Display

An IGZO TFTs integrated gate driver with an independent bootstrapping structure and an independent control unit is proposed for AMOLED display. The driver can output both programmable multi‐pulse scan signals and normally‐high‐voltage emission control signals. Compared with the traditional bootstrapping node structure with a long rising time of 1.06 µs, and the diode‐connected bootstrapping node structure with a long falling time of 1.60 µs, the proposed driver has a short and symmetric rising and falling time of about 0.64 µs. Cascaded simulation results show that the driver may be applicable to 5.5‐in full HD AMOLED panel.

Regularized Auto-Encoder-Based Separation of Defects from Backgrounds for Inspecting Display Devices

We investigated a novel method for separating defects from the background for inspecting display devices. Separation of defects has important applications such as determining whether the detected defects are truly defective and the quantification of the degree of defectiveness. Although many studies on estimating patterned background have been conducted, the existing studies are mainly based on the approach of approximation by low-rank matrices. Because the conventional methods face problems such as imperfect reconstruction and difficulty of selecting the bases for low-rank approximation, we have studied a deep-learning-based foreground reconstruction method that is based on the auto-encoder structure with a regression layer for the output. In the experimental studies carried out using mobile display panels, the proposed method showed significantly improved performance compared to the existing singular value decomposition method. We believe that the proposed method could be useful not only for inspecting display devices but also for many applications that involve the detection of defects in the presence of a textured background.

Tiled Stereoscopic 3D Display Wall - Concept, Applications and Evaluation

The Tiled Stereoscopic 3D Display Wall (TS3DW) is a monitor system consisting of six consumer 3D TVs. Two monitors reside on a mobile display mount. One standard configuration is to use them in a 135-degree angle to each other, having one mobile mount in the center, and one at each side. In this way, the system can be transported to multiple locations across a campus as well as used in different application scenarios. This system was already used for a number of research projects and presentations.

3D Object Manipulation Techniques in Handheld Mobile Augmented Reality Interface: A Review

Handheld mobile devices referring to mobile phones and tablets become the major output medium for augmented reality (AR), which have seen significant growth in popularity and usage among the public due to the growing release of consumer-oriented communication products nowadays, especially touchscreen smartphones. Unlike traditional desktop or tabletop AR (large display-based) and head-mounted display-based AR systems, small display-based AR (handheld mobile display) requires different interaction techniques that mostly utilize single-hand interaction as well as the limitation of small screen display and limited activity time due to the battery operation hour of handheld mobile devices. However, in handheld mobile AR, research is still lacking, especially research that focuses on 3D interaction in handheld mobile AR for virtual object manipulation. Thus, this paper provides an overview of 3D interaction techniques in handheld mobile AR with critical analysis. First, we describe three main interaction technique categories that applicable in handheld mobile AR, which is touch-based interaction, mid-air gestures-based interaction, and device-based interaction techniques, of their basic concepts on 3D object manipulation. Then, we classify and systematize the highlighted techniques and discuss the advantages and drawbacks of each. Previous studies for widely used techniques have been studied comprehensively. We then draw up a comparison among the different techniques based on the important elements considered in handheld mobile AR. The aim of this paper is to provide researchers with background information on AR and those who are interested in the field of 3D interaction, realizing each technique category.

Selecting Menu Items in Mobile Head-Mounted Displays: Effects of Selection Technique and Active Area

ABSTRACTHead-mounted displays (HMDs) are increasingly available to users after the launch of new-generation consumer devices. Moreover, mobile HMDs such as Samsung Gear VR and Google Daydream View allow users to experience VR through a smartphone, without requiring connection to a PC. Commercial applications for mobile HMDs exploit different techniques to perform menu selection tasks. This paper contrasts the two most used techniques, i.e., dwell-based and touchpad-based selection, which were not experimentally compared before. We consider different versions of a menu pointing and selection task in which participants interacted with a Samsung Gear VR. Results show that participants were slower with the dwell-based technique rather than the touchpad-based technique. However, the dwell-based technique led to fewer errors and was perceived as more usable, more comfortable and less fatiguing than the touchpad-based technique. We also evaluated two different active areas for the selection, discussing the results.

CrowdDBS: A Crowdsourced Brightness Scaling Optimization for Display Energy Reduction in Mobile Video

Mobile display has become one of the most power-hungry components in mobile video viewing. Currently, mobile devices can reduce the display energy by performing dynamic brightness scaling (DBS) under the distortion constraint of video signals. We observe that there is a pitfall preventing current practice from systematic display energy reduction. In particular, existing objective DBS schemes lack direct connection to the subjective human perception on DBS-enabled videos, which is the key to achieving human-centered energy-experience optimization. To overcome this pitfall, we present CrowdDBS, a crowdsourced display energy reduction framework for mobile video viewing. CrowdDBS is empowered by a set of crowdsourcing studies that uncover the relationship between human perception and DBS frequency, magnitude, and temporal consistency, respectively. Motivated by the insights obtained from these studies, CrowdDBS employs a suit of designs and a DBS optimization framework to optimize the energy-experience tradeoff in mobile video viewing. Comprehensive experimental results and user evaluations under a variety of practical settings show that CrowdDBS can achieve 37 percent device energy reduction on average while guaranteeing satisfactory user experience in mobile video viewing.

Improved angular color uniformity and hydrothermal reliability of phosphor-converted white light-emitting diodes by using phosphor sedimentation.

We investigated the effect of phosphor deposition methods on the correlated color temperature (CCT), luminous flux and thermal characteristics of packaged white light-emitting diodes (WLEDs) for use in mobile display products. For both the samples, the CCT decreased with increasing viewing angle. Phosphor sedimentation samples displayed much better angular color uniformity than phosphor dispersion samples. The phosphor sedimentation sample had higher luminous flux and luminous efficacy at 20 mA than the phosphor dispersion sample. The phosphor sedimentation sample displayed much better high-temperature/humidity (85 °C/85%) reliability and lower package temperatures compared with the phosphor dispersion sample.

Interactive Mobile Augmented Reality System for Image and Hand Motion Tracking

In recent years, augmented reality (AR) is considered a promising technology that combines virtual information such as videos, images, and three-dimensional objects with a real camera view in mobile platforms. Interactive AR further provides human–computer interaction to allow the user to interact with virtual objects on the mobile display. In this paper, we proposed a cloud-based mobile augmented reality interactive system (MARIS), which includes MARIS-I for image target tracking and MARIS-H for hand motion tracking. MARIS-I estimates the position of the image target by adopting a feature-based mean-shift algorithm, which is feasible for real-time applications with its small region feature detection. MARIS-H provides two tracking modes for fingertip and back of hand tracking to enhance user experiences (UX) for interaction. Either the center position for the back of hand or fingertip is first estimated by particle filtering technique, which calculates the weighting of each particle according to hand or fingertip model. Afterward, the contour of the fingertip is estimated by level-set-based contour evolution in the fingertip tracking mode. Furthermore, we implement a device/cloud architecture for the proposed MARIS to decrease memory requirement and computational complexity on the device side. Experimental results show that MARIS including MARIS-I and MARIS-H can outperform other existing methods for image and hand motion tracking, respectively. The proposed MARIS is demonstrated in a picture book to provide fruitful interactive UX for digital learning systems.

Adaptive bit-plane selection-based low complexity motion estimation for screen content coding

Screen content video coding has become an emerging research topic with the spread of applications such as cloud gaming, screen/desktop virtualization, and mobile or external display interfacing. Screen content videos have different features compared to conventional camcorder-captured scenes. In this work, a novel low bit-depth representation-based motion estimation approach is proposed to exploit screen content specific features to improve coding efficiency. The proposed approach is based on an adaptive selection of Gray-coded bit-planes in order to generate low bit-depth representation of original screen content frames. The experimental results show that the motion estimation performance of the proposed approach is significantly better compared to the methods in the same category for screen content videos.

IOT APPLICATION ON SECURE SMART SHOPPING SYSTEM

Internet of Things (IoT) is relying on exchange of information and work progress through radio frequency identification (RFID), which is an emerging technology and one of the most important technology in the computing world. This kind of technologies has found its applications in various fields ranging from healthcare, construction, smart shopping, hospitality to transportation sector and many more. In this system, Billing can be generated from the shopping cart. The idea is to save customers time by providing digital billing system which you get through the registered mail of our website. A compartment is kept in which all the products are attached with RFID tags/cards. The benefit of this project exceeds the development of a smart world in a particular field as inventory management becomes much easier, as well as all the items can be read by the RFID reader rather than scanned manually by the laborer. Purchasing product information will get stored in the database. The billing will get generated at the mobile display as well as on the server. This system shows how RFID technology makes life easier and secure and thus helpful in the future. This system describes about IoT, concentrating its use in improving and securing the future shopping.

Polarization-dependent liquid crystalline polymeric lens array with aberration-improved aspherical curvature for low 3D crosstalk in 2D/3D switchable mobile multi-view display.

We demonstrated a 2D/3D switchable mobile display using a polarization-dependent switching liquid crystalline polymeric (LCP) lens array film. In spite of short viewing distance and enough viewing window conditions provided by a small f-number lens for mobile displays, the 3D images when switched to the multi-view 3D mode showed a low crosstalk property owing to the improved lens aberration, as applying an aspherical lens curvature interface between the planar-convex LCP layer and the concave-planar isotropic polymer layer. Both 2D and 3D images were demonstrated based on a 5.5-inch quad high definition mobile display panel, where the binocular crosstalk of the 3D mode was 3.3%.