Visual Human-computer Interaction Research Articles

Large‐Area Knittable, Wash‐Durable, and Healable Smart Fibers for Dual‐Modal Sensing Applications

AbstractFiber‐based multimodal sensors with electrical/optical signals are highly desired for next‐generation wearable electronics. Despite the remarkable progress in this area, achieving large‐scale knittable, washable, and self‐healing performance in fiber‐based multimodal sensors simultaneously remains a great challenge. Here, a smart fiber capable of exhibiting piezoresistive/luminescent properties based on an H‐bonding connected multilayered core–shell nanostructure is developed. The core principle of this design involves constructing strong interfacial interactions between the fiber layers, which results in a sensor with high sensitivity (gauge factor = 12383500), exceptional water resistance, and robust self‐healing properties (tensile strength 30.9 MPa, healing efficiency 72.9%). Unlike traditional fiber‐based sensors where elaborate nanostructures are prone to shedding during knitting, this strategy enables the fiber sensors with excellent knittability to be patterned in the fabric, improving both optical and electrical sensitivities. This work is anticipated to make a significant contribution to the further development of wearable electronic products and visual human–computer interaction electronic devices.

MCD-LightGBM System for Intelligent Analyzing Heterogeneous Clinical Drug Therapeutic Effects.

Causal effect estimation of individual heterogeneity is a core issue in the field of causal inference, and its application in medicine poses an active and challenging problem. In high-risk decision-making domain such as healthcare, inappropriate treatments can have serious negative impacts on patients. Recently, machine learning-based methods have been proposed to improve the accuracy of causal effect estimation results. However, many of these methods concentrate on estimating causal effects of continuous outcome variables under binary intervention conditions, and give less consideration to multivariate intervention conditions or discrete outcome variables, thus limiting their scope of application. To tackle this issue, we combine the double machine learning framework with Light Gradient Boosting Machine (LightGBM) and propose a double LightGBM model. This model can estimate binary causal effects more accurately and in less time. Two cyclic structures were added to the model. Data correction method was introduced and improved to transform discrete outcome variables into continuous outcome variables. Multivariate Cyclic Double LightGBM model (MCD-LightGBM) was proposed to intelligently estimate multivariate treatment effects. A visual human-computer interaction system for heterogeneous causal effect estimation was designed, which can be applied to different types of data. This paper reports that the system improved the Logarithm of the Minimum Angle of Resolution (LogMAR) of visual acuity change after Vascular Endothelial Growth Factor (anti-VEGF) treatment in patients with diabetic macular degeneration. The improvement was observed in two clinical problems, from 0.05 to 0.33, and the readmission rate of diabetic patients after cure was reduced from 48.4% to 10.5%. The results above demonstrate the potential of the proposed system in predicting heterogeneous clinical drug treatment effects.

Retracted: A Visual Human-Computer Interaction System Based on Hybrid Visual Model

Retracted: A Visual Human-Computer Interaction System Based on Hybrid Visual Model

High-strength, anti-freeze, transparent and recyclable composite organohydrogel for flexible strain sensor

Hydrogels have received widespread attention as flexible strain sensors in the fields of human motion detection and health monitoring. However, the presence of large amounts of water in traditional hydrogels leads to poor mechanical properties and low anti-freezing, which severely limits their applications. Herein, a novel PVA/DMSO/CMC/AlCl3/TA/LiCl (PDCATL) composite organo-hydrogel was developed by freeze-thaw and salt solution immersion methods, using polyvinyl alcohol (PVA), carboxymethyl cellulose (CMC), aluminum chloride hexahydrate (AlCl3·6H2O) and tannic acid (TA) dissolved in a dimethyl sulfoxide (DMSO)/water binary solvent. The PDCATL organohydrogel exhibited high tensile strength and elongation at break (up to 1.90 MPa and 587%), which can withstand >6000 times its own weight, good anti-freezing and excellent transparency (>90% light transmission). The PDCATL organohydrogel still maintained excellent transparency, flexibility, recyclability and signal sensing capability at low temperatures (−20 °C). The PDCATL organohydrogel also had good strain sensitivity (GF = 1.82) as a wearable strain sensor and can be used to detect motion and physiological activity signals in various parts of the human body. This study will provide a viable avenue for the application of hydrogel sensors in the fields of visual human-computer interaction, wearable sensors, visual optical detection and medical health detection.

A Visual Human-Computer Interaction System Based on Hybrid Visual Model

The traditional human-computer interaction is mainly through the mouse, keyboard, remote control, and other peripheral equipment electromagnetic signal transmission. This paper aims to build a visual human-computer interaction system through a series of deep learning and machine vision models, so that people can achieve complete human-computer interaction only through the camera and screen. The established visual human-computer interaction system mainly includes the function modes of three basic peripherals in human-computer interaction: keyboard, mouse (X-Y position indicator), and remote control. The convex hull method was used to switch between these three modes. After issuing the mode command, Gaussian mixture was used to quickly identify the moving human body to narrow the scope of our image processing. Subsequently, finger detection in human body was realized based on the Faster-RCNN-ResNET50-FPN model structure, and realized the function of moving mouse and keyboard through the relationship between different fingers. At the same time, the recognition of human body posture was done by using MediaPipe BlazePose, and the action classification models were established through the Angle between body movements so as to realize the control function of remote control. In order to ensure the real-time performance of the interactive system, according to the characteristics of different data processing processes, CPU and GPU computing power resources are used to cross-process images to ensure the real-time performance. The recognition accuracy of the human-computer interaction system is above 0.9 for the key feature points of human body, and above 0.87 for the recognition accuracy of four kinds of command actions. It is hoped that vision-based human-computer interaction will become a widely used interaction mode in the future.

Human–computer interaction-based Decision Support System with Applications in Data Mining

Human–computer interaction (HCI) plays a vital role in modern intelligent systems, such as brain–machine integration, human action recognition, telemedicine, and somatosensory game. A decision support system is a combination of the information system and decision-making technology. Visual human–computer interaction decision-making is a key technique in the decision support system. This paper proposes a new visual decision-making system applicable for industrial applications, i.e. data mining topics. To assist the performance of the proposed visual decision-making system, it is designed for data mining technique applications. Furthermore, the architecture of the decision support system is analyzed based on practical data mining case study. The comprehensive experiment shows that the proposed method is effective and robust in comparison to other methods.

Development of measurement instrument for visual qualities of graphical user interface elements (VISQUAL): a test in the context of mobile game icons

Graphical user interfaces are widely common and present in everyday human–computer interaction, dominantly in computers and smartphones. Today, various actions are performed via graphical user interface elements, e.g., windows, menus and icons. An attractive user interface that adapts to user needs and preferences is progressively important as it often allows personalized information processing that facilitates interaction. However, practitioners and scholars have lacked an instrument for measuring user perception of aesthetics within graphical user interface elements to aid in creating successful graphical assets. Therefore, we studied dimensionality of ratings of different perceived aesthetic qualities in GUI elements as the foundation for the measurement instrument. First, we devised a semantic differential scale of 22 adjective pairs by combining prior scattered measures. We then conducted a vignette experiment with random participant (n = 569) assignment to evaluate 4 icons from a total of pre-selected 68 game app icons across 4 categories (concrete, abstract, character and text) using the semantic scales. This resulted in a total of 2276 individual icon evaluations. Through exploratory factor analyses, the observations converged into 5 dimensions of perceived visual quality: Excellence/Inferiority, Graciousness/Harshness, Idleness/Liveliness, Normalness/Bizarreness and Complexity/Simplicity. We then proceeded to conduct confirmatory factor analyses to test the model fit of the 5-factor model with all 22 adjective pairs as well as with an adjusted version of 15 adjective pairs. Overall, this study developed, validated, and consequently presents a measurement instrument for perceptions of visual qualities of graphical user interfaces and/or singular interface elements (VISQUAL) that can be used in multiple ways in several contexts related to visual human-computer interaction, interfaces and their adaption.

Specification and dialogue control of visual interaction through visual rewriting systems

Computers are increasingly being seen not only as computing tools but more so as communication tools, thus placing special emphasis on human-computer interaction (HCI). In this article, the focus is on visual HCI, where the messages exchanged between human and computer are images appearing on the computer screen, as usual in current popular user interfaces. We formalize interactive sessions of a human-computer dialogue as a structured set of legal visual sentences, i.e., as a visual language, and show how rewriting systems can be generalized to specify both the pictorial and the computational aspects of visual languages. To this end, Visual Conditional Attributed Rewriting (VCARW) systems are introduced, and use for specification of visual languages. These specifications are given as inputs to a procedure illustrated in the article as a system of algorithms, which automatically generates control mechanisms of the interaction, thus favoring the design of more reliable and usable systems.

Successful visual human-computer interaction is undecidable

We propose a model of visual interactive computing and define a family of visual languages which abstract some of its features. We prove that the problem whether or not successful communications exist is not decidable. (A human-computer communication by means of a visual language is successful when all human commands, in the form of pictures composed of icons, are correctly interpreted and answered by the computer.) The proof makes use of a codification of the Post Correspondence Problem by means of icons and pictures. This result warns about possible limitations of a purely algorithmic treatment of interactive computing via visual languages.

Defining visual languages for interactive computing

A novel definition of visual languages allows a uniform approach to satisfying the needs of visual reasoning faced in visual human-computer interaction. The way the machine associates a computational meaning with an image, and conversely, the way it generates an image on the screen from a computation are formally described. A definition of a visual sentence and of a visual language as a set of visual sentences is discussed. A hierarchy of visual languages is derived in relation to the requirements for intelligible, manageable and trustable interaction between humans and computers.