Haptic Texture Research Articles

Evaluation of Haptic Textures for Tangible Interfaces for the Tactile Internet

Every texture in the real world provides us with the essential information to identify the physical characteristics of real objects. In addition to sight, humans use the sense of touch to explore their environment. Through haptic interaction we obtain unique and distinct information about the texture and the shape of objects. In this paper, we enhance X3D 3D graphics files with haptic features to create 3D objects with haptic feedback. We propose haptic attributes such as static and dynamic friction, stiffness, and maximum altitude that provide the optimal user experience in a virtual haptic environment. After numerous optimization attempts on the haptic textures, we propose various haptic geometrical textures for creating a virtual 3D haptic environment for the tactile Internet. These tangible geometrical textures can be attached to any geometric shape, enhancing the haptic sense. We conducted a study of user interaction with a virtual environment consisting of 3D objects enhanced with haptic textures to evaluate performance and user experience. The goal is to evaluate the realism and recognition accuracy of each generated texture. The findings of the study aid visually impaired individuals to better understand their physical environment, using haptic devices in conjunction with the enhanced haptic textures.

It Sounds Cool: Exploring Sonification of Mid-Air Haptic Textures Exploration on Texture Judgments, Body Perception, and Motor Behaviour.

Ultrasonic mid-air haptic technology allows for the perceptual rendering of textured surfaces onto the user's hand. Unlike real textured surfaces, however, mid-air haptic feedback lacks implicit multisensory cues needed to reliably infer a texture's attributes (e.g., its roughness). In this article, we combined mid-air haptic textures with congruent sound feedback to investigate how sonification could influence people's 1) explicit judgment of the texture attributes, 2) explicit sensations of their own hand, and 3) implicit motor behavior during haptic exploration. Our results showed that audio cues (presented solely or combined with haptics) influenced participants' judgment of the texture attributes (roughness, hardness, moisture and viscosity), produced some hand sensations (the feeling of having a hand smoother, softer, looser, more flexible, colder, wetter and more natural), and changed participants' speed (moving faster or slower) while exploring the texture. We then conducted a principal component analysis to better understand and visualize the found results and conclude with a short discussion on how audio-haptic associations can be used to create embodied experiences in emerging application scenarios in the metaverse.

Development and Evaluation of a Learning-based Model for Real-time Haptic Texture Rendering.

Current Virtual Reality (VR) environments lack the haptic signals that humans experience during real-life interactions, such as the sensation of texture during lateral movement on a surface. Adding realistic haptic textures to VR environments requires a model that generalizes to variations of a user's interaction and to the wide variety of existing textures in the world. Current methodologies for haptic texture rendering exist, but they usually develop one model per texture, resulting in low scalability. We present a deep learning-based action-conditional model for haptic texture rendering and evaluate its perceptual performance in rendering realistic texture vibrations through a multi-part human user study. This model is unified over all materials and uses data from a vision-based tactile sensor (GelSight) to render the appropriate surface conditioned on the user's action in real-time. For rendering texture, we use a high-bandwidth vibrotactile transducer attached to a 3D Systems Touch device. The results of our user study shows that our learning-based method creates high-frequency texture renderings with comparable or better quality than state-of-the-art methods without the need to learn a separate model per texture. Furthermore, we show that the method is capable of rendering previously unseen textures using a single GelSight image of their surface.

Beyond sense-specific processing: decoding texture in the brain from touch and sonified movement

Texture, a fundamental object attribute, is perceived through multisensory information including touch and auditory cues. Coherent perceptions may rely on shared texture representations across different senses in the brain. To test this hypothesis, we delivered haptic textures coupled with a sound synthesizer to generate real-time textural sounds. Participants completed roughness estimation tasks with haptic, auditory, or bimodal cues in an MRI scanner. Somatosensory, auditory, and visual cortices were all activated during haptic and auditory exploration, challenging the traditional view that primary sensory cortices are sense-specific. Furthermore, audio-tactile integration was found in secondary somatosensory (S2) and primary auditory cortices. Multivariate analyses revealed shared spatial activity patterns in primary motor and somatosensory cortices, for discriminating texture across both modalities. This study indicates that primary areas and S2 have a versatile representation of multisensory textures, which has significant implications for how the brain processes multisensory cues to interact more efficiently with our environment.

Establishing haptic texture attribute space and predicting haptic attributes from image features using 1D-CNN

The current study strives to provide a haptic attribute space where texture surfaces are located based on their haptic attributes. The main aim of the haptic attribute space is to come up with a standardized model for representing and identifying haptic textures analogous to the RGB model for colors. To this end, a four dimensional haptic attribute space is established by conducting a psychophysical experiment where human participants rate 100 real-life texture surfaces according to their haptic attributes. The four dimensions of the haptic attribute space are rough-smooth, flat-bumpy, sticky-slippery, and hard-soft. The generalization and scalability of the haptic attribute space is achieved by training a 1D-CNN model for predicting attributes of haptic textures. The 1D-CNN is trained using the attribute data from psychophysical experiments and image features collected from the images of real textures. The prediction power granted by the 1D-CNN renders scalability to the haptic attribute space. The prediction accuracy of the proposed 1D-CNN model is compared against other machine learning and deep learning algorithms. The results show that the proposed method outperforms the other models on MAE and RMSE metrics.

Perceptual Substitution Based Haptic Texture Rendering for Narrow-Band Reproduction.

Recorded high-resolution texture vibration contains perceptually redundant spectral information due to tactile limitations of human skin. Also, accurate reproduction of recorded texture vibration is often infeasible for widely available haptic reproduction systems at mobile devices. Usually, haptic actuators can only reproduce narrow-bandwidth vibration. With the exception of research setups, rendering strategies need to be developed, that utilize the limited capabilities of various actuator systems and tactile receptors while minimizing a negative impact on perceived quality of reproduction. Therefore, the aim of this study is to substitute recorded texture vibrations with perceptually sufficient simple vibrations. Accordingly, similarity of band-limited noise, single sinusoid and amplitude-modulated signals on display are rated compared to real textures. Considering that low and high frequency bands of noise signals might be implausible and redundant, different combinations of cut-off frequencies are applied to noise vibrations. Moreover, suitability of amplitude-modulation signals are tested for coarse textures in addition to single sinusoids because of their capability of creating pulse-like roughness sensation without too low frequencies. With the set of experiments, narrowest band noise vibration with frequencies between 90 Hz to 400 Hz is determined according to the fine textures. Furthermore, AM vibrations are found to be more congruent than single sinusoids to reproduce too coarse textures.

Potential Haptic Perceptual Dimensionality of Rendered Compliance.

Studies have proven that humans perceive haptic textures through different perceptual dimensions, such as rough/smooth and soft/hard, which provide useful guidance in the design of haptic devices. However, few of these have focused on the perception of compliance, which is another important perceptual property in haptic interfaces. This research was conducted to investigate the potential basic perceptual dimensions of the rendered compliance and quantify the effects of the simulation parameters. Two perceptual experiments were designed based on 27 stimuli samples generated by a 3-DOF haptic feedback device. Subjects were asked to describe these stimuli using adjectives, classify the samples, and rate them according to corresponding adjective labels. Multi-dimensional scaling (MDS) methods were then used to project adjective ratings into 2D and 3D perception spaces. According to the results, hardness and viscosity are considered two basic perceptual dimensions of the rendered compliance, while crispness can be regarded as a subsidiary perceptual dimension. Then, the relations between simulation parameters and perceptual feelings were analyzed by the regression analysis. This paper may provide a better understanding of the compliance perception mechanism and useful guidance for the improvement of rendering algorithms and devices for haptic human-computer interaction.

Haptic Texture Rendering of 2D Image Based on Adaptive Fractional Differential Method

The fractional differential algorithm has a good effect on extracting image textures, but it is usually necessary to select an appropriate fractional differential order for textures of different scales, so we propose a novel approach for haptic texture rendering of two-dimensional (2D) images by using an adaptive fractional differential method. According to the fractional differential operator defined by the Grünvald–Letnikov derivative (G–L) and combined with the characteristics of human vision, we propose an adaptive fractional differential method based on the composite sub-band gradient vector of the sub-image obtained by wavelet decomposition of the image texture. We apply these extraction results to the haptic display system to reconstruct the three-dimensional (3D) texture force filed to render the texture surface of two-dimensional (2D) images. Based on this approach, we carry out the quantitative analysis of the haptic texture rendering of 2D images by using the multi-scale structural similarity (MS-SSIM) and image information entropy. Experimental results show that this method can extract the texture features well and achieve the best texture force file for 2D images.

The Matter of Manual Traces: Letters, Photographs and Bean Paste in Naomi Kawase’s Cinema of Touch

This article explores the representation of the hand in three recent fiction films by Naomi Kawase: Sweet Bean (2015), Radiance (2017) and True Mothers (2020). Extending current scholarship that discusses the director’s use of haptic visuality, I argue that Kawase’s haptic cinema further exhibits its hapticity by framing the hand as both a Derridean trace and conduit that leaves behind traces in objects such as bean paste, letters, photo cameras, and photographs. Kawase’s framing of these objects in close-up shots emphasizes not only their materiality and haptic texture but also their foreshadowed physical absence. By reading Kawase through Jacques Derrida’s notion of trace, the article highlights not only the relevance behind understanding Kawase’s recent fiction films through a Derridean lens but also how the director’s own attitude towards cinema, as one that aims to simultaneously frame absence and presence, can also be read as a Derridean approach to cinema.

Artificial Tactile Sensing System with Photoelectric Output for High Accuracy Haptic Texture Recognition and Parallel Information Processing.

Developing multifunctional artificial sensory systems is an important task for constructing future artificial neural networks. A system with multisignal output capability is highly required by the rising demand for high-throughput data processing in the Internet of Things (IoT) society. Here, a novel dual-output artificial tactile sensing (DOATS) system with parallel output of photoelectric signals was proposed. Because of the ionic-electronic coupling mechanism in light-emitting synaptic (LES) devices in the DOATS system, modulating electric current and light emission can coexist through ion accumulation and electron-hole recombination. As a result, the DOATS system can realize the simulation of human tactile information, and the recognition of 16 kinds of fabrics was demonstrated with an accuracy rate of 94.1%. A photoelectric hybrid artificial neural network was proposed, which achieved efficient and accurate multitask operation. The DOATS system proposed in this work is promising for implementing photoelectric hybrid neural network and promoting the development of interactive artificial intelligence.

Preference-Driven Texture Modeling Through Interactive Generation and Search.

Data-driven texture modeling and rendering has pushed the limit of realism in haptics. However, the lack of haptic texture databases, difficulties of model interpolation and expansion, and the complexity of real textures prevent data-driven methods from capturing a large variety of textures and from customizing models to suit specific output hardware or user needs. This work proposes an interactive texture generation and search framework driven by user input. We design a GAN-based texture model generator, which can create a wide range of texture models using Auto-Regressive processes. Our interactive texture search method, which we call "preference-driven," follows an evolutionary strategy given guidance from user's preferred feedback within a set of generated texture models. We implemented this framework on a 3D haptic device and conducted a two-phase user study to evaluate the efficiency and accuracy of our method for previously unmodeled textures. The results showed that by comparing the feel of real and generated virtual textures, users can follow an evolutionary process to efficiently find a virtual texture model that matched or exceeded the realism of a data-driven model. Furthermore, for 4 out of 5 real textures, ≥ 80% of the preference-driven models from participants were rated comparable to the data-driven models.

Data-Driven Haptic Texture Modeling and Rendering Based on Deep Spatio-Temporal Networks.

Data-driven approaches are commonly used to model and render haptic textures for rigid stylus-based interaction. Current state-of-the-art data-driven methodologies synthesize acceleration signals through the interpolation of samples with different input parameters based on neural networks or parametric spectral estimation methods. In this paper, we see the potential of emerging deep learning methods in this area. To this end, we designed a complete end-to-end data-driven framework to synthesize acceleration profiles based on the proposed deep spatio-temporal network. The network is trained using contact acceleration data collected through our manual scanning stylus and interaction parameters, i.e., scanning velocities, directions, and forces. The proposed network is composed of attention-aware 1D CNNs and attention-aware encoder-decoder networks to adequately capture both the local spatial features and the temporal dynamics of the acceleration signals, which are further augmented with attention mechanisms that assign weights to the features according to their contributions. For rendering, the trained network generates synthesized signals in real-time in accordance with the user's input parameters. The whole framework was numerically compared with existing state-of-the-art approaches, showing the effectiveness of the approach. Additionally, a pilot user study is conducted to demonstrate subjective similarity.

Soft Pneumatic Fingertip Actuator Incorporating a Dual Air Chamber to Generate Multi-Mode Simultaneous Tactile Feedback

A complete set of diverse haptic feedbacks is essential for a highly realistic and immersive virtual environment. In this sense, a multi-mode haptic interface that simultaneously generates multiple kinds of haptic signals is highly desirable. In this paper, we propose a new silicone-made pneumatically controlled fingertip actuator to render pressure and vibrotactile feedback concurrently to offer a realistic and effective haptic sensation. A new silicone-based stacked dual-layer air chamber was designed. The volume of the chambers is controlled by pneumatic valves with compressed air tanks. The top/upper air chamber renders vibration feedback, whereas the bottom/lower air chamber renders pressure feedback. The proposed silicone-made fingertip actuator is designed so that it can be easily worn at the fingertips. To demonstrate the potential of the system, a virtual environment for rendering three different types of haptic textures was implemented. Extensive performance evaluation and user studies were carried out to demonstrate the proposed actuator’s effectiveness compared to an actuator with single vibrotactile feedback.

Building a Navigable Fine Texture Design Space.

Friction modulation technology enables the creation of textural effects on flat haptic displays. However, an intuitive and manageably small design space for construction of such haptic textures remains an unfulfilled goal for user interface designers. In this paper, we explore perceptually relevant features of fine texture for use in texture construction and modification. Beginning with simple sinusoidal patterns of friction force that vary in frequency and amplitude, we define irregularity, essentially a variable amount of introduced noise, as a third building block of a texture pattern. We demonstrate using multidimensional scaling that all three parameters are scalable features perceptually distinct from each other. Additionally, participants' verbal descriptions of this 3-dimensional design space provide insight into their intuitive interpretation of the physical parameter changes.

Integration of visual and haptic texture information: Lederman & Abbott revisited

Integration of visual and haptic texture information: Lederman & Abbott revisited

Sensing and Rendering Method of 2-Dimensional Haptic Texture.

Nowadays, touchscreens have been used worldwide. However, most of them lack realistic haptic feedback. Several haptic feedback devices employ one-dimensional vibration only. We aim at a novel rendering method for direction-controlled 2-dimensional vibration display to present texture information. This paper proposed a rendering method of texture information that enables lateral-force-based 2-dimensional vibration in the X and Y-axis. Moreover, we proposed combining AKAZE image feature information of the textures to improve the fidelity for larger periodic textures. We held experiments to evaluate the fidelity of the proposed method. The result shows that the proposed method has higher fidelity in presenting randomized textures and large periodic textures than the conventional method.

Generating haptic texture using solid noise

Texture enhances haptic interaction by providing unique, distinguishable, and versatile surfaces. In computer haptics, texture can render environments more realistic and provide useful information. In this paper, an algorithm is proposed for virtual texture simulation by using solid noise, where only a few parameters need to be altered to generate a range of realistic and diverse textures by reproducing different frequencies similar to that of real vibrational signals in a virtual environment. The proposed method can capture the textural effect in a haptic simulation while retaining a simple overall geometry and stable update rate. This method also allows the user to change the texture at runtime and can be easily incorporated into any existing code and used in any traditional haptic device without affecting overall haptic-rendering performance. Moreover, the solid noise texture is independent of object geometry and can be applied to any shape without additional computations. We conducted a human-subject study to evaluate the recognition accuracy for each generated haptic texture as well as its realism and correspondence to real texture. The results indicated the high performance of the method and its ability to generate haptic textures with a very high recognition rate that were highly realistic.

The roles of lower- and higher-order surface statistics in tactile texture perception.

Humans can haptically discriminate surface textures when there is a significant difference in the statistics of the surface profile. Previous studies on tactile texture discrimination have emphasized the perceptual effects of lower-order statistical features such as carving depth, inter-ridge distance, and anisotropy, which can be characterized by local amplitude spectra or spatial-frequency/orientation subband histograms. However, the real-world surfaces we encounter in everyday life also differ in the higher-order statistics, such as statistics about correlations of nearby spatial-frequencies/orientations. For another modality, vision, the human brain has the ability to use the textural differences in both higher- and lower-order image statistics. In this work, we examined whether the haptic texture perception can use higher-order surface statistics as visual texture perception does, by three-dimensional (3-D)-printing textured surfaces transcribed from different "photos" of natural scenes such as stones and leaves. Even though the maximum carving depth was well above the haptic detection threshold, some texture pairs were hard to discriminate. Specifically, those texture pairs with similar amplitude spectra were difficult to discriminate, which suggests that the lower-order statistics have the dominant effect on tactile texture discrimination. To directly test the poor sensitivity of the tactile texture perception to higher-order surface statistics, we matched the lower-order statistics across different textures using a texture synthesis algorithm and found that haptic discrimination of the matched textures was nearly impossible unless the stimuli contained salient local features. We found no evidence for the ability of the human tactile system to use higher-order surface statistics for texture discrimination.NEW & NOTEWORTHY Humans can discriminate subtle spatial patterns differences in the surrounding world through their hands, but the underlying computation remains poorly understood. Here, we 3-D-printed textured surfaces and analyzed the tactile discrimination performance regarding the sensitivity to surface statistics. The results suggest that observers have sensitivity to lower-order statistics whereas not to higher-order statistics. That is, touch differs from vision not only in spatiotemporal resolution but also in (in)sensitivity to high-level surface statistics.

Masking interferes with haptic texture perception from sequential exploratory movements

Haptic texture perception is based on sensory information sequentially gathered during several lateral movements (“strokes”). In this process, sensory information of earlier strokes must be preserved in a memory system. We investigated whether this system may be a haptic sensory memory. In the first experiment, participants performed three strokes across each of two textures in a frequency discrimination task. Between the strokes over the first texture, participants explored an intermediate area, which presented either a mask (high-energy tactile pattern) or minimal stimulation (low-energy smooth surface). Perceptual precision was significantly lower with the mask compared with a three-strokes control condition without an intermediate area, approaching performance in a one-stroke-control condition. In contrast, precision in the minimal stimulation condition was significantly better than in the one-stroke control condition and similar to the three-strokes control condition. In a second experiment, we varied the number of strokes across the first stimulus (one, three, five, or seven strokes) and either presented no masking or repeated masking after each stroke. Again, masking between the strokes decreased perceptual precision relative to the control conditions without masking. Precision effects of masking over different numbers of strokes were fit by a proven model on haptic serial integration (Lezkan & Drewing, Attention, Perception, & Psychophysics 80(1): 177–192, 2018b) that modeled masking by repeated disturbances in the ongoing integration. Taken together, results suggest that masking impedes the processes of haptic information preservation and integration. We conclude that a haptic sensory memory, which is comparable to iconic memory in vision, is used for integrating sequentially gathered sensory information.

Synthesizing the Roughness of Textured Surfaces for an Encountered-Type Haptic Display Using Spatiotemporal Encoding.

Encountered-type haptic rendering provides realistic, free-to-touch, and move-and-collide haptic sensation to a user. However, inducing haptic-texture sensation without complicated tactile actuators is challenging for encountered-type haptic rendering. In this article, we propose a novel texture synthesizing method for an encountered-type haptic display using spatial and temporal encoding of roughness, which provides both active and passive touch sensation requiring no complicated tactile actuation. Focused on macro-scale roughness perception, we geometrically model the textured surface with a grid of hemiellipsoidal bumps, which can provide a variety of perceived roughness as the user explores the surface with one's bare hand. Our texture synthesis method is based on two important hypotheses. First, we assume that perceptual roughness can be spatially encoded along the radial direction of a textured surface with hemiellipsoidal bumps. Second, perceptual roughness temporally varies with the relative velocity of a scanning human hand with respect to the surface. To validate these hypotheses on our spatiotemporal encoding method, we implemented an encountered-type haptic texture rendering system using an off-the-shelf collaborative robot that can also track the user's hand using IR sensors. We performed psychophysical user tests with 25 participants and verified the main effects of spatiotemporal encoding of a textured model on the user's roughness perception. Our empirical experiments imply that the users perceive a more rough texture as the surface orientation or the relative hand motion increases. Based on these findings, we show that our visuo-haptic system can synthesize an appropriate level of roughness corresponding to diverse visual textures by suitably choosing encoding values.