Object Texture Research Articles

Position-Based ANN Impedance Control for A Three-Fingered Robot Hand

A feedforward ANN was previously developed for recognition of two objects i.e. a spongy ball and a plastic bottle but was verified through simulation only. In this work, the feasibility of the ANN model is tested by applying it to the robot’s impedance control which takes the exerted force at the finger as input while resulting in an output for the selection rule of the impedance stiffness parameter, named 𝐾𝑑. From the results, the different object textures can be distinguished by the ANN where the absolute peak values of measured rate force during contact with the ball reached 0.15 N and a slightly higher value of 0.32 N for the bottle. 𝐾𝑑 values were found to switch between 1000 and 250 based on the ANN outputs for the ball and bottle, respectively, thus affecting the dynamics of the fingertip through modified position reference of the fingertip. However, it is also observed that the object was incorrectly classified in some moments when the exerted force was not sufficient due to the weak grasp of the object. This shows that the nonlinear factor from the hardware defects needs to be considered when refining the 𝐾𝑑 selection rule in future studies.

Neural 3D Scene Reconstruction With Indoor Planar Priors.

This paper addresses the challenge of reconstructing 3D indoor scenes from multi-view images. Many previous works have shown impressive reconstruction results on textured objects, but they still have difficulty in handling low-textured planar regions, which are common in indoor scenes. An approach to solving this issue is to incorporate planar constraints into the depth map estimation in multi-view stereo-based methods, but the per-view plane estimation and depth optimization lack both efficiency and multi-view consistency. In this work, we show that the planar constraints can be conveniently integrated into the recent implicit neural representation-based reconstruction methods. Specifically, we use an MLP network to represent the signed distance function as the scene geometry. Based on the Manhattan-world assumption and the Atlanta-world assumption, planar constraints are employed to regularize the geometry in floor and wall regions predicted by a 2D semantic segmentation network. To resolve the inaccurate segmentation, we encode the semantics of 3D points with another MLP and design a novel loss that jointly optimizes the scene geometry and semantics in 3D space. Experiments on ScanNet and 7-Scenes datasets show that the proposed method outperforms previous methods by a large margin on 3D reconstruction quality.

A soft robotic system imitating the multimodal sensory mechanism of human fingers for intelligent grasping and recognition

The sensory function is crucial for achieving precise feedback control and environmental interaction in soft robots. Therefore, the multimodal sensory capabilities that mimic human fingers have always been a research hotspot. This study proposes a design method for a soft robotic gripper that can emulate the multimodal perception mechanism of human fingers and achieve high-precision recognition of grasped objects using machine learning algorithms. In addition, a finger texture structure inspired by human fingerprints is designed using a negative pressure-induced buckling method to enhance the soft gripper's ability to perceive minute surface textures. Inspired by slow adapting (SA) receptors and fast adapting (FA) receptors of human fingers, we have innovatively designed a capacitive curvature sensor (CS) and a triboelectric texture sensor (TS) to detect the size, material, and texture of objects. Furthermore, a non-contact medium identification sensor (MIS), having a sensitivity of more than 1300 times higher compared to the traditional MISs, is fabricated using the principle of electromagnetic resonance. It is shown that by employing a deep learning-based multi-channel feature fusion network, the soft grasping system with multimodal sensing has achieved a recognition accuracy of 98.43 % for different objects. This work provides an innovative approach for the application of soft robots in various fields, such as logistics sorting and food safety.

EASI-Tex: Edge-Aware Mesh Texturing from Single Image

We present a novel approach for single-image mesh texturing , which employs a diffusion model with judicious conditioning to seamlessly transfer an object's texture from a single RGB image to a given 3D mesh object. We do not assume that the two objects belong to the same category, and even if they do, there can be significant discrepancies in their geometry and part proportions. Our method aims to rectify the discrepancies by conditioning a pre-trained Stable Diffusion generator with edges describing the mesh through ControlNet, and features extracted from the input image using IP-Adapter to generate textures that respect the underlying geometry of the mesh and the input texture without any optimization or training. We also introduce Image Inversion , a novel technique to quickly personalize the diffusion model for a single concept using a single image , for cases where the pre-trained IP-Adapter falls short in capturing all the details from the input image faithfully. Experimental results demonstrate the efficiency and effectiveness of our edge-aware single-image mesh texturing approach, coined EASI-Tex, in preserving the details of the input texture on diverse 3D objects, while respecting their geometry. Code : https://github.com/sairajk/easi-tex

An effective textured Novel Object Recognition Test (tNORT) for repeated measure of whisker sensitivity of rodents

Rodents use their whisker system to discriminate surface texture. Whisker-based texture discrimination tasks are often used to investigate the mechanisms encoding tactile sensation. One such task is the textured Novel Object Recognition Test (tNORT). It takes advantage of a tendency of rodents to explore novel objects more than familiar ones and assesses the sensitivity of whiskers in discriminating different textures of objects. It requires little training of the animals and the equipment involved is a simple arena with typically two objects placed inside. The success of the test relies on rodents spending sufficient time exploring these objects. Animals may lose interests in such tasks when performed repetitively within a limited time frame. However, such repeated tests may be crucial when establishing a sensitivity threshold of the whisker system. Here we present an adapted rodent tNORT protocol designed to maintain sustained interest in the objects even with repeated testing. We constructed complex objects from three simple-shaped objects. Different textures were provided by sandpapers of varying grit sizes. To minimise olfactory clues, we used the sandy and the laminar side of the same sandpaper as the familiar and novel textures assigned at random. We subsequently conducted repeated tNORTs on eight rats in order to identify a critical threshold of the sandpaper grit size below which rats would be unable to discriminate the sandy from the laminar side. With an inter-test-interval of seven days and after five tNORTs, the protocol enabled us to successfully identify the threshold. We suggest that the proposed tNORT is a useful tool for investigating the sensitivity threshold of the whisker system of rodent, and for testing the effectiveness of an intervention by comparing sensitivity threshold pre- and post-intervention.

Redefining Accuracy: Underwater Depth Estimation for Irregular Illumination Scenes.

Acquiring underwater depth maps is essential as they provide indispensable three-dimensional spatial information for visualizing the underwater environment. These depth maps serve various purposes, including underwater navigation, environmental monitoring, and resource exploration. While most of the current depth estimation methods can work well in ideal underwater environments with homogeneous illumination, few consider the risk caused by irregular illumination, which is common in practical underwater environments. On the one hand, underwater environments with low-light conditions can reduce image contrast. The reduction brings challenges to depth estimation models in accurately differentiating among objects. On the other hand, overexposure caused by reflection or artificial illumination can degrade the textures of underwater objects, which is crucial to geometric constraints between frames. To address the above issues, we propose an underwater self-supervised monocular depth estimation network integrating image enhancement and auxiliary depth information. We use the Monte Carlo image enhancement module (MC-IEM) to tackle the inherent uncertainty in low-light underwater images through probabilistic estimation. When pixel values are enhanced, object recognition becomes more accessible, allowing for a more precise acquisition of distance information and thus resulting in more accurate depth estimation. Next, we extract additional geometric features through transfer learning, infusing prior knowledge from a supervised large-scale model into a self-supervised depth estimation network to refine loss functions and a depth network to address the overexposure issue. We conduct experiments with two public datasets, which exhibited superior performance compared to existing approaches in underwater depth estimation.

Objective texture evaluation of cosmetic sericite powders mixed with oil using constant-volume shear cell tester

Although sensory evaluation is often used to evaluate cosmetic powder textures, such evaluations are problematic because they are subjective and depend on human senses. Consequently, an objective evaluation is highly desired. We therefore report herein the use of a constant-volume shear tester to objectively evaluate the physical properties of a cosmetic powder. The goal of the study is to detect the differences in the physical properties of powders, which is done by applying a constant-volume shear test to oil-wet powders of differing textures mixed with dimethicone or squalane. Differences between dimethicone and squalane mixtures appear in the stress relaxation ratio (softness), the shear cohesion (moist feeling), the internal friction coefficient for friction (for normal stress of <10 kPa) between powder beds (smoothness), and the dynamic friction coefficient for friction between powder beds. Although the oils presented a similar degree of the powder yield locus, the shape of the powder yield locus indicates the state of the powder beds. The results of the study show that constant-volume shear testing can be used to objectively evaluate oil-wet powder textures.

Maternal immune activation affects female offspring whisker movements during object exploration in a rat model of neurodevelopmental disorders

Poly I:C rat offspring are used to investigate the effects of in utero exposure to maternal immune activation (MIA) and have been suggested as a model of neurodevelopmental disorders (NDD). The behavioural symptoms of this model are diverse and can vary with external factors, including the choice of background strain and husbandry practices. Measuring whisker movements provides quantitative, robust measurements of sensory, motor and cognitive behaviours in rodents. In this study, whisker movements were investigated in 50-day-old male and female offspring of MIA-exposed rat dams and compared to age-matched offspring of control (vehicle) dams. Rat offspring were filmed using high-speed videography in a sequential object exploration task with smooth and textured objects. Poly I:C treatment effects were found in female offspring that did not increase whisker mean angular position during object exploration, especially for the smooth object, indicating an attentional deficit. Whisker tracking during object exploration is demonstrated here, for the first time, as a useful, quick and non-invasive tool to identify both treatment effects and sex differences in a model of MIA-induced NDDs.

A 3D Reconstruction Method Based on Homogeneous De Bruijn-Encoded Structured Light

Structured light three-dimensional reconstruction is one of the important methods for non-contact acquisition of sparse texture object surfaces. Variations in ambient illumination and disparities in object surface reflectance can significantly impact the fidelity of three-dimensional reconstruction, introducing considerable inaccuracies. We introduce a robust method for color speckle structured light encoding, which is based on a variant of the De Bruijn sequence, termed the Homogeneous De Bruijn Sequence. This innovative approach enhances the reliability and accuracy of structured light techniques for three-dimensional reconstruction by utilizing the distinctive characteristics of Homogeneous De Bruijn Sequences. Through a pruning process applied to the De Bruijn sequence, a structured light pattern with seven distinct color patches is generated. This approach ensures a more equitable distribution of speckle information.

Experiment and modelling of texture and sliding direction dependence on finger friction behavior

Humans rely on their fingers to sense and interact with external environment. Understanding the tribological behavior between finger skin and object surface is crucial for various fields, including tactile perception, product appearance design, and electronic skin research. Quantitatively describing finger frictional behavior is always challenging, given the complex structure of the finger. In this study, the texture and sliding direction dependence of finger skin friction was quantified based on explicit mathematic models. The proposed double-layer model of finger skin effectively described the nonlinear elastic response of skin and predicted the scaling-law of effective elastic modulus with contact radius. Additionally, the skin friction model on textured surface considering adhesion and deformation factors was established. It revealed that adhesive term dominated finger friction behavior in daily life, and suggested that object texture size mainly influenced friction-induced vibrations rather than the average friction force. Combined with digital image correlation (DIC) technique, the effect of sliding direction on finger friction was analyzed. It was found that the anisotropy in finger friction was governed by the finger’s ratchet pawl structure, which also contributes to enhanced stick-slip vibrations in the distal sliding direction. The proposed friction models can offer valuable insights into the underlying mechanism of skin friction under various operating conditions, and can provide quantitative guidance for effectively encoding friction into haptics.

A miniaturized array microneedle tactile sensor for intelligent object recognition

Tactile sensors can provide intuitive information about the shape, size, and texture of objects, which helps robots make more accurate decisions. In this paper, based on the principle of triboelectrification and electrostatic induction, a miniature tactile sensor (MTS) for smart object recognition is designed using composite materials based on silica gel. Its diameter is only 6 mm, and the electrodes are arranged in a circumferential array, which has excellent shape adaptability and can recognize curved materials with curvature of 1/15 mm−1. By optimizing the sensor structure and process to obtain a microstructure with a spike-like shape, the sensitivity of the sensor is successfully improved, and the smallest and accurately detected surface material side length is 5 mm. In addition, the article provides a smart sensor system that can enhance the recognition function of robots with the help of machine learning technology, and the discrimination accuracy reaches 95.7%, which lays a solid foundation for practical applications.

Recasting Regional Lighting for Shadow Removal

Removing shadows requires an understanding of both lighting conditions and object textures in a scene. Existing methods typically learn pixel-level color mappings between shadow and non-shadow images, in which the joint modeling of lighting and object textures is implicit and inadequate. We observe that in a shadow region, the degradation degree of object textures depends on the local illumination, while simply enhancing the local illumination cannot fully recover the attenuated textures. Based on this observation, we propose to condition the restoration of attenuated textures on the corrected local lighting in the shadow region. Specifically, We first design a shadow-aware decomposition network to estimate the illumination and reflectance layers of shadow regions explicitly. We then propose a novel bilateral correction network to recast the lighting of shadow regions in the illumination layer via a novel local lighting correction module, and to restore the textures conditioned on the corrected illumination layer via a novel illumination-guided texture restoration module. We further annotate pixel-wise shadow masks for the public SRD dataset, which originally contains only image pairs. Experiments on three benchmarks show that our method outperforms existing state-of-the-art shadow removal methods. Project page in: yuhaoliu7456.github.io/RRL-Net.

MDFL: Multi-Domain Diffusion-Driven Feature Learning

High-dimensional images, known for their rich semantic information, are widely applied in remote sensing and other fields. The spatial information in these images reflects the object's texture features, while the spectral information reveals the potential spectral representations across different bands. Currently, the understanding of high-dimensional images remains limited to a single-domain perspective with performance degradation. Motivated by the masking texture effect observed in the human visual system, we present a multi-domain diffusion-driven feature learning network (MDFL) , a scheme to redefine the effective information domain that the model really focuses on. This method employs diffusion-based posterior sampling to explicitly consider joint information interactions between the high-dimensional manifold structures in the spectral, spatial, and frequency domains, thereby eliminating the influence of masking texture effects in visual models. Additionally, we introduce a feature reuse mechanism to gather deep and raw features of high-dimensional data. We demonstrate that MDFL significantly improves the feature extraction performance of high-dimensional data, thereby providing a powerful aid for revealing the intrinsic patterns and structures of such data. The experimental results on three multi-modal remote sensing datasets show that MDFL reaches an average overall accuracy of 98.25%, outperforming various state-of-the-art baseline schemes. Code available at https://github.com/LDXDU/MDFL-AAAI-24.

Skin-Inspired Multi-Modal Mechanoreceptors for Dynamic Haptic Exploration.

Active sensing is a fundamental aspect of human and animal interactions with the environment, providing essential information about the hardness, texture, and tackiness of objects. This ability stems from the presence of diverse mechanoreceptors in the skin, capable of detecting a wide range of stimuli and from the sensorimotor control of biological mechanisms. In contrast, existing tactile sensors for robotic applications typically excel in identifying only limited types of information, lacking the versatility of biological mechanoreceptors and the requisite sensing strategies to extract tactile information proactively. Here, inspired by human haptic perception, a skin-inspired artificial 3D mechanoreceptor (SENS) capable of detecting multiple mechanical stimuli is developed to bridge sensing and action in a closed-loop sensorimotor system for dynamic haptic exploration. A tensor-based non-linear theoretical model is established to characterize the 3D deformation (e.g., tensile, compressive, and shear deformation) of SENS, providing guidance for the design and optimization of multimode sensing properties with high fidelity. Based on SENS, a closed-loop robotic system capable of recognizing objects with improved accuracy (≈96%) is further demonstrated. This dynamic haptic exploration approach shows promise for a wide range of applications such as autonomous learning, healthcare, and space and deep-sea exploration.

Transformer network for stereo matching of weak texture objects

Transformer network for stereo matching of weak texture objects

Accurate 3D Measurement of Complex Texture Objects by Height Compensation Using a Dual-Projector Structure.

Fringe projection profilometry is a widely used technique for 3D measurement due to its high accuracy and speed. However, the accuracy significantly decreases when measuring complex texture objects, especially in the junction of different colors. This paper analyzes the causes of errors resulting from complex textures and proposes a height compensation method to revise the error by employing a dual-projector structure. Moreover, the dual-projector is capable of acquiring a pair of errors with opposite signs, which can be utilized to calculate the accurate 3D information after determining the ratio of this pair of errors. Experiments provide significant improvement in measuring complex texture objects, demonstrating the proposed method's ability.

TDDFusion: A Target-Driven Dual Branch Network for Infrared and Visible Image Fusion.

In the field of image fusion, the integration of infrared and visible images aims to combine complementary features into a unified representation. However, not all regions within an image bear equal importance. Target objects, often pivotal in subsequent decision-making processes, warrant particular attention. Conventional deep-learning approaches for image fusion primarily focus on optimizing textural detail across the entire image at a pixel level, neglecting the pivotal role of target objects and their relevance to downstream visual tasks. In response to these limitations, TDDFusion, a Target-Driven Dual-Branch Fusion Network, has been introduced. It is explicitly designed to enhance the prominence of target objects within the fused image, thereby bridging the existing performance disparity between pixel-level fusion and downstream object detection tasks. The architecture consists of a parallel, dual-branch feature extraction network, incorporating a Global Semantic Transformer (GST) and a Local Texture Encoder (LTE). During the training phase, a dedicated object detection submodule is integrated to backpropagate semantic loss into the fusion network, enabling task-oriented optimization of the fusion process. A novel loss function is devised, leveraging target positional information to amplify visual contrast and detail specific to target objects. Extensive experimental evaluation on three public datasets demonstrates the model's superiority in preserving global environmental information and local detail, outperforming state-of-the-art alternatives in balancing pixel intensity and maintaining the texture of target objects. Most importantly, it exhibits significant advantages in downstream object detection tasks.

A dynamic alignment and illumination‐aware convolution for shadow removal

AbstractShadow removal is a challenging task because the variety of shadows is influenced by surface texture and lighting. This paper proposes a dynamic alignment and illumination‐aware convolution (DAIC), which consists of a Feature Alignment Module (FAM) and a Dynamic Weight Module (DWM). FAM aligns the downsampled deep features with the original features and helps to extract the optimal local information to ensure that the object texture features are not corrupted. DWM generates weights according to different lighting variations for a better shadow removal result. The shadow removal approach is based on an image decomposition algorithm using a multi‐exposure image fusion model. Here, the shadow removal network and refinement network use U‐Net framework, and the transposed convolution operations are replaced with DAIC in the decoder part of U‐Net to improve the performance of the network. The experiments are conducted on two large shadow removal datasets, ISTD+ and SRD. Compared to state‐of‐the‐art methods, this model achieves optimal performance in terms of Root Mean Square Error (RMSE) for the non‐shadow region. It also achieves performance comparable to the state‐of‐the‐art method in terms of RMSE for the shadow region and structural similarity index measurement for the entire image.

Binary-1D/2D nanomaterial-functionalization toward strong, stretchable, and anti-freezing electrically conductive organohydrogels for self-powered operation monitoring of robotic hand

Machines and robots have become greatly significant in liberating human beings from heavy and repetitive physical work, which requires intelligent monitoring of their operation states. However, simultaneous realization of self-power, high mechanical strength, ultra-stretchability, and low-temperature tolerance for sensing devices remains challenging. Here, a triboelectric nanogenerator (TENG) based on anti-freezing, mechanically robust and electrically conductive organohydrogels is demonstrated via a binary-nanomaterial-functionalization and solvent-exchange approach. The organohydrogel exhibits high anti-freezing behaviors with maintained electrical conductivity (6.2 S/m) and mechanical properties (2.7 MPa and 820 %) at extreme subzero temperatures. Notably, molecular interactions and geometrical synergy of the nanophases enable the high operation durability of the TENG with at least 15,000 working cycles and a peak power density of ∼ 120 mW m−2 (loading resistance of 5 × 107 Ω and pressure of 40 N) in such harsh environments. Furthermore, the TENG is employed to create a monitoring system capable of real-time wirelessly transmitting electrical signals for identifying the sizes and textures of objects grasped by a robotic hand. Featuring these characteristics, this work may provide a new perspective for designing new strong yet tough and cold-tolerant materials for intelligent and self-powered human–machine interactive systems.

Solar potential on facades at urban scale: an integrated approach combining solar and digital building modelling

The paper presents an integrated approach to improve the solar radiation modelling on facades in large-scale built-up areas. The modelling of the built environment must first be improved in terms of level of detail. Thus, from aerial oblique images, a digital twin of the urban scene is created, allowing to process the facades as textured objects and to detect windows using an artificial intelligence image processing. Reflected radiation is significant on vertical surfaces, but complex to model on large areas. The developed model is based on a simplified radiosity approach, which reduces the volume of analysis, storage and thus the computation time, while producing reliable results. A demonstration of the integrated tool is presented for an urban area in Geneva (1 km2), including a solar energy balance assessment for one of the buildings using the results from the solar modelling. The perspective is to generalise the approach to a larger scale and to complete the solar cadastre of the roofs of the Greater Geneva area with the facades.