Appearance Model Research Articles

Automation of conceptual design and modification of aircraft type unmanned aerial vehicles using multidisciplinary optimization and evolutionary algorithms. Part 1: Methods and models

This paper proposes a method for selecting rational parameters for large-size aircraft-type unmanned aerial vehicles at the initial design stages using an optimization algorithm of differential evolution and numerical mathematical modeling of aerodynamic problems. The method assumes implementation of weight and aerodynamic balance in the main flight modes, it can consider aircraft-type unmanned aerial vehicles with one or two lifting surfaces, applies parallel calculations, and automatically generates a three-dimensional geometric model of the aircraft appearance based on the optimization results. A method for accelerating by more than three times the process of solving the problem of optimizing aircraft takeoff weight parameters by introducing the target function into the set of design variables is proposed and demonstrated. The results of assessing the reliability of the mathematical models used for aerodynamics and the correct calculation of the target function are presented, taking into account various constraints. A comprehensive check of the operability and effectiveness of the method were considered by solving demonstration problems by optimizing more than ten main design parameters of the appearance of two existing heavy-class unmanned aerial vehicles with known characteristics from open sources. Examples of using the optimization results to modify prototypes are provided.

Measuring concern about smile appearance among adults.

To adapt and estimate the psychometric properties of Utrecht Questionnaire for esthetic outcome assessment in rhinoplasty (OAR) to assess concern about smile appearance and to estimate the influence of demographic characteristics on this concern in adults. This was a cross-sectional observational study. Individuals aged between 18 and 40 years participated in the study. The Portuguese version of OAR was adapted for smile assessment in dental practice and was named Questionnaire for Outcome Assessment of Smile Aesthetic (OA-Smile). Data validity was estimated using factorial validity [confirmatory factor analysis (CFA)-CFI, Tucker-Lewis index (TLI), SRMR)] and convergent validity (average variance extracted). Reliability was assessed using the alpha ordinal coefficient (αordinal) and the omega coefficient (ω). A structural model was elaborated to assess the contribution of demographic characteristics to smile appearance concerns. Model fit was evaluated, and the z-test (α = 5%) was used to estimate the significance of the path estimates (β). 2.523 subjects participated in the study [mean age = 32.86 (SD = 11.39) years, 68.1% female]. The factor model of orofacial appearance (OA)-Smile presented an adequate fit to the data [CFA: comparative fit index (CFI) = 0.99, TLI = 0.99, SRMR ≤ 0.05]. Convergent validity (AVE ≥ 0.80) and reliability (αordinal and ω ≥ 0.85) were adequate. The structural model presented an acceptable fit (CFI = 0.974; TLI = 0.991 and SRMR = 0.053). Women, younger people, single people, people with lower income, people using dental prostheses, undergoing dental treatment, and those who do not like their smile were more concerned about their smile appearance. Nonprobability sampling, online data collection, and cross-sectional design are considered limitations of the study. OA-Smile is a suitable scale to assess smile appearance concerns, and the data obtained with this scale were valid and reliable. Demographic characteristics should be considered when measuring concerns about smile appearance.

Remote physiological signal recovery with efficient spatio-temporal modeling.

Contactless physiological signal measurement has great applications in various fields, such as affective computing and health monitoring. Physiological measurements based on remote photoplethysmography (rPPG) are realized by capturing the weak periodic color changes. The changes are caused by the variation in the light absorption of skin surface during systole and diastole stages of a functioning heart. This measurement mode has advantages of contactless measurement, simple operation, low cost, etc. In recent years, several deep learning-based rPPG measurement methods have been proposed. However, the features learned by deep learning models are vulnerable to motion and illumination artefacts, and are unable to fully exploit the intrinsic temporal characteristics of the rPPG. This paper presents an efficient spatiotemporal modeling-based rPPG recovery method for physiological signal measurements. First, two modules are utilized in the rPPG task: 1) 3D central difference convolution for temporal context modeling with enhanced representation and generalization capacity, and 2) Huber loss for robust intensity-level rPPG recovery. Second, a dual branch structure for both motion and appearance modeling and a soft attention mask are adapted to take full advantage of the central difference convolution. Third, a multi-task setting for joint cardiac and respiratory signals measurements is introduced to benefit from the internal relevance between two physiological signals. Last, extensive experiments performed on three public databases show that the proposed method outperforms prior state-of-the-art methods with the Pearson's correlation coefficient higher than 0.96 on all three datasets. The generalization ability of the proposed method is also evaluated by cross-database and video compression experiments. The effectiveness and necessity of each module are confirmed by ablation studies.

TriHuman: A Real-time and Controllable Tri-plane Representation for Detailed Human Geometry and Appearance Synthesis

Creating controllable, photorealistic, and geometrically detailed digital doubles of real humans solely from video data is a key challenge in Computer Graphics and Vision, especially when real-time performance is required. Recent methods attach a neural radiance field (NeRF) to an articulated structure, e.g., a body model or a skeleton, to map points into a pose canonical space while conditioning the NeRF on the skeletal pose. These approaches typically parameterize the neural field with a multi-layer perceptron (MLP) leading to a slow runtime. To address this drawback, we propose TriHuman a novel human-tailored, deformable, and efficient tri-plane representation, which achieves real-time performance, state-of-the-art pose-controllable geometry synthesis as well as photorealistic rendering quality. At the core, we non-rigidly warp global ray samples into our undeformed tri-plane texture space, which effectively addresses the problem of global points being mapped to the same tri-plane locations. We then show how such a tri-plane feature representation can be conditioned on the skeletal motion to account for dynamic appearance and geometry changes. Our results demonstrate a clear step toward higher quality in terms of geometry and appearance modeling of humans as well as runtime performance.

Advancing Photorealism Enhancement Using Convolutional Neural Networks (CNNs)

Photorealism has been the defining goal of computer graphics for half a century. A look at even the most sophisticated real-time games will quickly reveal that photorealism has not been achieved. An ineffable difference in the appearance of simulation and reality remains. In recent years, a complementary set of techniques has been developed in computer vision and machine learning. These techniques, based on deep learning, convolutional networks, and adversarial training, bypass physical modeling of geometric layout, material appearance, and light transport. Instead, images are synthesized by convolutional networks trained on large datasets. These techniques have been used to synthesize representative images from a given domain to convert semantic label maps to photographic images and to attempt to bridge the appearance gap between synthetic and real images. The images are enhanced by a convolutional network that leverages intermediate representations produced by conventional rendering pipelines. The network is trained via a novel adversarial objective, which provides strong supervision at multiple perceptual levels.

3D bone shape from CT-scans provides an objective measure of osteoarthritis severity: Data from the IMI-APPROACH study

Decisions regarding total knee arthroplasty are usually made using a patient's own assessment of pain and the structural disposition of the joint as seen on plain film radiographs. Pain severity can fluctuate, and radiographs can be misleading, with the apparent joint status affected by anatomical orientation. An important component of the surgical management of knee osteoarthritis (OA) is the timing of surgical intervention: knee arthroplasty performed too early in the course of the disease may increase the need for revision surgery.Femoral 3D bone shape (B-score) from MR images is an objective measure of OA severity and has been correlated with current and future risk of pain. We aimed to derive the B-score from CT images and compare it against the B-score derived from MR images.We used baseline and 24-month image data from the IMI-APPROACH 2-year prospective cohort study, comprising pairs of CT and MR images taken for each subject-timepoint. The femur was automatically segmented in both CT and MR modalities using an active appearance model, a machine-learning method, to measure the B-score. Linear regression was used to test for correlation between measures. Limits of agreement and bias were tested using Bland-Altman analysis.CT-MR pairs of the same knee were available from 424 participants (78 % women). B-scores from CT and MR were strongly correlated (CCC = 0.980) with negligible bias of 0.0106 (95 % CI: −0. 0281, +0.0493).The strong correlation and small B-score bias suggests that B-scores may be measured reliably using CT images. Since CT images are used in planning robot-assisted knee arthroplasty, with further study B-scores derived from CT surgical planning images could in principle provide a useful objective input to deciding the appropriateness, timing and type of knee arthroplasty.

Assessment of age-dependent sexual dimorphism in paediatric vertebral size and density using a statistical shape and statistical appearance modelling approach

This work focuses on the growth patterns of the human fourth lumbar vertebra (L4) in a paediatric population, with specific attention to sexual dimorphism. The study aims to understand morphological and density changes in the vertebrae through age-dependent statistical shape and statistical appearance models, which can describe full three-dimensional anatomy. Results show that the main growth patterns are associated with isotropic volumetric vertebral growth, a decrease in the relative size of the vertebral foramen, and an increase in the length of the transverse processes. Moreover, significant sexual dimorphism was demonstrated during puberty. We observe significant age and sex interaction in the anterior vertebral body height (P = 0.005), where females exhibited an earlier increase in rates of vertebral height evolution. Moreover, we also observe an increase in cross-sectional area (CSA) with age (P = 0.020), where the CSA is smaller in females than in males (significant sex effect P = 0.042). Finally, although no significant increase in trabecular bone density with age is observed (P = 0.363), a trend in the statistical appearance model suggests an increase in density with age.

NEFELI: A deep-learning detection and tracking pipeline for enhancing autonomy in advanced air mobility

Efficient detection and accurate collision estimation for non-cooperative aerial vehicles are crucial for the realization of fully autonomous aircraft and Advanced Air Mobility (AAM). This paper introduces NEFELI, a machine learning software utilizing optical sensors to detect and track non-cooperative aerial vehicles. NEFELI's detector employs an enhanced YOLOv5-large model, strengthened with a sliced inference step to enhance detection capabilities for distant, diminutive objects. Furthermore, NEFELI introduces several innovations in its tracking component. A key advancement lies in the creation and utilization of the first large-scale re-identification (Re-ID) dataset of aerial objects. This dataset is used to train the deep learning appearance (Re-ID) model of the tracking module and integrates appearance information into the detection and tracking pipeline, resulting in more robust and reliable tracking performance. Moreover, the tracking model combines the deep learning appearance model with a Kalman Filter-based motion model to address the challenge of precisely tracking distant aerial objects. Notably, an extensive comparative analysis that was conducted showed that NEFELI outperforms state-of-the-art detection and tracking models in terms of Higher Order Tracking Accuracy (HOTA) metric, ID switches, and Association Accuracy (AssA) by a wide margin. A crucial aspect of this work is NEFELI's software architecture design, which enables efficient implementation on a low SWaP (Size, Weight, and Power) edge Graphic Processing Unit (GPU). To further showcase NEFELI's generalization capabilities and edge implementation performance, real-world flight experiments with small UAVs were carried out. The experimental results demonstrate NEFELI's ability to detect and track small UAVs at distances of up to 145 m in real-time speed of 6.7 fps.

Correlation-Embedded Transformer Tracking: A Single-Branch Framework.

Developing robust and discriminative appearance models has been a long-standing research challenge in visual object tracking. In the prevalent Siamese-based paradigm, the features extracted by the Siamese-like networks are often insufficient to model the tracked targets and distractor objects, thereby hindering them from being robust and discriminative simultaneously. While most Siamese trackers focus on designing robust correlation operations, we propose a novel single-branch tracking framework inspired by the transformer. Unlike the Siamese-like feature extraction, our tracker deeply embeds cross-image feature correlation in multiple layers of the feature network. By extensively matching the features of the two images through multiple layers, it can suppress non-target features, resulting in target-aware feature extraction. The output features can be directly used to predict target locations without additional correlation steps. Thus, we reformulate the two-branch Siamese tracking as a conceptually simple, fully transformer-based Single-Branch Tracking pipeline, dubbed SBT. After conducting an in-depth analysis of the SBT baseline, we summarize many effective design principles and propose an improved tracker dubbed SuperSBT. SuperSBT adopts a hierarchical architecture with a local modeling layer to enhance shallow-level features. A unified relation modeling is proposed to remove complex handcrafted layer pattern designs. SuperSBT is further improved by masked image modeling pre-training, integrating temporal modeling, and equipping with dedicated prediction heads. Thus, SuperSBT outperforms the SBT baseline by 4.7%,3.0%, and 4.5% AUC scores in LaSOT, TrackingNet, and GOT-10K. Notably, SuperSBT greatly raises the speed of SBT from 37 FPS to 81 FPS. Extensive experiments show that our method achieves superior results on eight VOT benchmarks. Code is available at https://github.com/phiphiphi31/SBT.

Spatial attention inference model for cascaded siamese tracking with dynamic residual update strategy

Target representation is crucial for visual tracking. Most Siamese-based trackers try their best to establish target models by using various deep networks. However, they neglect the exploration of correlation among features, which leads to the inability to learn more representative features. In this paper, we propose a spatial attention inference model for cascaded Siamese tracking with dynamic residual update strategy. First, a spatial attention inference model is constructed. The model fuses interlayer multi-scale features generated by dilation convolution to enhance the spatial representation ability of features. On this basis, we use self-attention to capture interaction between target and context, and use cross-attention to aggregate interdependencies between target and background. The model infers potential feature information by exploiting the correlations among features for building better appearance models. Second, a cascaded localization-aware network is introduced to bridge a gap between classification and regression. We propose an alignment-aware branch to resample and learn object-aware features from the predicted bounding boxes for obtaining localization confidence, which is used to correct the classification confidence by weighted integration. This cascaded strategy alleviates the misalignment problem between classification and regression. Finally, a dynamic residual update strategy is proposed. This strategy utilizes the Context Fusion Network (CFNet) to fuse the templates of historical and current frames to generate the optimal templates. Meanwhile, we use a dynamic threshold function to determine when to update by judging the tracking results. The strategy uses temporal context to fully explore the intrinsic properties of the target, which enhances the adaptability to changes in the target’s appearance. We conducted extensive experiments on seven tracking benchmarks, including OTB100, UAV123, TC128, VOT2016, VOT2018, GOT10k and LaSOT, to validate the effectiveness of our proposed algorithm.

Practical Appearance Model for Foundation Cosmetics

AbstractCosmetic products have found their place in various aspects of human life, yet their digital appearance reproduction has received little attention. We present an appearance model for cosmetics, in particular for foundation layers, that reproduces a range of existing appearances of foundation cosmetics: from a glossy to a matte to an almost velvety look. Our model is a multilayered BSDF that reproduces the stacking of multiple layers of cosmetics. Inspired by the microscopic particulates used in cosmetics, we model each individual layer as a stochastic participating medium with two types of scatterers that mimic the most prominent visual features of cosmetics: spherical diffusers, resulting in a uniform distribution of radiance; and platelets, responsible for the glossy look of certain cosmetics. We implement our model on top of the position‐free Monte Carlo framework, that allows us to include multiple scattering. We validate our model against measured reflectance data, and demonstrate the versatility and expressiveness of our model by thoroughly exploring the range of appearances that it can produce.

Neural Appearance Model for Cloth Rendering

AbstractThe realistic rendering of woven and knitted fabrics has posed significant challenges throughout many years. Previously, fiber‐based micro‐appearance models have achieved considerable success in attaining high levels of realism. However, rendering such models remains complex due to the intricate internal scatterings of hundreds of fibers within a yarn, requiring vast amounts of memory and time to render. In this paper, we introduce a new framework to capture aggregated appearance by tracing many light paths through the underlying fiber geometry. We then employ lightweight neural networks to accurately model the aggregated BSDF, which allows for the precise modeling of a diverse array of materials while offering substantial improvements in speed and reductions in memory. Furthermore, we introduce a novel importance sampling scheme to further speed up the rate of convergence. We validate the efficacy and versatility of our framework through comparisons with preceding fiber‐based shading models as well as the most recent yarn‐based model.

From microfacets to participating media: A unified theory of light transport with stochastic geometry

Stochastic geometry models have enjoyed immense success in graphics for modeling interactions of light with complex phenomena such as participating media, rough surfaces, fibers, and more. Although each of these models operates on the same principle of replacing intricate geometry by a random process and deriving the average light transport across all instances thereof, they are each tailored to one specific application and are fundamentally distinct. Each type of stochastic geometry present in the scene is firmly encapsulated in its own appearance model, with its own statistics and light transport average, and no cross-talk between different models or deterministic and stochastic geometry is possible. In this paper, we derive a theory of light transport on stochastic implicit surfaces , a geometry model capable of expressing deterministic geometry, microfacet surfaces, participating media, and an exciting new continuum in between containing aggregate appearance, non-classical media, and more. Our model naturally supports spatial correlations , missing from most existing stochastic models. Our theory paves the way for tractable rendering of scenes in which all geometry is described by the same stochastic model, while leaving ample future work for developing efficient sampling and rendering algorithms.

Binary Opacity Grids: Capturing Fine Geometric Detail for Mesh-Based View Synthesis

While surface-based view synthesis algorithms are appealing due to their low computational requirements, they often struggle to reproduce thin structures. In contrast, more expensive methods that model the scene's geometry as a volumetric density field (e.g. NeRF) excel at reconstructing fine geometric detail. However, density fields often represent geometry in a "fuzzy" manner, which hinders exact localization of the surface. In this work, we modify density fields to encourage them to converge towards surfaces, without compromising their ability to reconstruct thin structures. First, we employ a discrete opacity grid representation instead of a continuous density field, which allows opacity values to discontinuously transition from zero to one at the surface. Second, we anti-alias by casting multiple rays per pixel, which allows occlusion boundaries and subpixel structures to be modelled without using semi-transparent voxels. Third, we minimize the binary entropy of the opacity values, which facilitates the extraction of surface geometry by encouraging opacity values to binarize towards the end of training. Lastly, we develop a fusion-based meshing strategy followed by mesh simplification and appearance model fitting. The compact meshes produced by our model can be rendered in real-time on mobile devices and achieve significantly higher view synthesis quality compared to existing mesh-based approaches. Our interactive webdemo is available at https://binary-opacity-grid.github.io.

Colour difference detection algorithm for warp‐knitted fabric based on image colour appearance model

AbstractDetecting colour differences in warp‐knitting fabric is essential to ensure its quality. Machine vision algorithms are commonly used for automatic colour difference detection. However, the current algorithm directly extracts RGB (red, green, blue) colour stimulus values from the image without considering the image's colour attributes, leading to a significant error in colour difference detection. To solve this issue, a colour appearance model is introduced into the field of colour difference detection for warp‐knitted fabric. Initially, a colour appearance model based on CAM16 is established, and the critical parameters are calculated to obtain the colour appearance attribute of the fabric. The colour difference calculation formula is then constructed in the uniform colour space of CAM16‐UCS. A template selection algorithm based on principal component analysis was designed to select warp‐knitted cloth images with standard colours. Subsequently, a cloth colour difference detection algorithm was developed using the cloth colour profile model. The performance of the colour difference formula based on the colour profile model was evaluated using the PF/3 method. To compare the CIELab, CMC(2:1), and CIEDE2000 colour difference formulas, standardised residual sum of squares was used. The results indicated that the colour difference formula based on the colour‐appearance model is about 5.32% different from the visual colour difference perceived by the human eye. However, it can perform as well as the CMC(2:1) colour difference formula, which is widely used in the textile industry.

Correlation filter based single object tracking: A review

In recent years, correlation filter-based (CF) tracking algorithms have gained momentum in the field of visual tracking. CF tracking algorithms have achieved compelling performance by addressing its limitations such as boundary effect and filter corruption during various tracking and target appearance variations. Many researchers have attempted to provide better efficiency and tracking results by extracting handcrafted and deep features either from vision sensors or specialized sensors in the CF tracking framework. Handcrafted features are integrated with deep features, thermal features, and depth features to prevent tracking failures during dense tracking challenges. To provide a detailed understanding of CF-based trackers, the tracking algorithms are categorized either as kernelized CF trackers or fusion-based CF trackers in this work. This is the first review of its kind, which categorizes various correlation-based tracking algorithms based on the key methodologies and the exploited features in the appearance model. Under each category, salient features, detailed overview, and current advancement are discussed and tabulated to provide future research directions. In addition, the performance of CF-based state-of-the-art is experimentally evaluated on multiple datasets namely, OTB100 and VOT2017 under tough tracking situations.

Real-time Neural Appearance Models

We present a complete system for real-time rendering of scenes with complex appearance previously reserved for offline use. This is achieved with a combination of algorithmic and system level innovations. Our appearance model utilizes learned hierarchical textures that are interpreted using neural decoders, which produce reflectance values and importance-sampled directions. To best utilize the modeling capacity of the decoders, we equip the decoders with two graphics priors. The first prior—transformation of directions into learned shading frames—facilitates accurate reconstruction of mesoscale effects. The second prior—a microfacet sampling distribution—allows the neural decoder to perform importance sampling efficiently. The resulting appearance model supports anisotropic sampling and level-of-detail rendering, and allows baking deeply layered material graphs into a compact unified neural representation. By exposing hardware accelerated tensor operations to ray tracing shaders, we show that it is possible to inline and execute the neural decoders efficiently inside a real-time path tracer. We analyze scalability with increasing number of neural materials and propose to improve performance using code optimized for coherent and divergent execution. Our neural material shaders can be over an order of magnitude faster than non-neural layered materials. This opens up the door for using film-quality visuals in real-time applications such as games and live previews.

A Hierarchical Architecture for Neural Materials

AbstractNeural reflectance models are capable of reproducing the spatially‐varying appearance of many real‐world materials at different scales. Unfortunately, existing techniques such as NeuMIP have difficulties handling materials with strong shadowing effects or detailed specular highlights. In this paper, we introduce a neural appearance model that offers a new level of accuracy. Central to our model is an inception‐based core network structure that captures material appearances at multiple scales using parallel‐operating kernels and ensures multi‐stage features through specialized convolution layers. Furthermore, we encode the inputs into frequency space, introduce a gradient‐based loss, and employ it adaptive to the progress of the learning phase. We demonstrate the effectiveness of our method using a variety of synthetic and real examples.

Correlation Filters for UAV Online Tracking Based on Complementary Appearance Model and Reversibility Reasoning

Correlation Filters for UAV Online Tracking Based on Complementary Appearance Model and Reversibility Reasoning

Self-supervised learning for RGB-D object tracking

Recently, there has been a growing interest in RGB-D object tracking thanks to its promising performance achieved by combining visual information with auxiliary depth cues. However, the limited volume of annotated RGB-D tracking data for offline training has hindered the development of a dedicated end-to-end RGB-D tracker design. Consequently, the current state-of-the-art RGB-D trackers mainly rely on the visual branch to support the appearance modelling, with the depth map utilised for elementary information fusion or failure reasoning of online tracking. Despite the achieved progress, the current paradigms for RGB-D tracking have not fully harnessed the inherent potential of depth information, nor fully exploited the synergy of vision-depth information. Considering the availability of ample unlabelled RGB-D data and the advancement in self-supervised learning, we address the problem of self-supervised learning for RGB-D object tracking. Specifically, an RGB-D backbone network is trained on unlabelled RGB-D datasets using masked image modelling. To train the network, the masking mechanism creates a selective occlusion of the input visible image to force the corresponding aligned depth map to help with discerning and learning vision-depth cues for the reconstruction of the masked visible image. As a result, the pre-trained backbone network is capable of cooperating with crucial visual and depth features of the diverse objects and background in the RGB-D image. The intermediate RGB-D features output by the pre-trained network can effectively be used for object tracking. We thus embed the pre-trained RGB-D network into a transformer-based tracking framework for stable tracking. Comprehensive experiments and the analysis of the results obtained on several RGB-D tracking datasets demonstrate the effectiveness and superiority of the proposed RGB-D self-supervised learning framework and the following tracking approach.