Graphics Applications Research Articles

A COMPREHENSIVE IMPLEMENTATION ALGORITHM 2D TRANSFORMATION IN GRAPHIC WEB-EDITORS BASED ON THE METHODS OF AFFINE TRANSFORMATIONS

This article presents a complex algorithm for the optimized implementation of 2D transformations in graphic web editors based on the methods of affine transformations. The object of this study is the algorithms and methods of affine transformations for the implementation of 2D transformations in graphic web editors. The research is aimed at applying the principles of these transformations and their impact on graphic objects, as well as at finding effective solutions for their implementation. An analysis of the main existing algorithms and methods for implementing 2D transformations, such as the Bresenham and Wu algorithms for visualizing straight and curved lines, algorithms for filling areas using affine transformation methods, was carried out. Therefore, the issue of improving and simplifying algorithms and methods of affine transformations for implementing 2D transformations in graphic web editors is relevant and important for modern web design and development. The work considers the main algorithms of affine transformations that ensure the implementation of scaling, rotation, shift, skew functions and their interaction with graphic objects. The proposed algorithm includes the main advantages of implementing the above functions and allows you to create more efficient and high-quality graphic applications, providing at the same time greater functionality and productivity. Applying such an approach creates new opportunities for the development and use of new types of web editors in the field of web design. Based on the proposed algorithm, a web application has been developed that allows you to create various transformations and animations of figures on the plane. The results of the conducted testing confirm the effectiveness and possibility of applying the proposed approach in real projects for the development of graphic web editors for working with 2D transformations.

Interaction of Asymmetric Adaptive Network Structures and Parameter Balance in Image Feature Extraction and Recognition

To better process irregular sample images for their image feature extraction and recognition, this essay proposes asymmetric adaptive neural network (AACNN) structures, including dual structures of an adaptive image feature extraction network (AT-CNN) and adaptive image recognition network (AT-ACNN). They both comprise an Adaptive Transform (AT) module and a deep learning network, but the ACNN comprises pixel-adaptive convolutional (PAC) kernels that CNN does not have, reflecting the asymmetry of these network structures. Structural analysis and comparative testing experiments indicated that the proposed method is more appropriate and effective for dealing with irregular sample images with different sizes and views, mainly focusing on their feature extraction accuracy and image recognition efficiency. The proposed method constructs the interaction between asymmetric dual network structures, essential in improving model performance and efficiency. It specifically manifests that the PAC kernels in an ACNN resolves the problem of content-agnostic convolution in image recognition by learning image features from a pre-trained CNN. On the other hand, it improves image recognition efficiency by using feature maps extracted from the pre-trained CNN to train the classifiers in the ACNN. We also found that parameter balance was essential in adaptive neural network structure for better performance in further testing experiments. When setting the Dropout layer parameter at 0.5 and the iteration number was 32, the proposed model achieved adequate recognition accuracy and efficiency. Smaller parameters affect model performance, but more extensive parameters significantly increase computational burden and loss. Comparative testing experiments fully validated its superiority compared with traditional methods based on CNNs. Using traditional carving patterns from Anhui Province as an example, we conducted image recognition and feature graphic application under ideal parameter balance conditions and thereby demonstrated the practicality and value of the proposed method.

Entity-Driven New Paradigm of Mine Data: Model Construction and Application

In the construction of a three-dimensional real scene (3D real scene) of a mine, we encounter challenges in organizing and correlating multi-source data. To surmount these challenges, we innovatively adopted the entity organization method. We proposed a definition of a “mining entity” and conducted an in-depth analysis of its main characteristics. Based on these characteristics, we classified and coded the main mining entities and further constructed an entity expression pattern layer that could fully reflect the characteristics of the mine. Based on multi-source heterogeneous mine data, we constructed a mining entity data layer. Furthermore, by adopting a graph database approach, we were able to create a graphical representation and correlation application of the complex relationships among mining entities. This work shows that compared to traditional data management methods, a mine data organization approach centered on entities can more effectively integrate and correlate multi-source mine data, providing strong support for the digital management of mines and the construction of 3D real scenes.

The text2term tool to map free-text descriptions of biomedical terms to ontologies.

There is an ongoing need for scalable tools to aid researchers in both retrospective and prospective standardization of discrete entity types-such as disease names, cell types, or chemicals-that are used in metadata associated with biomedical data. When metadata are not well-structured or precise, the associated data are harder to find and are often burdensome to reuse, analyze, or integrate with other datasets due to the upfront curation effort required to make the data usable-typically through retrospective standardization and cleaning of the (meta)data. With the goal of facilitating the task of standardizing metadata-either in bulk or in a one-by-one fashion, e.g. to support autocompletion of biomedical entities in forms-we have developed an open-source tool called text2term that maps free-text descriptions of biomedical entities to controlled terms in ontologies. The tool is highly configurable and can be used in multiple ways that cater to different users and expertise levels-it is available on Python Package Index and can be used programmatically as any Python package; it can also be used via a command-line interface or via our hosted, graphical user interface-based web application or by deploying a local instance of our interactive application using Docker. Database URL: https://pypi.org/project/text2term.

Multi-Resolution Real-Time Deep Pose-Space Deformation

We present a hard-real-time multi-resolution mesh shape deformation technique for skeleton-driven soft-body characters. Producing mesh deformations at multiple levels of detail is very important in many applications in computer graphics. Our work targets applications where the multi-resolution shapes must be generated at fast speeds ("hard-real-time", e.g., a few milliseconds at most and preferably under 1 millisecond), as commonly needed in computer games, virtual reality and Metaverse applications. We assume that the character mesh is driven by a skeleton, and that high-quality character shapes are available in a set of training poses originating from a high-quality (slow) rig such as volumetric FEM simulation. Our method combines multi-resolution analysis, mesh partition of unity, and neural networks, to learn the pre-skinning shape deformations in an arbitrary character pose. Combined with linear blend skinning, this makes it possible to reconstruct the training shapes, as well as interpolate and extrapolate them. Crucially, we simultaneously achieve this at hard real-time rates and at multiple mesh resolution levels. Our technique makes it possible to trade deformation quality for memory and computation speed, to accommodate the strict requirements of modern real-time systems. Furthermore, we propose memory layout and code improvements to boost computation speeds. Previous methods for realtime approximations of quality shape deformations did not focus on hard real-time, or did not investigate the multi-resolution aspect of the problem. Compared to a "naive" approach of separately processing each hierarchical level of detail, our method offers a substantial memory reduction as well as computational speedups. It also makes it possible to construct the shape progressively level by level and interrupt the computation at any time, enabling graceful degradation of the deformation detail. We demonstrate our technique on several examples, including a stylized human character, human hands, and an inverse-kinematics-driven quadruped animal.

3D Gaussian Ray Tracing: Fast Tracing of Particle Scenes

Particle-based representations of radiance fields such as 3D Gaussian Splatting have found great success for reconstructing and re-rendering of complex scenes. Most existing methods render particles via rasterization, projecting them to screen space tiles for processing in a sorted order. This work instead considers ray tracing the particles, building a bounding volume hierarchy and casting a ray for each pixel using high-performance GPU ray tracing hardware. To efficiently handle large numbers of semi-transparent particles, we describe a specialized rendering algorithm which encapsulates particles with bounding meshes to leverage fast ray-triangle intersections, and shades batches of intersections in depth-order. The benefits of ray tracing are well-known in computer graphics: processing incoherent rays for secondary lighting effects such as shadows and reflections, rendering from highly-distorted cameras common in robotics, stochastically sampling rays, and more. With our renderer, this flexibility comes at little cost compared to rasterization. Experiments demonstrate the speed and accuracy of our approach, as well as several applications in computer graphics and vision. We further propose related improvements to the basic Gaussian representation, including a simple use of generalized kernel functions which significantly reduces particle hit counts.

MV2MV: Multi-View Image Translation via View-Consistent Diffusion Models

Image translation has various applications in computer graphics and computer vision, aiming to transfer images from one domain to another. Thanks to the excellent generation capability of diffusion models, recent single-view image translation methods achieve realistic results. However, directly applying diffusion models for multi-view image translation remains challenging for two major obstacles: the need for paired training data and the limited view consistency. To overcome the obstacles, we present a first unified multi-view image to multi-view image translation framework based on diffusion models, called MV2MV. Firstly, we propose a novel self-supervised training strategy that exploits the success of off-the-shelf single-view image translators and the 3D Gaussian Splatting (3DGS) technique to generate pseudo ground truths as supervisory signals, leading to enhanced consistency and fine details. Additionally, we propose a latent multi-view consistency block, which utilizes the latent-3DGS as the underlying 3D representation to facilitate information exchange across multi-view images and inject 3D prior into the diffusion model to enforce consistency. Finally, our approach simultaneously optimizes the diffusion model and 3DGS to achieve a better trade-off between consistency and realism. Extensive experiments across various translation tasks demonstrate that MV2MV outperforms task-specific specialists in both quantitative and qualitative.

Evaluating Visual Perception of Object Motion in Dynamic Environments

Precisely understanding how objects move in 3D is essential for broad scenarios such as video editing, gaming, driving, and athletics. With screen-displayed computer graphics content, users only perceive limited cues to judge the object motion from the on-screen optical flow. Conventionally, visual perception is studied with stationary settings and singular objects. However, in practical applications, we---the observer---also move within complex scenes. Therefore, we must extract object motion from a combined optical flow displayed on screen, which can often lead to mis-estimations due to perceptual ambiguities. We measure and model observers' perceptual accuracy of object motions in dynamic 3D environments, a universal but under-investigated scenario in computer graphics applications. We design and employ a crowdsourcing-based psychophysical study, quantifying the relationships among patterns of scene dynamics and content, and the resulting perceptual judgments of object motion direction. The acquired psychophysical data underpins a model for generalized conditions. We then demonstrate the model's guidance ability to significantly enhance users' understanding of task object motion in gaming and animation design. With applications in measuring and compensating for object motion errors in video and rendering, we hope the research establishes a new frontier for understanding and mitigating perceptual errors caused by the gap between screen-displayed graphics and the physical world.

GaussianHeads: End-to-End Learning of Drivable Gaussian Head Avatars from Coarse-to-fine Representations

Real-time rendering of human head avatars is a cornerstone of many computer graphics applications, such as augmented reality, video games, and films, to name a few. Recent approaches address this challenge with computationally efficient geometry primitives in a carefully calibrated multi-view setup. Albeit producing photorealistic head renderings, they often fail to represent complex motion changes, such as the mouth interior and strongly varying head poses. We propose a new method to generate highly dynamic and deformable human head avatars from multi-view imagery in real time. At the core of our method is a hierarchical representation of head models that can capture the complex dynamics of facial expressions and head movements. First, with rich facial features extracted from raw input frames, we learn to deform the coarse facial geometry of the template mesh. We then initialize 3D Gaussians on the deformed surface and refine their positions in a fine step. We train this coarse-to-fine facial avatar model along with the head pose as learnable parameters in an end-to-end framework. This enables not only controllable facial animation via video inputs but also high-fidelity novel view synthesis of challenging facial expressions, such as tongue deformations and fine-grained teeth structure under large motion changes. Moreover, it encourages the learned head avatar to generalize towards new facial expressions and head poses at inference time. We demonstrate the performance of our method with comparisons against the related methods on different datasets, spanning challenging facial expression sequences across multiple identities. We also show the potential application of our approach by demonstrating a cross-identity facial performance transfer application. We make the code available on our project page.

NeuroClick: Advanced Software for Designing Blood-Brain Barrier-Permeable Drugs Using Click Reaction Simulations.

NeuroClick is a software tool designed for the in silico execution of azide-alkyne cycloaddition reactions, commonly known as click reactions. We developed this graphical user interface application to expedite the drug discovery process by generating libraries of 1,2,3-triazole compounds. NeuroClick enables users to input reagent SMILES strings, rapidly generating and screening extensive combinatorial libraries at a pace of 10,000 molecules per minute. The software applies stringent criteria to ensure the relevance and accuracy of the generated compounds, excluding molecules without azide groups or those with multiple reactive functional groups to maintain dataset integrity. NeuroClick incorporates advanced filtering options based on Lipinski's rule of five and blood-brain barrier (BBB) permeability predictors, allowing researchers to identify drug-like molecules with potential central nervous system activity. The software's high-throughput and user-friendly interface significantly enhance the efficiency of early-stage drug development by facilitating the exploration of vast chemical spaces and identifying promising lead compounds for further development. This article provides comprehensive guidance on the installation, usage, and features of NeuroClick, ensuring that users can leverage its full potential in their research endeavors. © 2024 Wiley Periodicals LLC. Basic Protocol: NeuroClick workflow for generating BBB-permeating drugs.

129Xe Image Processing Pipeline: An open-source, graphical user interface application for the analysis of hyperpolarized 129Xe MRI.

Hyperpolarized 129Xe MRI presents opportunities to assess regional pulmonary microstructure and function. Ongoing advancements in hardware, sequences, and image processing have helped it become increasingly adopted for both research and clinical use. As the number of applications and users increase, standardization becomes crucial. To that end, this study developed an executable, open-source 129Xe image processing pipeline (XIPline) to provide a user-friendly, graphical user interface-based analysis pipeline to analyze and visualize 129Xe MR data, including scanner calibration, ventilation, diffusion-weighted, and gas exchange images. The customizable XIPline is designed in MATLAB to analyze data from all three major scanner platforms. Calibration data is processed to calculate optimal flip angle and determine129Xe frequency offset. Data processing includes loading, reconstructing, registering, segmenting, and post-processing images. Ventilation analysis incorporates three common algorithms to calculate ventilation defect percentage and novel techniques to assess defect distribution and ventilation texture. Diffusion analysis features ADC mapping, modified linear binning to account for ADC age-dependence, and common diffusion morphometry methods. Gas exchange processing uses a generalized linear binning for data acquired using 1-point Dixon imaging. The XIPline workflow is demonstrated using analysis from representative calibration, ventilation, diffusion, and gas exchange data. The application will reduce redundant effort when implementing new techniques across research sites by providing an open-source framework for developers. In its current form, it offers a robust and adaptable platform for 129Xe MRI analysis to ensure methodological consistency, transparency, and support for collaborative research across multiple sites and MRI manufacturers.

МЕТОД КЛАСИФІКАЦІЇ ТЕКСТІВ ЗА ВМІСТОМ ПРОПАГАНДИ НЕЙРОМЕРЕЖЕВИМИ МОДЕЛЯМИ ГЛИБОКОГО НАВЧАННЯ

The method for classifying texts by propaganda content by neural network models of deep learning is proposed, based on combining traditional recurrent neural networks with long-term memory with transformers, which can provide a deeper understanding of sequence and context in text content. The peculiarity of proposed method is that it allows detecting both explicit and hidden propaganda messages, based on combining the capabilities of traditional recurrent neural networks with long-term memory and neural networks-transformers, as well as using the mechanism of training text data augmentation, which allows expand the number of training samples. To evaluate the effectiveness of developed method of classifying texts by the propaganda content using deep learning neural network models, the software implementation was created, which consists of notebooks implemented in the cloud service "Google Colab" and the application for user interaction with the model. Notebooks are used to train the hybrid architecture neural network model and to expand the obtained data set by method of text augmentation. The graphical user interface application developed by Python using the PyCharm development environment. The dataset of more than 25,000 records was created to train the neural network and the corresponding software was developed to investigate the method effectiveness. It was established that with the use of augmentation, better performance is achieved with larger number of epochs, which is explained by the expansion of the training sample, which leads to the need for a larger number of epochs. At the same time, when using augmentation, it was possible to achieve an accuracy of 97.83%, while without augmentation this indicator reached the maximum level of 96.94%. The obtained results demonstrate the ability of the proposed method to effectively classify texts based on the content of propaganda by neural network models of deep learning, and the use of the additional category "suspicious text" made it possible to raise the Precision and Recall indicators, which in turn makes it possible to automate the moderation of texts on the subject of propaganda with errors of no more than 1.83 % for false propaganda detection.

DIGITAL METHOD FOR PROCESSING EVENTS OF (γ,np)-REACTIONS ON 12C, 14N, AND 16O NUCLEI, OBTAINED USING BY TRACK 4π-DETECTOR

To study (γ,np)-reactions on the nuclei of 12C, 14N, and 16O, an algorithm-sequence of specific actions has been developed to obtain physical information about events using digital methods. A set of corresponding graphical applications has been created in the Python programming language, which allows one specialist to perform all procedures to obtain physical information regarding the studied reactions on a computer. An automated method for determining the coordinates of points along the track has been developed, significantly speeding up the process of measuring tracks. A comparison of kinematic parameters obtained by the digital methods and with the help of specialized measuring devices has been made. It has been found that the digital method improves the quality of measurement and reduces the absolute measurement error.

Application and integration of computer vision technologies for automated recognition and recording of chemical experiments

Abstract Advancements in computer vision have significantly improved motion and object recognition accuracy. These advancements should aid the automatic recognition of chemical experiments, potentially contributing to the recording of experiments. Creating an electronic laboratory notebook from experiment filming enhances convenience and allows more detailed information storage compared to traditional manual recording methods. Our previous research focused on employing object detection and action recognition to automate the recognition of chemical experiments. This paper presents a novel system that combines object detection, action recognition, multiple object tracking, and barcode recognition to automatically generate experimental flowcharts. We implemented our system as a graphical user interface-based application for laboratory use that successfully constructs flowcharts from videos of chemical experiments, including simple chemical manipulations.

Wireless image transfer by various antennas using transmitter and receiver modules at 5.8 GHz

Abstract This research proposes an inexpensive technique for wireless image transfer for security and surveillance applications. The technique uses a 5.8 GHz transmitter and receiver module, along with external antennas in the real-time image transfer within a radius of 100 m. The transferred images are stored in a laptop using a Python code-based graphical user interface application. Different antennas, dipole, circular split-ring resonators, hexagonal split-ring resonators, and metamaterial antennas are utilized for comparison. The Blind/Referenceless Image Spatial Quality Evaluator method is used to assess the picture quality of transferred images to quantify image transfer performance when no ground truth or reference photos are supplied. According to the presented results, images transferred using metamaterial antennas have higher quality than those transferred with other types of antennas. For security considerations, such a system can communicate and store the images in real time.

Compiler Support for Sparse Tensor Convolutions

This paper extends prior work on sparse tensor algebra compilers to generate asymptotically efficient code for tensor expressions with affine subscript expressions. Our technique enables compiler support for a wide range of sparse computations, including sparse convolutions and pooling that are widely used in ML and graphics applications. We propose an approach that gradually rewrites compound subscript expressions to simple subscript expressions with loops that exploit the sparsity pattern of the input sparse tensors. As a result, the time complexity of the generated kernels is bounded by the number of stored elements and not by the shape of the tensors. Our approach seamlessly integrates into existing frameworks and is compatible with recent advances in compilers for sparse computations, including the flexibility to efficiently handle arbitrary combinations of different sparse tensor formats. The implementation of our algorithm is open source and upstreamed to the MLIR sparse compiler. Experimental results show that our method achieves 19.5x speedup when compared with the state-of-the-art compiler-based method at 99.9% sparsity. The generated sparse kernels start to outperform dense convolution implementations at about 80% sparsity.

Mesh Convolution With Continuous Filters for 3-D Surface Parsing.

Geometric feature learning for 3-D surfaces is critical for many applications in computer graphics and 3-D vision. However, deep learning currently lags in hierarchical modeling of 3-D surfaces due to the lack of required operations and/or their efficient implementations. In this article, we propose a series of modular operations for effective geometric feature learning from 3-D triangle meshes. These operations include novel mesh convolutions, efficient mesh decimation, and associated mesh (un)poolings. Our mesh convolutions exploit spherical harmonics as orthonormal bases to create continuous convolutional filters. The mesh decimation module is graphics processing unit (GPU)-accelerated and able to process batched meshes on-the-fly, while the (un)pooling operations compute features for upsampled/downsampled meshes. We provide an open-source implementation of these operations, collectively termed Picasso. Picasso supports heterogeneous mesh batching and processing. Leveraging its modular operations, we further contribute a novel hierarchical neural network for perceptual parsing of 3-D surfaces, named PicassoNet++. It achieves highly competitive performance for shape analysis and scene segmentation on prominent 3-D benchmarks. The code, data, and trained models are available at https://github.com/EnyaHermite/Picasso.

Efficient Task Scheduling for Large-scale Graph Data Processing in Cloud Computing: A Particle Swarm Optimization Approach

To fulfil the requirements of task scheduling for processing massive amounts of graph data in cloud computing environments, this thesis offers the LGPPSO method, which is based on Particle Swarm Optimisation. The LGPPSO algorithm considers the task’s overall structure when initialising numerous particles in order to broaden the search range and raise the likelihood of finding an approximation optimal solution. We evaluated it in large-scale simulation trials with 100 performance-heterogeneous virtual machines (VMs) using both randomly generated and real application graphs, and evaluated its effectiveness against the CCSH and HEFT algorithms. The experimental findings demonstrate that, in both randomly generated graphs and real graph structure applications, significantly reducing the scheduling duration of large-scale graph data is the LGPPSO algorithm. For randomly generated 200 and 400 node tasks, respectively, the scheduling length is shortened by approximately 8.3% and 9.7% on average when compared to the HEFT algorithm. The LGPPSO technique minimises the scheduling length for actual graphical structure applications by an average of 14.6% and 16.9% for the Gaussian 100 and 200 node examples, respectively.

Three-Dimensional Reconstruction of Indoor Scenes Based on Implicit Neural Representation.

Reconstructing 3D indoor scenes from 2D images has always been an important task in computer vision and graphics applications. For indoor scenes, traditional 3D reconstruction methods have problems such as missing surface details, poor reconstruction of large plane textures and uneven illumination areas, and many wrongly reconstructed floating debris noises in the reconstructed models. This paper proposes a 3D reconstruction method for indoor scenes that combines neural radiation field (NeRFs) and signed distance function (SDF) implicit expressions. The volume density of the NeRF is used to provide geometric information for the SDF field, and the learning of geometric shapes and surfaces is strengthened by adding an adaptive normal prior optimization learning process. It not only preserves the high-quality geometric information of the NeRF, but also uses the SDF to generate an explicit mesh with a smooth surface, significantly improving the reconstruction quality of large plane textures and uneven illumination areas in indoor scenes. At the same time, a new regularization term is designed to constrain the weight distribution, making it an ideal unimodal compact distribution, thereby alleviating the problem of uneven density distribution and achieving the effect of floating debris removal in the final model. Experiments show that the 3D reconstruction effect of this paper on ScanNet, Hypersim, and Replica datasets outperforms the state-of-the-art methods.

Fast 3D solvers for interactive computational mechanics

While interactive simulations have been mostly limited to Computer Graphics applications, new generations of Graphics Processing Units (GPUs) allow the realization of industrial-grade interactive 3D physics simulations. By combining an immersed boundary method with efficient GPU-based MINRES and CG solvers using a GPU-based geometric multigrid preconditioner, we demonstrate a fast industrial 3D computational mechanics solver. The various implementation aspects - specifically how they differ from similar concepts used in the Computer Graphics community - are discussed in detail. The proposed concept opens up new classes of industrial simulation applications allowing a democratization beyond today’s expert users, from designer centric simulation to operational and service decisions based on 3D simulations. To support this, we provide various benchmark cases including a real-world study of a simulation-based service decision for a damaged gear-box mount.