Graphics Card Research Articles

MODERN VISUALIZATION BY DIGITALLY MODELING NEOLITHIC CRAFTED HUMAN SKULLS

Abstract. Our digital modeling in 3D aims to visualize Neolithic crafted skulls found in the Near East for their preservation and study taking into account both the possibilities of skull deformation in vivo as well as crafting them postmortem. Decapitation and burying or caching human skulls is met already in Palaeolithic contexts. Postmortem cranial crafting by drilling and carving, or modelling with plaster and asphalt using human skulls as basis was practiced in the Near East during the Pre-Pottery Neolithic and Late Neolithic period. The first examples of plastered human skulls were discovered at Jericho in the 1950s, then belonging to Jordan, after which to Israel and now to the Palestinian territories. Similar skulls were later found in various sites in the Near East. The examples digitally reconstructed here include skulls from Göbekli Tepe and Köşk Höyük found in Turkey, from the cave at Nahal Hemar at the Dead Sea in the Judean mountains of Israel and skulls from Jericho in the Palestinian territories. Both drawings and photographs were used in digitally reconstructing the skulls in 3D. The Blender software allowed us to sculpt the complex shape of the skull from a base mesh. Graphic Processing Unit (GPU) rendering sped up rendering thanks to Nvidia graphics cards. UV mapping was carried out for importing the texture. The visualization enabled us to make further anthropological observations. Beside the generally acknowledged Neolithic "skull cult" we also wish briefly to discuss other reasons for the phenomena and practices.

3D scanning digital models for virtual museums

AbstractA great number of virtual museums exhibit archaeological artifacts in an interactive form using 3D digital models to disseminate them to as many users as possible. Currently, 3D models obtained as a result of scanning museum objects have gained a great popularity. This technique allows to faithfully reproduce objects and to expose artifacts too delicate and precious to be presented in the real world. The aim of this article is to create a virtual museum in the VR world using 3D models obtained by 3D scanning with structured light. The advantages of optimizing mesh models for their use in virtual exhibitions are discussed. Unity and Unreal Engine, were used to create virtual museums in the VR world. Two twin test applications were prepared. This allowed to compare the performance of the developed applications (processor, graphics card, and RAM load). A survey was also conducted among the users of the implemented virtual museum application. For immersion in the world of VR, a low‐cost solution was used, involving the use of cheap VR frames, a smartphone, and a computer. The frames together with the smartphone were used to display the image, while the entire rendering process was performed on the computer.

Deep-Learning-Based Precision Visual Tracking

In this paper, we target the “precision visual tracking” problem, which is the precise tracking of a target point on a moving object. Compared with the abundant efforts in object-level tracking, which aims at accurately predicting the bounding box (or the mask) of the object, much less attention has been drawn to precision visual tracking. To this end, we present a template tracking framework that consists of two parts, template matching and template updating. For template matching, we trained a deep architecture to directly estimate the projective transformation that deforms the template to the search image. For template updating, we came up with a systematic strategy to update the initial and new templates. To avoid drift build-ups, we incorporate a fast-running dense correspondence matching module into the template update step. The proposed method was extensively tested on both synthetic and real data. To generate the synthetic data, we created a dataset of 480 image sequences. Each image sequence comes with the trajectory ground truth of a target point moving across all the frames. We compared the proposed method with six comparative approaches on this dataset. The tracking accuracy achieved by the proposed method was as follows: around 44% of the tracking errors were less than one pixel, about 78% of them were less than three pixels, and only about 14% of them exceeded five pixels. The proposed method is fast, running at 14 frames-per-second (fps) on [Formula: see text] image sequences on a workstation equipped with a Nvidia RTX 2080Ti graphic card. Qualitative results also show that the proposed method is applicable to real-world image sequences. The related code, pre-trained models, and the test data will be made publicly available at https://github.com/XM-Peng/Precision-Visual-Tracking/ .

Troid: A Real-Time Multiplayer FPS Game

Abstract: Troid is a real-time multiplayer first-person shooter (FPS) game designed to be played on a web browser without the need for a high-end PC or dedicated graphics card. The game allows two players to join a room and engage in a fast-paced, action-packed gameplay experience. The purpose of Troid is to provide a high-quality FPS game that can be enjoyed on any device with a modern web browser and an internet connection, regardless of hardware specifications. Unlike most FPS games that require high-end hardware to run smoothly, Troid is optimized for minimal requirements. Troid's front-end is built using a reactive web framework that facilitates the creation of fast and interactive user interfaces, while the game's graphics and animation are created using a 3D graphics library. Real-time communication between players is facilitated using a reliable real-time communication library, and an in-memory data store is utilized to store player coordinates and other in-game data. Overall, Troid exemplifies the potential of creating highquality multiplayer games that can be played on any device with a modern web browser and an internet connection, without compromising on the gameplay experience.

Efficient and Low Color Information Dependency Skin Segmentation Model

Skin segmentation involves segmenting the human skin region in an image. It is a preprocessing technique mainly used in many applications such as face detection, hand gesture recognition, and remote biosignal measurements. As the performance of skin segmentation directly affects the performance of these applications, precise skin segmentation methods have been studied. However, previous skin segmentation methods are unsuitable for real-world environments because they rely heavily on color information. In addition, deep-learning-based skin segmentation methods incur high computational costs, even though skin segmentation is mainly used for preprocessing. This study proposes a lightweight skin segmentation model with a high performance. Additionally, we used data augmentation techniques that modify the hue, saturation, and values, allowing the model to learn texture or contextual information better without relying on color information. Our proposed model requires 1.09M parameters and 5.04 giga multiply-accumulate. Through experiments, we demonstrated that our proposed model shows high performance with an F-score of 0.9492 and consistent performance even for modified images. Furthermore, our proposed model showed a fast processing speed of approximately 68 fps, based on 3 × 512 × 512 images and an NVIDIA RTX 2080TI GPU (11GB VRAM) graphics card.

Deep learning using a residual deconvolutional network enables real-time high-density single-molecule localization microscopy.

High-density localization based on deep learning is a very effective method to accelerate single molecule localization microscopy (SMLM). Compared with traditional high-density localization methods, deep learning-based methods enable a faster data processing speed and a higher localization accuracy. However, the reported high-density localization methods based on deep learning are still not fast enough to enable real time data processing for large batches of raw images, which is probably due to the heavy computational burden and computation complexity in the U-shape architecture used in these models. Here we propose a high-density localization method called FID-STORM, which is based on an improved residual deconvolutional network for the real-time processing of raw images. In FID-STORM, we use a residual network to extract the features directly from low-resolution raw images rather than the U-shape network from interpolated images. We also use a model fusion from TensorRT to further accelerate the inference of the model. In addition, we process the sum of the localization images directly on GPU to obtain an additional speed gain. Using simulated and experimental data, we verified that the FID-STORM method achieves a processing speed of 7.31 ms/frame at 256 × 256 pixels @ Nvidia RTX 2080 Ti graphic card, which is shorter than the typical exposure time of 10∼30 ms, thus enabling real-time data processing in high-density SMLM. Moreover, compared with a popular interpolated image-based method called Deep-STORM, FID-STORM enables a speed gain of ∼26 times, without loss of reconstruction accuracy. We also provided an ImageJ plugin for our new method.

Performance Analysis of Parallel Computing in Image Classification

The use of parallel computing can speed up the training of deep learning models. The traditional neural network model ResNet is chosen for testing in this paper on the effectiveness of data parallelism in image classification, and test data is provided in a 6-GPU environment. In this paper, it is suggested that various factors should be considered when building affordable hardware configurations to expedite model training in practical application scenarios. The communication costs are not insignificant because today's large computing clusters are primarily offered via cloud computing. Another crucial point to remember is that the number of CUDA cores is the primary hardware foundation for GPU acceleration technology. Therefore, acceleration may not be affected by more incredible video memory or fewer CUDA cores for some particular graphics cards. In addition, the issue of beyond-model performance in parallel computing is another issue that must be disregarded. Due to the parallel strategy's limitations, it is essential to tweak the super parameters while speeding up model training. The model's performance is more likely to be ensured by the lower learning rate and Batch size. This paper's experimental conclusion can support building an appropriate hardware configuration scheme.

Pedestrian detection model based on Tiny-Yolov3 architecture for wearable devices to visually impaired assistance.

Introduction: Wearable assistive devices for the visually impaired whose technology is based on video camera devices represent a challenge in rapid evolution, where one of the main problems is to find computer vision algorithms that can be implemented in low-cost embedded devices. Objectives and Methods: This work presents a Tiny You Only Look Once architecture for pedestrian detection, which can be implemented in low-cost wearable devices as an alternative for the development of assistive technologies for the visually impaired. Results: The recall results of the proposed refined model represent an improvement of 71% working with four anchor boxes and 66% with six anchor boxes compared to the original model. The accuracy achieved on the same data set shows an increase of 14% and 25%, respectively. The F1 calculation shows a refinement of 57% and 55%. The average accuracy of the models achieved an improvement of 87% and 99%. The number of correctly detected objects was 3098 and 2892 for four and six anchor boxes, respectively, whose performance is better by 77% and 65% compared to the original, which correctly detected 1743 objects. Discussion: Finally, the model was optimized for the Jetson Nano embedded system, a case study for low-power embedded devices, and in a desktop computer. In both cases, the graphics processing unit (GPU) and central processing unit were tested, and a documented comparison of solutions aimed at serving visually impaired people was performed. Conclusion: We performed the desktop tests with a RTX 2070S graphics card, and the image processing took about 2.8ms. The Jetson Nano board could process an image in about 110ms, offering the opportunity to generate alert notification procedures in support of visually impaired mobility.

Garment image style transfer based on deep learning

Before neural networks, image style transfer procedures had a common idea: analyze images with a certain style, build a mathematical or statistical model for the style, and then change the image to be transferred, so that it better fits the established model. In this paper, k-means and semantic closed natural matting algorithm is combined for image segmentation, the style and content in the image are extracted based on neural network, and the resulting image is synthesized by image reconstruction technology to realize the migration of national costume styles. Due to the serious artifacts of the output image, an improved image style transfer algorithm is adopted to constrain the transformation from the input image to the output image in the local affine transformation of the color space, and this constraint is expressed as a completely differentiable parameter term, image distortion is suppressed effectively. In the process of real photo style transfer, there is also space inconsistency. Smoothing is done to ensure that the space style is consistent after style processing, it greatly speeds up the operation speed. On NVIDIA GTX1080TI graphics card, algorithm is tested with 256×256 resolution images. It includes three indicators of average running time, memory usage and the number of styles generated by a single model, which are 0.06 s, 136.06 MB and 1 respectively. These indicators can reflect the efficiency and flexibility of the algorithm.

The VM2D Open Source Code for Two-Dimensional Incompressible Flow Simulation by Using Fully Lagrangian Vortex Particle Methods

This article describes the open-source C++ code VM2D for the simulation of two-dimensional viscous incompressible flows and solving fluid-structure interaction problems. The code is based on the Viscous Vortex Domains (VVD) method developed by Prof. G. Ya. Dynnikova, where the viscosity influence is taken into account by introducing the diffusive velocity. The original VVD method was supplemented by the author’s algorithms for boundary condition satisfaction, which made it possible to increase the accuracy of flow simulation near the airfoil’s surface line and reduce oscillations when calculating hydrodynamic loads. This paper is aimed primarily at assessing the efficiency of the parallelization of the algorithm. OpenMP, MPI, and Nvidia CUDA parallel programming technologies are used in VM2D, which allow performing simulations on computer systems of various architectures, including those equipped with graphics accelerators. Since the VVD method belongs to the particle methods, the efficiency of parallelization with the usage of graphics accelerators turns out to be quite high. It is shown that in a real simulation, one graphics card can replace about 80 nodes, each of which is equipped with 28 CPU cores. The source code of VM2D is available on GitHub under GNU GPL license.

Fast and Efficient Simulation of the FEBID Process with Thermal Effects

Focused electron-beam-induced deposition (FEBID) is a highly versatile direct-write approach with particular strengths in the 3D nanofabrication of functional materials. Despite its apparent similarity to other 3D printing approaches, non-local effects related to precursor depletion, electron scattering and sample heating during the 3D growth process complicate the shape-true transfer from a target 3D model to the actual deposit. Here, we describe an efficient and fast numerical approach to simulate the growth process, which allows for a systematic study of the influence of the most important growth parameters on the resulting shape of the 3D structures. The precursor parameter set derived in this work for the precursor Me3PtCpMe enables a detailed replication of the experimentally fabricated nanostructure, taking beam-induced heating into account. The modular character of the simulation approach allows for additional future performance increases using parallelization or drawing on the use of graphics cards. Ultimately, beam-control pattern generation for 3D FEBID will profit from being routinely combined with this fast simulation approach for optimized shape transfer.

Integrated Photonic Neural Networks: Opportunities and Challenges

Photonic neural networks benefit from the use of photons to perform intelligent inference computing with ultrafast and ultralow energy consumption in ultra-high-throughput, providing the efficient photonic hardware for the new generation of intelligent computing, and the effective way to support large-scale integration for on-chip all-optical computing chips. With the rapid development of photonic neural networks, demands for efficient computation power have increased dramatically. However, the weak and impractical optical nonlinear activations, the lack of suitable configurations for integrated photonic hardware, and proper optical storage mediums pose challenges to this field. In this Perspective, we propose our current point of view and a suggestive roadmap in the field of integrated photonic platform for optical neural networks. Throughout the discussion, we highlight recent progresses meeting with major challenges. We also identify some next challenges still ahead to realize integrated photonic neural networks capable of matching the current computational power of graphic cards.

A parallel algorithm of multiple face detection on multi-core system

<p><span lang="EN-US">This work offers a graphics processing unit (GPU)-based system for real-time face recognition, which can detect and identify faces with high accuracy. This work created and implemented novel parallel strategies for image integral, computation scan window processing, and classifier amplification and correction as part of the face identification phase of the Viola-Jones cascade classifier. Also, the algorithm and parallelized a portion of the testing step during the facial recognition stage were experimented with. The suggested approach significantly improves existing facial recognition methods by enhancing the performance of two crucial components. The experimental findings show that the proposed method, when implemented on an NVidia GTX 570 graphics card, outperforms the typical CPU program by a factor of 19.72 in the detection phase and 1573 in the recognition phase, with only 2000 images trained and 40 images tested. The recognition rate will plateau when the hardware's capabilities are maxed out. This demonstrates that the suggested method works well in real-time.</span></p>

Reduced volume of diabetic pancreatic islets in rodents detected by synchrotron X-ray phase-contrast microtomography and deep learning network

The pancreatic islet is a highly structured micro-organ that produces insulin in response to rising blood glucose. Here we develop a label-free and automatic imaging approach to visualize the islets in situ in diabetic rodents by the synchrotron radiation X-ray phase-contrast microtomography (SRμCT) at the ID17 station of the European Synchrotron Radiation Facility. The large-size images (3.2mm×15.97mm) were acquired in the pancreas in STZ-treated mice and diabetic GK rats. Each pancreas was dissected by 3000 reconstructed images. The image datasets were further analysed by a self-developed deep learning method, AA-Net. All islets in the pancreas were segmented and visualized by the three-dimension (3D) reconstruction. After quantifying the volumes of the islets, we found that the number of larger islets (=>1500 μm3) was reduced by 2-fold (wt 1004±94 vs GK 419±122, P<0.001) in chronically developed diabetic GK rat, while in STZ-treated diabetic mouse the large islets were decreased by half (189±33 vs 90±29, P<0.001) compared to the untreated mice. Our study provides a label-free tool for detecting and quantifying pancreatic islets in situ. It implies the possibility of monitoring the state of pancreatic islets in vivo diabetes without labelling.

Improving the performance of web-streaming by super-resolution upscaling techniques

In recent years, we have seen significant progress in advanced image upscaling techniques, sometimes called super-resolution, ML-based, or AI-based upscaling. Such algorithms are now available not only in form of specialized software but also in drivers and SDKs supplied with modern graphics cards. Upscaling functions in NVIDIA Maxine SDK is one of the recent examples. However, to take advantage of this functionality in video streaming applications, one needs to (a) quantify the impacts of super-resolution techniques on the perceived visual quality, (b) implement video rendering incorporating super-resolution upscaling techniques, and (c) implement new bitrate+resolution adaptation algorithms in streaming players, enabling such players to deliver better quality of experience or better efficiency (e.g. reduce bandwidth usage) or both. Towards this end, in this paper, we propose several techniques that may be helpful to the implementation community. First, we offer a model quantifying the impacts of super resolution upscaling on the perceived quality. Our model is based on the Westerink-Roufs model connecting the true resolution of images/videos to perceived quality, with several additional parameters added, allowing its tuning to specific implementations of super-resolution techniques. We verify this model by using several recent datasets including MOS scores measured for several conventional up-scaling and super-resolution algorithms. Then, we propose an improved adaptation logic for video streaming players, considering video bitrates, encoded video resolutions, player size, and the upscaling method. This improved logic relies on our modified Westerink-Roufs model to predict perceived quality and suggests choices of renditions that would deliver the best quality for given display and upscaling method characteristics. Finally, we study the impacts of the proposed techniques and show that they can deliver practically appreciable results in terms of the expected QoE improvements and bandwidth savings.

Adaptive Lightweight License Plate Image Recovery Using Deep Learning Based on Generative Adversarial Network

Many Convolutional Neural Networks (CNNs) methods have already surpassed traditional approaches to image restoration tasks. Those CNNs models were usually designed to enhance single tasks such as an image resolution (super-resolution) or image denoising, but we came up with unconventional goals, that is, multiple recovery tasks from a single network design. Although the Transformer design has recently gained attention in image recovery task but they are too slow. In order to work with license plate images from a traffic camera stream, the system has to be fast. So, we proposed a lightweight deep learning-based data recovery system using a Generative Adversarial Network (GAN) principle named License Plate Recovery GAN (LPRGAN). The design has a proposed encoder-decoder style inspired by an autoencoder aided by dual classification networks. This style is suitable for problem-characteristic learning because strong contextual information retrieves from the down-scaled representations. This proposed system has three main features, identifying a problem, data recovery, and a fail-safe mechanism. The data recovery unit (LPRGAN) is used to recover license plate images from multiple degraded input images. Most existing image restoration systems do not have self-awareness, leading to an inefficiency problem. Unlike existing works, this system has anomaly detection and will only process on a degraded input, reducing workload overhead and improving efficiency as well as a fail-safe feature to prevent an unexpected bad output. Hence, it is a light and efficient system so it could be possible to deploy on a low-power machine such as edge computing devices, opening up new possibilities in on-device computing. Our proposed research can recover several degraded problems up to 720p resolution at 15 frames per second on a single graphic card, 256x128 resolution at 17 frames per second on a CPU-only workstation machine, or 7 frames per second on an ultra-low-power tablet PC.

Comparison Analysis of Liquid and Air-cooling Systems for GPU

Abstract: In order to increase reliability and prevent early failure, heat produced by electronic components and circuits must be drained. Heat sinks, fans, and other air-cooling devices, as well as alternative computer cooling methods like liquid cooling, can all be used as heat dissipation techniques. Here the paper's main focus is on a numerical analysis of temperature distribution in a GPU scenario with various input velocity circumstances, including v = 1 m/s, 3 m/s, and 5 m/s. Air and water are the two distinct fluids employed in this investigation to cool the graphics card. An analysis of the fluid flow and heat transfer simulation of a desktop graphics card heat sink is conducted using the CFD software ANSYS Fluent. According to the findings, cooling with water was 10% more effective than cooling with air. The outcome also shows an inverse correlation between the graphic card chip's intake velocity and temperature. The rate of heat transfer to the heat sink increases as the inflow velocity rises which is helped by the convection.

Comparative analysis of the performance of the Flax engine and Unity

This article compares the two game engines Unity and Flax engine. The research consisted in comparing the efficiency of individual engines using two similar applications. The first one generates from several hundred to several thousand physical objects and examines the efficiency of engines in terms of physics simulations. The second application generates a forest area filled with the same tree model. The application aims to study the behavior of the engine for various graphics settings. In both applications, parameters such as the number of frames per second, consumption of the processor, graphics card and RAM were tested. Research shows that Unity is more efficient than Flax Engine.

IMAGE-IN: Interactive web-based multidimensional 3D visualizer for multi-modal microscopy images.

Advances in microscopy hardware and storage capabilities lead to increasingly larger multidimensional datasets. The multiple dimensions are commonly associated with space, time, and color channels. Since "seeing is believing", it is important to have easy access to user-friendly visualization software. Here we present IMAGE-IN, an interactive web-based multidimensional (N-D) viewer designed specifically for confocal laser scanning microscopy (CLSM) and focused ion beam scanning electron microscopy (FIB-SEM) data, with the goal of assisting biologists in their visualization and analysis tasks and promoting digital workflows. This new visualization platform includes intuitive multidimensional opacity fine-tuning, shading on/off, multiple blending modes for volume viewers, and the ability to handle multichannel volumetric data in volume and surface views. The software accepts a sequence of image files or stacked 3D images as input and offers a variety of viewing options ranging from 3D volume/surface rendering to multiplanar reconstruction approaches. We evaluate the performance by comparing the loading and rendering timings of a heterogeneous dataset of multichannel CLSM and FIB-SEM images on two devices with installed graphic cards, as well as comparing rendered image quality between ClearVolume (the ImageJ open-source desktop viewer), Napari (the Python desktop viewer), Imaris (the closed-source desktop viewer), and our proposed IMAGE-IN web viewer.

HH-NIDS: Heterogeneous Hardware-Based Network Intrusion Detection Framework for IoT Security

This study proposes a heterogeneous hardware-based framework for network intrusion detection using lightweight artificial neural network models. With the increase in the volume of exchanged data, IoT networks’ security has become a crucial issue. Anomaly-based intrusion detection systems (IDS) using machine learning have recently gained increased popularity due to their generation’s ability to detect unseen attacks. However, the deployment of anomaly-based AI-assisted IDS for IoT devices is computationally expensive. A high-performance and ultra-low power consumption anomaly-based IDS framework is proposed and evaluated in this paper. The framework has achieved the highest accuracy of 98.57% and 99.66% on the UNSW-NB15 and IoT-23 datasets, respectively. The inference engine on the MAX78000EVKIT AI-microcontroller is 11.3 times faster than the Intel Core i7-9750H 2.6 GHz and 21.3 times faster than NVIDIA GeForce GTX 1650 graphics cards, when the power drawn was 18mW. In addition, the pipelined design on the PYNQ-Z2 SoC FPGA board with the Xilinx Zynq xc7z020-1clg400c device is optimised to run at the on-chip frequency (100 MHz), which shows a speedup of 53.5 times compared to the MAX78000EVKIT.