Point-Based Rendering Research Articles

Large-scale assessment of date palm plantations based on UAV remote sensing and multiscale vision transformer

Timely and efficient mapping of date palm plantations through unmanned aerial vehicle (UAV) remote sensing is critical for continuous observation, health and risk evaluation, pest management, resource optimization, and ensuring the long-term sustainability of the dates industry. This study presents an efficient and cost-effective transformer-based approach to identify, countify, monitor, and evaluate the overall well-being of palm trees using extensive UAV imagery. The suggested approach integrates an improved multiscale vision transformer, feature pyramid network, Mask R–CNN, and improved slicing-aided hyper inference for practical large-scale assessments. This combination enabled the extraction of multiscale features, capturing long-range dependencies in the data and boosting the model's generalizability. The proposed architecture outperformed several CNN-based architectures (including Mask R–CNN, Cascade Mask R–CNN, Point-based Rendering, and You Only Look At CoefficientTs), achieving F-scores of 94.33% and 94.2% for date palm tree detection and segmentation, respectively. The transformer-based architecture was optimized using transfer learning to differentiate between healthy and unhealthy date palm trees, particularly those with severe infestations. The potential generic condition of date palm trees was predicted with an F-score of 88.4%. Further advancements in this field could pave the way for a proactive strategy, enabling timely detection, which would aid in pest management and support the sustainable growth of the dates sector.

Enhanced rendering-based approach for improved quality of instance segmentation in detecting green gram (Vigna Rediata) pods

The emergence of Artificial Intelligence, deep learning, and current computer vision algorithms are the main contributors to innovations in the agricultural domain. The most recent detection algorithms capable of giving real-time detections at the edge nodes tackle most agricultural problems, such as disease, pest or insect detections, and maturity level detection of crops (fruits and vegetables). Modern harvesters and fruit-picking robots rely heavily on the detection capability of the algorithms used. Various detection algorithms have been proposed and used in literature, having good performance in terms of mean average precision. Still, the current agricultural systems require not only high mean average accuracy but also algorithms should have high inference speeds. The research proposes a Detectron2-based framework with PointRend (Point-based Rendering), capable of providing enhanced, high-quality pixel-level instance segmentation in identifying and detecting green gram pods or Mung Bean (Vigna Radiata) in natural field conditions rendering crisp and smooth boundaries for accurately locating the green gram pods. The results indicate that the proposed framework outperforms the famous Mask R-CNN model to obtain higher mean average precision and improved quality of detections.

Deep convolutional neural networks and Swin transformer-based frameworks for individual date palm tree detection and mapping from large-scale UAV images

Timely and reliable mapping of individual date palm trees is essential for their monitoring, health and risk assessment, pest control, and sustainable management of the date palm industry. This study presents an instance segmentation framework for large-scale detection and mapping of date palm trees using unmanned aerial vehicle (UAV)-based images. First, a data conversion framework is created to convert UAV image tiles and ground-truth vector data into annotation format of Common Objects in Context. Second, this study examines the efficacy of various instance segmentation models, namely, mask region convolutional neural network (Mask R-CNN), Mask Scoring R-CNN, You Only Look At CoefficientTs, Point-based Rendering, Segmenting Objects by Locations (SOLO), and SOLOv2) with varying residual learning networks (ResNets) in detecting and delineating individual date palm trees. Furthermore, the performance of two variants of Swin Transformer networks with a feature pyramid network (FPN) (Swin-small-FPN and Swin-tiny-FPN) as Mask R-CNN network backbones was also evaluated. Third, we assess the generalizability of the evaluated instance segmentation models and backbones on different testing datasets with varying spatial resolutions. Results show that Mask R-CNN models based on Swin Transformers backbones outperform those with ResNets in the detection and segmentation of date palm trees with mAP50 of 92% and 91% and F-measures of 94% and 93%. Moreover, the Mask scoring R-CNN-based ResNet-50 and Mask R-CNN with a Swin-small-FPN backbone outperform the evaluated models and demonstrate great generalizability in different datasets with diverse spatial resolutions. The proposed instance segmentation framework provides an efficient tool for date palm tree mapping from multi-scale UAV-based images and is valuable and suitable for individual tree crown delineations and other earth-related applications.

Detection and Semiquantitative Analysis of Cardiomegaly, Pneumothorax, and Pleural Effusion on Chest Radiographs.

To develop and evaluate deep learning models for the detection and semiquantitative analysis of cardiomegaly, pneumothorax, and pleural effusion on chest radiographs. In this retrospective study, models were trained for lesion detection or for lung segmentation. The first dataset for lesion detection consisted of 2838 chest radiographs from 2638 patients (obtained between November 2018 and January 2020) containing findings positive for cardiomegaly, pneumothorax, and pleural effusion that were used in developing Mask region-based convolutional neural networks plus Point-based Rendering models. Separate detection models were trained for each disease. The second dataset was from two public datasets, which included 704 chest radiographs for training and testing a U-Net for lung segmentation. Based on accurate detection and segmentation, semiquantitative indexes were calculated for cardiomegaly (cardiothoracic ratio), pneumothorax (lung compression degree), and pleural effusion (grade of pleural effusion). Detection performance was evaluated by average precision (AP) and free-response receiver operating characteristic (FROC) curve score with the intersection over union greater than 75% (AP75; FROC score75). Segmentation performance was evaluated by Dice similarity coefficient. The detection models achieved high accuracy for detecting cardiomegaly (AP75, 98.0%; FROC score75, 0.985), pneumothorax (AP75, 71.2%; FROC score75, 0.728), and pleural effusion (AP75, 78.2%; FROC score75, 0.802), and they also weakened boundary aliasing. The segmentation effect of the lung field (Dice, 0.960), cardiomegaly (Dice, 0.935), pneumothorax (Dice, 0.827), and pleural effusion (Dice, 0.826) was good, which provided important support for semiquantitative analysis. The developed models could detect cardiomegaly, pneumothorax, and pleural effusion, and semiquantitative indexes could be calculated from segmentations.Keywords: Computer-Aided Diagnosis (CAD), Thorax, CardiacSupplemental material is available for this article.© RSNA, 2021.

Application of Stochastic Point-Based Rendering to Laser-Scanned Point Clouds of Various Cultural Heritage Objects

Recently, we proposed stochastic point-based rendering, which enables precise and interactive-speed transparent rendering of large-scale laser-scanned point clouds. This transparent visualization method does not suffer from rendering artifact and realizes correct depth feel in the created 3D image.In this paper, we apply the method to several kinds of large-scale laser-scanned point clouds of cultural heritage objects and prove its wide applicability.In addition, we prove better image quality is realized by properly eliminating points to realize better distributional uniformity of points. Here, the distributional uniformity means uniformity of inter-point distances between nearest-neighbor points.We also demonstrate that highlighting feature regions, especially edges, in the transparent visualization helps us understand 3D internal structures of complex laser-scanned objects. The feature regions are highlighted by properly increasing local opacity of the regions.

Point-Based Rendering for Homogeneous Participating Media with Refractive Boundaries.

Illumination effects in translucent materials are a combination of several physical phenomena: refraction at the surface, absorption and scattering inside the material. Because refraction can focus light deep inside the material, where it will be scattered, practical illumination simulation inside translucent materials is difficult. In this paper, we present an a Point-Based Global Illumination method for light transport on homogeneous translucent materials with refractive boundaries. We start by placing light samples inside the translucent material and organizing them into a spatial hierarchy. At rendering, we gather light from these samples for each camera ray. We compute separately the sample contributions for single, double and multiple scattering, and add them. We present two implementations of our algorithm: an offline version for high-quality rendering and an interactive GPU implementation. The offline version provides significant speed-ups and reduced memory footprints compared to state-of-the-art algorithms, with no visible impact on quality. The GPU version yields interactive frame rates: 30fps when moving the viewpoint, 25fps when editing the light position or the material parameters.

Adaptive image contrast enhancement algorithm for point-based rendering

Surgical simulation is a major application in computer graphics and virtual reality, and most of the existing work indicates that interactive real-time cutting simulation of soft tissue is a fundamental but challenging research problem in virtual surgery simulation systems. More specifically, it is difficult to achieve a fast enough graphic update rate (at least 30 Hz) on commodity PC hardware by utilizing traditional triangle-based rendering algorithms. In recent years, point-based rendering (PBR) has been shown to offer the potential to outperform the traditional triangle-based rendering in speed when it is applied to highly complex soft tissue cutting models. Nevertheless, the PBR algorithms are still limited in visual quality due to inherent contrast distortion. We propose an adaptive image contrast enhancement algorithm as a postprocessing module for PBR, providing high visual rendering quality as well as acceptable rendering efficiency. Our approach is based on a perceptible image quality technique with automatic parameter selection, resulting in a visual quality comparable to existing conven- tional PBR algorithms. Experimental results show that our adaptive image contrast enhancement algorithm produces encouraging results both visually and numerically compared to representative algorithms, and experi- ments conducted on the latest hardware demonstrate that the proposed PBR framework with the postprocessing module is superior to the conventional PBR algorithm and that the proposed contrast enhancement algorithm can be utilized in (or compatible with) various variants of the conventional PBR algorithm. © 2015 SPIE and IS&T (DOI: 10.1117/1.JEI.24.2.023033)

Subsurface Scattering for Realistic Point-based Rendering

Subsurface Scattering for Realistic Point-based Rendering

Nearest Neighbor Searches on the GPU

We introduce a GPU grid-based data structure for massively parallel nearest neighbor searches for dynamic point clouds. The implementation provides real-time performance and it is executed on GPU, both grid construction and nearest neighbors (approximate or exact) searches. This minimizes the memory transfer between device and system memories, improving overall performance. The proposed algorithm may be used across different applications with static and dynamic scenarios. Moreover, our data structure supports three-dimensional point clouds and given its dynamic nature, the user can change the data structure’s parameters at runtime. The same applies to the number of neighbors to be found. Performance comparisons were made against previous works, endorsing the benefits of our solution. Finally, we were able to develop a real-time Point-Based Rendering application for validation of the data structure. Its drawbacks and data distribution’s impact on performance were analysed and some directions for further investigation are given.

Point-Based Rendering of Non-Manifold Surfaces

We are concerned with producing high-quality images of parametric and implicit surfaces, in particular those with non-manifold features. We present a point-based technique for rendering implicit surfaces that uses octree spatial subdivision with a natural interval exclusion test that guarantees that no parts of the surface are missed. This allows us to render non-manifold implicit surfaces at speeds comparable to parametric surfaces. We also derive criteria that guarantee complete pixel coverage of the surface. The point-based method allows for hidden surface elimination using a z-buffer, and shadow casting using a shadow buffer. We illustrate the technique with a number of surfaces, and discuss its advantages and disadvantages.

Deferred blending: Image composition for single-pass point rendering

In this paper, we propose novel GPU accelerated algorithms for interactive point-based rendering (PBR) and high-quality shading of transparent point surfaces. By introducing the concept of deferred blending we are able to formulate the smooth point interpolation problem as an image compositing post-processing task. Consequently, our new PBR algorithm does not suffer from an extra visibility-splatting pre-render pass, for conservative @?-z-buffer visibility culling, as this is eventually performed together with the smooth point interpolation during image compositing. Moreover, this new deferred blending concept enables hardware accelerated transparent PBR with combined effects of multi-layer transparency, refraction, specular reflection, and per-fragment shading. Deferred blending is based on a separation of the point data into not self-overlapping minimal independent groups, a multi-target rendering pass and an image compositing post-processing stage. We present different grouping algorithms for off-line and on-line processing. For basic opaque surface rendering and simple transparency effects, our novel algorithm only needs a single geometry rendering pass. For high-quality transparent image synthesis one extra rendering pass is sufficient. Besides transparency, per-fragment reflective and refractive multi-layer effects (e.g. environment mapping) are supported in our algorithm.