Interactive Graphics Applications Research Articles

Fast and Accurate Illumination Estimation Using LDR Panoramic Images for Realistic Rendering.

A high dynamic range (HDR) image is commonly used to reveal stereo illumination, which is crucial for generating high-quality realistic rendering effects. Compared to the high-cost HDR imaging technique, low dynamic range (LDR) imaging provides a low-cost alternative and is preferable for interactive graphics applications. However, the limited LDR pixel bit depth significantly bothers accurate illumination estimation using LDR images. The conflict between the realism and promptness of illumination estimation for realistic rendering is yet to be resolved. In this paper, an efficient method that accurately infers illuminations of real-world scenes using LDR panoramic images is proposed. It estimates multiple lighting parameters, including locations, types and intensities of light sources. In our approach, a new algorithm that extracts illuminant characteristics during the exposure attenuation process is developed to locate light sources and outline their boundaries. To better predict realistic illuminations, a new deep learning model is designed to efficiently parse complex LDR panoramas and classify detected light sources. Finally, realistic illumination intensities are calculated by recovering the inverse camera response function and extending the dynamic range of pixel values based on previously estimated parameters of light sources. The reconstructed radiance map can be used to compute high-quality image-based lighting of virtual models. Experimental results demonstrate that the proposed method is capable of efficiently and accurately computing comprehensive illuminations using LDR images. Our method can be used to produce better realistic rendering results than existing approaches.

Image features influence reaction time

We aim to ask and answer an essential question " how quickly do we react after observing a displayed visual target?" To this end, we present psychophysical studies that characterize the remarkable disconnect between human saccadic behaviors and spatial visual acuity. Building on the results of our studies, we develop a perceptual model to predict temporal gaze behavior, particularly saccadic latency, as a function of the statistics of a displayed image. Specifically, we implement a neurologically-inspired probabilistic model that mimics the accumulation of confidence that leads to a perceptual decision. We validate our model with a series of objective measurements and user studies using an eye-tracked VR display. The results demonstrate that our model prediction is in statistical alignment with real-world human behavior. Further, we establish that many sub-threshold image modifications commonly introduced in graphics pipelines may significantly alter human reaction timing, even if the differences are visually undetectable. Finally, we show that our model can serve as a metric to predict and alter reaction latency of users in interactive computer graphics applications, thus may improve gaze-contingent rendering, design of virtual experiences, and player performance in e-sports. We illustrate this with two examples: estimating competition fairness in a video game with two different team colors, and tuning display viewing distance to minimize player reaction time.

Latency and Cybersickness: Impact, Causes, and Measures. A Review

Latency is a key characteristic inherent to any computer system. Motion-to-Photon (MTP) latency describes the time between the movement of a tracked object and its corresponding movement rendered and depicted by computer-generated images on a graphical output screen. High MTP latency can cause a loss of performance in interactive graphics applications and, even worse, can provoke cybersickness in Virtual Reality (VR) applications. Here, cybersickness can degrade VR experiences or may render the experiences completely unusable. It can confound research findings of an otherwise sound experiment. Latency as a contributing factor to cybersickness needs to be properly understood. Its effects need to be analyzed, its sources need to be identified, good measurement methods need to be developed, and proper counter measures need to be developed in order to reduce potentially harmful impacts of latency on the usability and safety of VR systems. Research shows that latency can exhibit intricate timing patterns with various spiking and periodic behavior. These timing behaviors may vary, yet most are found to provoke cybersickness. Overall, latency can differ drastically between different systems interfering with generalization of measurement results. This review article describes the causes and effects of latency with regard to cybersickness. We report on different existing approaches to measure and report latency. Hence, the article provides readers with the knowledge to understand and report latency for their own applications, evaluations, and experiments. It should also help to measure, identify, and finally control and counteract latency and hence gain confidence into the soundness of empirical data collected by VR exposures. Low latency increases the usability and safety of VR systems.

Exploratory Volumetric Deep Earth Visualization by 2.5D Interactive Compositing.

In this contribution we consider the visualization of global, deep Earth volume datasets for display and researcher interaction. While the algorithms and data analysis techniques that produce such volumetric results have become more sophisticated, the manner of visualizing these findings can be improved. We address the challenge of making an illustrative, exploratory visualization of a global geoscience dataset using a combined seismic tomography result, the primary means by which geoscientists infer structure and process in the deep Earth. We present a novel, interactive graphical application suite and associated workflow that uses an intuitive 2.5D layer compositing approach. This allows the user to adjust the separation between data-slices, control graphics variables such as color mapping, opacity and compositing, and facilitate exploration and annotation of the architecture of the lithosphere. Graphics outputs from our applications are enabled for immersive systems such as dome displays. In a case study we visualize the deep Earth structure beneath the Indian Ocean region. We anticipate that the application methodology will find use in the visualization of multiple datasets representing aspects of the Earth's deep interior and atmosphere, and in the interaction with the increasing number of rich datasets from missions to our neighboring planets.

UnCOVERApp: an interactive graphical application for clinical assessment of sequence coverage at the base-pair level.

Next-generation sequencing is increasingly adopted in the clinical practice largely thanks to concurrent advancements in bioinformatic tools for variant detection and annotation. However, the need to assess sequencing quality at the base-pair level still poses challenges for diagnostic accuracy. One of the most popular quality parameters is the percentage of targeted bases characterized by low depth of coverage (DoC). These regions potentially 'hide' clinically relevant variants, but no annotation is usually returned with them. However, visualizing low-DoC data with their potential functional and clinical consequences may be useful to prioritize inspection of specific regions before re-sequencing all coverage gaps or making assertions about completeness of the diagnostic test. To meet this need, we have developed unCOVERApp, an interactive application for graphical inspection and clinical annotation of low-DoC genomic regions containing genes. unCOVERApp interactive plots allow to display gene sequence coverage down to the base-pair level, and functional and clinical annotations of sites below a user-defined DoC threshold can be downloaded in a user-friendly spreadsheet format. Moreover, unCOVERApp provides a simple statistical framework to evaluate if DoC is sufficient for the detection of somatic variants. A maximum credible allele frequency calculator is also available allowing users to set allele frequency cut-offs based on assumptions about the genetic architecture of the disease. In conclusion, unCOVERApp is an original tool designed to identify sites of potential clinical interest that may be 'hidden' in diagnostic sequencing data. unCOVERApp is a free application developed with Shiny packages and available in GitHub (https://github.com/Manuelaio/uncoverappLib). Supplementary data are available at Bioinformatics online.

No-reference synthetic image quality assessment with convolutional neural network and local image saliency

Depth-image-based rendering (DIBR) is widely used in 3DTV, free-viewpoint video, and interactive 3D graphics applications. Typically, synthetic images generated by DIBR-based systems incorporate various distortions, particularly geometric distortions induced by object dis-occlusion. Ensuring the quality of synthetic images is critical to maintaining adequate system service. However, traditional 2D image quality metrics are ineffective for evaluating synthetic images as they are not sensitive to geometric distortion. In this paper, we propose a novel no-reference image quality assessment method for synthetic images based on convolutional neural networks, introducing local image saliency as prediction weights. Due to the lack of existing training data, we construct a new DIBR synthetic image dataset as part of our contribution. Experiments were conducted on both the public benchmark IRCCyN/IVC DIBR image dataset and our own dataset. Results demonstrate that our proposed metric outperforms traditional 2D image quality metrics and state-of-the-art DIBR-related metrics.

Interactive Web Apps for Visualizations in Mathematics and Engineering

AbstractDepicting numbers in a graphical way has always been crucial to understand and explain the physical phenomena behind. Interactive graphical applications bring the understanding to a higher level, where the users get actively engaged. Moreover, interactive visualization of data, equations etc.… help to motivate students and strengthen the sharing of knowledge. At the Technical University of Munich, the Chair of Structural Mechanics and the Research Group Algebra developed interactive applications in the fields of mechanics and higher mathematics. Students can access the applications online at any time, without installation or special requirements, and, thus, can actively explore the content with a very low entry barrier. For example, they can experiment with a spring‐mass‐damper system and let it enter the resonance by changing the frequency of the excitation; or they can search for local maxima and minima of a function with side conditions with the Lagrange multipliers in an interactive and graphical way. The interactive process enhances the understanding, even if the user does not interact with a strategy but just uses his intuition. These applications can be also used to explain complex concepts to the general public. The source code is publicly available on https://github.com/ChairOfStructuralMechanicsTUM/Mechanics_Apps .

Animated analysis of geoscientific datasets: An interactive graphical application

Geoscientists are required to analyze and draw conclusions from increasingly large volumes of data. There is a need to recognise and characterise features and changing patterns of Earth observables within such large datasets. It is also necessary to identify significant subsets of the data for more detailed analysis.We present an innovative, interactive software tool and workflow to visualise, characterise, sample and tag large geoscientific datasets from both local and cloud-based repositories. It uses an animated interface and human-computer interaction to utilise the capacity of human expert observers to identify features via enhanced visual analytics. ‘Tagger’ enables users to analyze datasets that are too large in volume to be drawn legibly on a reasonable number of single static plots. Users interact with the moving graphical display, tagging data ranges of interest for subsequent attention. The tool provides a rapid pre-pass process using fast GPU-based OpenGL graphics and data-handling and is coded in the Quartz Composer visual programing language (VPL) on Mac OSX. It makes use of interoperable data formats, and cloud-based (or local) data storage and compute.In a case study, Tagger was used to characterise a decade (2000–2009) of data recorded by the Cape Sorell Waverider Buoy, located approximately 10 km off the west coast of Tasmania, Australia. These data serve as a proxy for the understanding of Southern Ocean storminess, which has both local and global implications. This example shows use of the tool to identify and characterise 4 different types of storm and non-storm events during this time. Events characterised in this way are compared with conventional analysis, noting advantages and limitations of data analysis using animation and human interaction. Tagger provides a new ability to make use of humans as feature detectors in computer-based analysis of large-volume geosciences and other data.

An improved approach for measuring the impact of multiple CO2 conductances on the apparent photorespiratory CO2 compensation point through slope-intercept regression.

Biochemical models of leaf photosynthesis, which are essential for understanding the impact of photosynthesis to changing environments, depend on accurate parameterizations. One such parameter, the photorespiratory CO2 compensation point can be measured from the intersection of several CO2 response curves measured under sub-saturating illumination. However, determining the actual intersection while accounting for experimental noise can be challenging. Additionally, leaf photosynthesis model outcomes are sensitive to the diffusion paths of CO2 released from the mitochondria. This diffusion path of CO2 includes both chloroplastic as well as cell wall resistances to CO2 , which are not readily measurable. Both the difficulties of determining the photorespiratory CO2 compensation point and the impact of multiple intercellular resistances to CO2 can be addressed through application of slope-intercept regression. This technical report summarizes an improved framework for implementing slope-intercept regression to evaluate measurements of the photorespiratory CO2 compensation point. This approach extends past work to include the cases of both Rubisco and Ribulose-1,5-bisphosphate (RuBP)-limited photosynthesis. This report further presents two interactive graphical applications and a spreadsheet-based tool to allow users to apply slope-intercept theory to their data.

Second-Year Integrative Project for Computer and Software Engineering Students at Polytechnique Montréal

In 2005, Polytechnique Montréal introduced project-based learning (PBL) in its curriculum. The second-year integrative project for computer and software engineering students is software-oriented, as the main goal is to develop an interactive graphical software application. Students are given a software requirements specification (SRS), which details every feature the application must comply to. The students must then implement these features, starting from a provided framework, and various common software libraries. The main technical concepts integrated in the project are software design patterns, computer graphics, unit testing, and user interface design. Students are also provided with tools to put into practice project management concepts such as scope management, time management, cost management, and risk management. The second-year project is also a good opportunity for students to develop their non-technical skills, such as collaboration, communication and critical thinking.

Controllable data sampling in the space of human poses

AbstractMarkerless human pose recognition using a single‐depth camera plays an important role in interactive graphics applications and user interface design. Recent pose recognition algorithms have adopted machine learning techniques, utilizing a large collection of motion capture data. The effectiveness of the algorithms is greatly influenced by the diversity and variability of training data. We present a new sampling method that resamples a collection of human motion data to improve the pose variability and achieve an arbitrary size and level of density in the space of human poses. The space of human poses is high dimensional, and thus, brute‐force uniform sampling is intractable. We exploit dimensionality reduction and locally stratified sampling to generate either uniform or application specifically biased distributions in the space of human poses. Our algorithm learns to recognize such challenging poses as sitting, kneeling, stretching, and doing yoga using a remarkably small amount of training data. The recognition algorithm can also be steered to maximize its performance for a specific domain of human poses. We demonstrate that our algorithm performs much better than the Kinect software development kit for recognizing challenging acrobatic poses while performing comparably for easy upright standing poses. Copyright © 2015 John Wiley & Sons, Ltd.

A Hybrid Scheme Based on Pipelining and Multitasking in Mobile Application Processors for Advanced Video Coding

One of the key requirements for mobile devices is to provide high-performance computing at lower power consumption. The processors used in these devices provide specific hardware resources to handle computationally intensive video processing and interactive graphical applications. Moreover, processors designed for low-power applications may introduce limitations on the availability and usage of resources, which present additional challenges to the system designers. Owing to the specific design of the JZ47x series of mobile application processors, a hybrid software-hardware implementation scheme for H.264/AVC encoder is proposed in this work. The proposed scheme distributes the encoding tasks among hardware and software modules. A series of optimization techniques are developed to speed up the memory access and data transferring among memories. Moreover, an efficient data reusage design is proposed for the deblock filter video processing unit to reduce the memory accesses. Furthermore, fine grained macroblock (MB) level parallelism is effectively exploited and a pipelined approach is proposed for efficient utilization of hardware processing cores. Finally, based on parallelism in the proposed design, encoding tasks are distributed between two processing cores. Experiments show that the hybrid encoder is 12 times faster than a highly optimized sequential encoder due to proposed techniques.

Interactive Graphics for Presentation and Exploration of Student Performance Profiles in IGLU/TIMSS 2011 Educational Surveys

This paper advocates the application of interactive graphics as a qualitative research method for comparative large-scale assessments, known to supplement and extend analytical techniques. Graphics can be easily generated for effective display of information. Interaction is essential for exploring data in a flexible and controlled manner. The national results of the common sample of the large-scale educational assessment studies Internationale Grundschul-Lese-Untersuchung (IGLU)/ Trends in International Mathematics and Science Study (TIMSS) 2011 demonstrated the effectiveness of interactive graphics. The performance profiles of elementary school students in terms of the competency domains of reading, mathematics, and science are analyzed. Furthermore, spineplots are used to investigate the background characteristics of students of different performance types in order to identify possible educational disadvantage. The study aims to simplify data exploration before reporting, and to more effectively present the findings obtained from comparison studies on school achievement.

Iterative Smoothing of Curves and Surfaces

Algorithms for drawing smooth curves and surfaces are well-known but may be too expensive in CPU time to use for certain high speed interactive graphics applications. This paper looks at some new geometrically-based smoothing algorithms that arrive at the final smooth curve or surface after the application of an infinite number of iterations of the algorithm using subdivision refinement. When CPU time is scarce low iteration counts can be used to provide an acceptable level of approximation of smoothness and when CPU time is more plentiful higher levels of iteration can be applied for greater visual smoothing accuracy. Three new algorithms are presented and analysed. Convergence is proved geometrically for each and their timings are reported. Mixing iterations provides new opportunities for achieving various different effects in curve and surface smoothing.

Dynamical model to estimate carrying capacity in reservoirs with fish farming

We propose a dynamical model to estimate the carrying capacity of artificial water bodies with intensive fish farming. The dynamical model builds from the traditional phosphorus budget model by Dillon & Rigler (1974) on cage fish farming on artificial reservoirs. The model consists of two ordinary differential equations. The model determines biomass, fish and phosphorus temporal evolution. The model takes into account additional parameters such as phosphorus loading from external sources, variations on feeding regimes and the further influence of these parameters on the ecosystem eutrophication. Results show that the Dillon & Rigler model is a particular case of the new model by taking a constant feeding rate. In addition, an interactive graphical application was developed to simulate several scenarios, using the software Mathematica 7 ®. This work contributes to fish farming planning and management on artificial reservoirs and other water bodies.

Simulación del péndulo invertido rotacional usando Easy Java Simulations y Matlab

En este artículo se presenta el análisis, diseño y construcción de un simulador virtual, que representa gráficamente el comportamiento de un sistema no lineal como el del péndulo invertido rotacional. Esta interfaz se presenta por medio de un Applet de Java que permite a los usuarios hacer la variación de los parámetros del modelo, formando un puente entre los conceptos teóricos y los comportamientos reales del proceso. Del lado del servidor se utiliza Matlab/Simulink como motor de cálculo numérico dada su facilidad para construir modelos no lineales mediante diagramas de bloques. La interfaz de usuario ha sido diseñada mediante la herramienta de software gratuito Easy Java Simulations, que permite crear aplicaciones gráficas con alto grado de interactivi-dad como interfaces con objetos en 3D. Esta herramienta es de gran ayuda para la enseñanza del control automático.

Nonpinhole approximations for interactive rendering.

Researchers have used depth images to approximate scene geometry in a variety of interactive 3D graphics applications. Previous researchers constructed images using orthographic or perspective projection, which limits the approximation quality to what's visible along a single view direction or from a single viewpoint. Images constructed with nonpinhole cameras can improve approximation quality at little additional cost, if the camera offers fast projection. For such a camera, the fundamental operation of ray-and-depth-image intersection proceeds efficiently by searching along the 1D projection of the ray onto the image. A proposed method extends epipolar geometry constraints to nonpinhole cameras for two-camera configurations. Researchers have demonstrated nonpinhole depth images' advantages in the context of reflections, refractions, relief texture mapping, and ambient occlusion. The Web extra is a video that shows how nonpinhole depth images provide advantages regarding reflection, refraction, relief texture mapping, and ambient occlusion.

MDO-LWIGT: An Mobile Device Oriented Lightweight Interactive Graphics Toolkit

In the development of interactive graphics application on mobile devices, we are puzzled by two problems: The one is how to make graphics application adapt to the intrinsic limitations of mobile devices; The other one is how to rapidly and effectively build graphics application. This paper proposes a MDO-LWIGT(Mobile Device Oriented Lightweight Interactive Graphics Toolkit) to provide a graphics application model that can help developers rapidly build graphics application on mobile devices. The application based on such a model not only can be adaptive to the limitations of mobile devices but also can effectively represent and manage graphics objects, maitain the dynamic relationship between the application datas related to graphics objects.

Real-Time Shape Illustration Using Laplacian Lines

This paper presents a novel object-space line drawing algorithm that can depict shapes with view-dependent feature lines in real time. Strongly inspired by the Laplacian-of-Gaussian (LoG) edge detector in image processing, we define Laplacian lines as the zero-crossing points of the Laplacian of the surface illumination. Compared to other view-dependent feature lines, Laplacian lines are computationally efficient because most expensive computations can be preprocessed. We further extend Laplacian lines to volumetric data and develop the algorithm to compute volumetric Laplacian lines without isosurface extraction. We apply the proposed Laplacian lines to a wide range of real-world models and demonstrate that Laplacian lines are more efficient than the existing computer generated feature lines, and can be used in interactive graphics applications.

A deformation transformer for real-time cloth animation

Achieving interactive performance in cloth animation has significant implications in computer games and other interactive graphics applications. Although much progress has been made, it is still much desired to have real-time high-quality results that well preserve dynamic folds and wrinkles. In this paper, we introduce a hybrid method for real-time cloth animation. It relies on data-driven models to capture the relationship between cloth deformations at two resolutions. Such data-driven models are responsible for transforming low-quality simulated deformations at the low resolution into high-resolution cloth deformations with dynamically introduced fine details. Our data-driven transformation is trained using rotation invariant quantities extracted from the cloth models, and is independent of the simulation technique chosen for the lower resolution model. We have also developed a fast collision detection and handling scheme based on dynamically transformed bounding volumes. All the components in our algorithm can be efficiently implemented on programmable graphics hardware to achieve an overall real-time performance on high-resolution cloth models.