Reference View Research Articles

Estimation of Virtual View Synthesis Distortion Toward Virtual View Position.

We propose an analytical model to estimate the depth-error-induced virtual view synthesis distortion (VVSD) in 3D video, taking the distance between reference and virtual views (virtual view position) into account. In particular, we start with a comprehensive preanalysis and discussion over several possible VVSD scenarios. Taking intrinsic characteristic of each scenario into consideration, we specifically classify them into four clusters: 1) overlapping region; 2) disocclusion and boundary region; 3) edge region; and 4) infrequent region. We propose to model VVSD as the linear combination of the distortion under different scenarios (DDSs) weighted by the probability under different scenarios (PDSs). We show analytically that DDS and PDS can be related to the virtual view position using quadratic/biquadratic models and linear models, respectively. Experimental results verify that the proposed model is capable of estimating the relationship between VVSD and the distance between reference and virtual views. Therefore, our model can be used to inform a reference view setup for capturing, or distortion at certain virtual view positions, when depth information is compressed.

This and That Revisited: A Social and Multimodal Approach to Spatial Demonstratives.

As humans, we have the capacity to refer to the things in the world around us. In everyday spoken communication, we often use words to describe intended referents (such as objects, people, and events), and our bodies (e.g., eyes, head, and hands) to indicate the location to which our addressee should focus her attention in order to further identify what we are talking about (Buhler, 1934; Clark and Bangerter, 2004). Traditionally, referring has been described as an autonomous and addressee-blind act that speakers do on their own without taking into account beliefs about their addressees' knowledge about a referent (e.g., Olson, 1970; see Clark and Bangerter, 2004). In contrast, more recent views consider it rather a collaborative enterprise that requires that speaker and addressee work together, for instance in reaching mutual agreement on how to conceptualize and name a particular entity (e.g., Clark and Wilkes-Gibbs, 1986; Brennan and Clark, 1996; Clark and Bangerter, 2004). Such agreement is established through interaction, and the addressee is at least as important as the speaker in reaching agreement and establishing reference. In prototypical instances of successful referring, speakers often produce spatial demonstratives like this and that to establish joint attention between speaker and addressee to a visible entity (Buhler, 1934; Levinson, 1983). Such demonstratives are among the most frequently used words in language, among the first words infants produce (Clark and Sengul, 1978), and possibly primordial in phylogeny (Diessel, 2006; Tomasello, 2008). Surprisingly, despite the advances made toward a social, collaborative account of referring more generally, the prevailing theoretical view on spatial demonstratives has remained deeply individual and egocentric, as illustrated by the following claims: “[T]he anchoring point of deictic expressions is egocentric (or, better, speaker-centric). Adult speakers skillfully relate what they are talking about to this me-here-now” (Levelt, 1989, p. 46). Spatial demonstratives “indicate the relative distance of an object, location, or person vis-a-vis the deictic center (…), which is usually associated with the location of the speaker” (Diessel, 1999, p. 36). “[D]emonstratives are interpreted based on the speaker's body” ((Diessel, 2014), p. 122). This egocentric account is intuitively appealing and still influential (e.g., Diessel, 2014; Stevens and Zhang, 2014). In the current paper, we question this account from both the production and the comprehension side, and discuss recent accumulating observational, experimental, and neuroscientific evidence that suggests an alternative social and multimodal view of demonstrative reference.

Optimal reference view selection algorithm for low complexity disparity estimation

Multi-view video coding (MVC) is composed of multiple video sequences captured simultaneously by multiple closely spaced cameras. Such a system demands a higher computational complexity due to the increased number of cameras. Disparity estimation (DE) is one of the compression techniques used to remove inter-view redundancy in MVC, which causes high complexity. In order to overcome the problem of high computational complexity in real-time MVC, an optimal reference view selection algorithm is proposed. The proposed scheme decreases the coding complexity by reducing one-directional DE operation and minimizing the decrease in coding performance in MVC.

Reference View Selection in DIBR-Based Multiview Coding.

Augmented reality, interactive navigation in 3D scenes, multiview video, and other emerging multimedia applications require large sets of images, hence larger data volumes and increased resources compared with traditional video services. The significant increase in the number of images in multiview systems leads to new challenging problems in data representation and data transmission to provide high quality of experience on resource-constrained environments. In order to reduce the size of the data, different multiview video compression strategies have been proposed recently. Most of them use the concept of reference or key views that are used to estimate other images when there is high correlation in the data set. In such coding schemes, the two following questions become fundamental: 1) how many reference views have to be chosen for keeping a good reconstruction quality under coding cost constraints? And 2) where to place these key views in the multiview data set? As these questions are largely overlooked in the literature, we study the reference view selection problem and propose an algorithm for the optimal selection of reference views in multiview coding systems. Based on a novel metric that measures the similarity between the views, we formulate an optimization problem for the positioning of the reference views, such that both the distortion of the view reconstruction and the coding rate cost are minimized. We solve this new problem with a shortest path algorithm that determines both the optimal number of reference views and their positions in the image set. We experimentally validate our solution in a practical multiview distributed coding system and in the standardized 3D-HEVC multiview coding scheme. We show that considering the 3D scene geometry in the reference view, positioning problem brings significant rate-distortion improvements and outperforms the traditional coding strategy that simply selects key frames based on the distance between cameras.

Rate Distortion Optimized Inter-View Frame Level Bit Allocation Method for MV-HEVC

In multi-view video coding, since inter-view prediction has been adopted as an important coding tool which could improve coding efficiency greatly, inter-view dependency is inevitable, i.e., the distortion of the reference view (RV) picture could be propagated to the non-reference view (NRV) pictures . Therefore, in order to achieve higher coding efficiency , the inter-view dependency must be taken into account for inter-view bit allocation. In this paper, the inter-view dependency is analyzed in detail, and a rate-distortion (RD) model for NRVs is derived by taking the distortion of RV into account. Based on the derived RD model, the inter-view bit allocation is represented as a mathematical problem with an analytic form, and is solved by a convex optimization (Lagrangian Multiplier) method. Experimental results demonstrate that the RD performance and the inter-view quality consistency of the proposed method is better than existing methods, while the complexity of the proposed method is comparable with the existing methods.

Faithful Disocclusion Filling in Depth Image Based Rendering Using Superpixel-Based Inpainting

Disocclusion filling is a critical problem in depth- based view synthesis. Exposed regions in the target view that correspond to occluded areas in the reference view have to be filled in a meaningful way. Current approaches aim to do this in a plausible way, mostly inspired by image inpainting techniques . However, disocclusion filling is a video-based problem which exhibits more information than just the current frame. By utilizing texture found in temporally adjacent frames, we propose to fill disocclusions in a faithful way, i.e., using texture that a real camera would observe in place of the virtual camera. Only if faithful information is not available we fall back to plausible filling. Our approach is designed for single view video-plus-depth where neighboring camera views are not available for disocclusion filling. In contrast to previous approaches , our method uses superpixels instead of square patches as filling entities to reduce the amount of artifacts introduced into the filling region. Despite its importance , faithfulness has not obtained the due attention yet. Our experiments show that situations are common where a simple plausible filling does not lead to satisfying filling results. Thus, it is important to stress faithful disocclusion filling. Our current work is an attempt in this direction.

Encoder-Driven Inpainting Strategy in Multiview Video Compression.

In free viewpoint video systems, a user has the freedom to select a virtual view from which an image of the 3D scene is rendered, and the scene is commonly represented by color and depth images of multiple nearby viewpoints. In such representation, there exists data redundancy across multiple dimensions: 1) a 3D voxel may be represented by pixels in multiple viewpoint images (inter-view redundancy); 2) a pixel patch may recur in a distant spatial region of the same image due to self-similarity (inter-patch redundancy); and 3) pixels in a local spatial region tend to be similar (inter-pixel redundancy). It is important to exploit these redundancies during inter-view prediction toward effective multiview video compression. In this paper, we propose an encoder-driven inpainting strategy for inter-view predictive coding, where explicit instructions are transmitted minimally, and the decoder is left to independently recover remaining missing data via inpainting, resulting in lower coding overhead. In particular, after pixels in a reference view are projected to a target view via depth-image-based rendering at the decoder, the remaining holes in the target view are filled via an inpainting process in a block-by-block manner. First, blocks are ordered in terms of difficulty-to-inpaint by the decoder. Then, explicit instructions are only sent for the reconstruction of the most difficult blocks. In particular, the missing pixels are explicitly coded via a graph Fourier transform or a sparsification procedure using discrete cosine transform, leading to low coding cost. For blocks that are easy to inpaint, the decoder independently completes missing pixels via template-based inpainting. We apply our proposed scheme to frames in a prediction structure defined by JCT-3V where inter-view prediction is dominant, and experimentally we show that our scheme achieves up to 3-dB gain in peak-signal-to-noise-ratio in reconstructed image quality over a comparable 3D-High Efficiency Video Coding implementation using fixed 16 $\times $ 16 block size.

Principles of Transformative Learning Workshop (USAFA January 2015)

This six-hour workshop (video links below) was delivered to a group of Air Force professionals. This workshop aimed to ground participants in the principles of transformative learning so that they were equipped to design their own unique methods and models. The following was given to participants before they arrived at the workshop:Why do such a workshop?The Air Force is committed to developing leaders who embody the core values and model what it means to be a member of the Profession of Arms. This workshop answers the question of why some Air Force development efforts are effective and why some efforts are not effective. This workshop is designed to produce a breakthrough in your capacity to design and deliver professional development curriculum.Who this workshop is for: • Individuals who have an accountability to design and/or deliver curriculum to military service members in the areas of leadership, character, and the Profession of Arms, and • Individuals who are interested in becoming more effective in developing others from the Air Force’s newest Airmen to the most senior leaders.What you will take away from this workshop:• An answer to the question, “why do some personal and organizational interventions work some of the time but not all the time, and with some people but not all people?”• The difference between informative learning and transformative learning.• A new access to create 1st-person “on the court” learning experiences as contrasted with providing 3rd-person “in the stands” conceptual understanding.• An elevated capacity to design curriculum that gets to the core of what it takes to alter one’s orientation (view or frame of reference) towards oneself, others, the profession, and leadership situations.• The importance of, and how to create, choice, responsibility and commitment in learning and development efforts.Videos: Part 1Part 2Part 3Part 4

Sequential block-based disparity map estimation algorithm for stereoscopic image coding

This paper deals with the problem of block-based disparity map estimation for stereoscopic image coding where the estimated map is transmitted to the decoder in order to predict one view from the other. The estimation problem of the disparity map is thus a trade-off between the quality prediction and the binary cost of the disparities to be stored or transmitted. This trade-off is modeled as a joint entropy-distortion metric assuming that the disparity map is encoded with an entropy coder; and one of the two views is fully predicted using this map when applied to the other view. However minimizing this joint metric is a complex combinatorial optimization problem where choices of disparities are all interrelated. A sub-optimal optimization solution is then proposed. It is based on a tree structure which is constructed sequentially whenever a block is matched. The developed algorithm, called Modified M-Algorithm (MMA), processes the reference view in a raster scanning order and assumes that the disparities to be selected in the unprocessed area are likely to follow a chosen disparity distribution. This algorithm has the ability at each step of the process not only to retain the M-best paths of the tree in terms of entropy-distortion cost but also to explore all possible extensions of each of these M paths until reading the last block of the view. Simulations, conducted on stereoscopic images extracted from Middlebury and Deimos datasets, show the advantage of our MMA compared to the conventional Block Matching Algorithm (BMA) with and without regularization both in terms of reducing bitrate and distortion.

Influence of management behavior on the skilled labor migrations’ unsafe behavior

Purpose: The purpose of this study was to summarize safety management of manager into two aspects (design behavior and management behavior) and to figure out the different impact these two behaviors might have. Design/methodology/approach: In order to verify the reasonableness of the assumptions, expert investigation was used by the means of semi-structured interview. And the Structural Equation Modeling?SEM? is estimated using 850 individual questionnaire responses from five companies in the form of Likert-type scale. What’s more, taking the measurement error causing by common method biases into consideration, Univariate Testing was taken to measure the deviation effect. Findings: The results obtained with this description showed that certain measures should be adopt by managers to develop purposively the safety knowledge and safety motivation of the skilled labor migrations (SLMs). Research limitations/implications: Unsafe behavior, which has aroused extensive concern in recent years, is the subject of many safety management studies. However, there have not been any studies on the influence of management behavior on SLMs unsafe behavior. Practical implications: As the unsafe behavior of SLMs is the most important accident reason, this paper may help reduce the incidence of accidents. Originality/value: The conclusion of this study will certainly provide the beneficial reference views on the management behavior.

Object Files, Properties, and Perceptual Content

Object files are mental representations that enable perceptual systems to keep track of objects as numerically the same. How is their reference fixed? A prominent approach, championed by Zenon Pylyshyn and John Campbell, makes room for a non-satisfactional use of properties to fix reference. This maneuver has enabled them to reconcile a singularist view of reference with the intuition that properties must play a role in reference fixing. This paper examines Campbell’s influential defense of this strategy. After criticizing it, a new approach is sketched. The alternative view introduces representational contents to explain perceptual individuation. After arguing that those contents are not satisfactional, it is concluded that there is room for a third view of reference fixing that does not fit into the singularist/descriptivist dichotomy.

View Synthesis Distortion Estimation With a Graphical Model and Recursive Calculation of Probability Distribution

Depth-image-based rendering (DIBR) is frequently used in multiview video applications such as free-viewpoint television. In this paper, we consider the two DIBR algorithms used in the Moving Picture Experts Group view synthesis reference software, and develop a scheme for the encoder to estimate the distortion of the synthesized virtual view at the decoder when the reference texture and depth sequences experience transmission errors such as packet loss. We first develop a graphical model to analyze how random errors in the reference depth image affect the synthesized virtual view. The warping competition rule adopted in the DIBR algorithms is explicitly represented by the graphical model. We then consider the case where packet loss occurs to both the encoded texture and depth images during transmission and develop a recursive optimal distribution estimation (RODE) method to calculate the per-pixel texture and depth probability distributions in each frame of the reference views. The RODE is then integrated with the graphical model method to estimate the distortion in the synthesized view caused by packet loss. Experimental results verify the accuracy of the graphical model method, the RODE, and the combined estimation scheme.

On Teaching Future Time To EFL Learners: Problems And Solutions

This study in all its overall presentation seeks to give a comprehensive account of the difficulties involved in teaching future time to students who learn English as a foreign language; and the (pedagogical) solutions humbly suggested for learners, teachers, text-book writers, linguists and psychologists since they are expected to be the best who can deal with the problems that impede the acquisition of the foreign language concepts. The work like many language teachers' works is a reaction to the frustrating state of the students who cannot use their knowledge in a real communicative situation. It spotlights the reasons of students' inefficient use of the English future structures; and projects, through a questionnaire, the most possible reasons of this problem propounding some useful techniques to overcome the problem depending on what is written in the literature of language learning and teaching. The study tackles the topic of tense and time as an entrance to investigate the nature of future structures. Apart from presenting a pedagogical view of future time references, it provides almost all the constructions used to express futurity and the indications they refer to getting use of the idea of Fleischman's time-line. The study ends with many results, findings and pedagogical suggestions. Keyword: Futurity, Pedagogy and pedagogical grammar, EFL acquisition, Tense and time, Time-Line

Insights gained from three-dimensional imaging modalities for closure of ventricular septal defects.

Ventricular septal defects (VSDs) are the commonest group of congenital heart lesions, accounting for ≈30% of congenital heart defects in liveborn infants.1 Defects in the ventricular septum are classified according to their position when the ventricular septum is viewed from the right ventricle and these may vary in their location, size, and number.2,3 VSDs can occur as isolated lesions or as part of more complex disease, for example, transposition of the great arteries or double outlet right ventricle, where the location and size of such defects assume great importance in planning the optimal mode of surgical repair.4 In most cases, surgical closure remains the mainstay treatment for VSDs, but selected lesions may be amenable to catheter closure.5,6 Cross-sectional echocardiography is the most commonly used noninvasive diagnostic method to delineate the position of a VSD before surgery.7 Multiple sonographic cuts and cross-sectional sweeps of the region of interest are used to build a virtual reconstruction of the three-dimensional (3D) anatomy in the mind of the echocardiographer. However, for the surgeon who deals solely with 3D images there can remain an element of uncertainty when called to operate on the patient. The number, size, location of VSDs, and their relationship to adjacent structures are central for selecting whether the defects might be best closed via the tricuspid valve, via the semilunar valves, or via a ventriculotomy or whether these might not be readily approachable and a palliative pulmonary artery banding is desirable.8,9 In this review, we present key 3D echocardiographic projections and examples of different types of VSDs where the images have a direct effect on the approach taken to VSD closure. Comparison of 3D echocardiography (3DE) with other 3D imaging modalities is also described. Images are presented in an …

Improved interview video error concealment on whole frame packet loss

An improved DIBR-based (Depth image based rendering) whole frame error concealment method for multiview video with depth is designed. An optimal reference view selection is first proposed. The paper further includes three modified parts for the DIBRed pixels. First, the missing 1-to-1 pixels are concealed by the pixels from another view. The light differences between views are taken care of by the information of the motion vector of the projected coordination and a reverse DIBR procedure. Second, the generation of the many-to-1 pixels is improved via their depth information. Third, the hole pixels are found using the estimated motion vectors derived efficiently from a weighted function of the neighboring available motion vectors and their distance to the target hole pixel. The experimental results show that, compared to the state-of-the-art method, the combined system of the four proposed methods is superior and improves the performance by 5.53dB at maximum.

Present and future climatologies in the phase I CREMA experiment

We provide an overall assessment of the surface air temperature and precipitation present day (1976–2005) and future (2070–2099) ensemble climatologies in the Phase I CREMA experiment. This consists of simulations performed with different configurations (physics schemes) of the ICTP regional model RegCM4 over five CORDEX domains (Africa, Mediterranean, Central America, South America, South Asia), driven by different combinations of three global climate models (GCMs) and two greenhouse gas (GHG) representative concentration pathways (RCP8.5 and RCP4.5). The biases (1976–2005) in the driving and nested model ensembles compared to observations show a high degree of spatial variability and, when comparing GCMs and RegCM4, similar magnitudes and more similarity for precipitation than for temperature. The large scale patterns of change (2070–2099 minus 1976–2005) are broadly consistent across the GCM and RegCM4 ensembles and with previous analyses of GCM projections, indicating that the GCMs selected in the CREMA experiment are representative of the more general behavior of current GCMs. The RegCM4, however, shows a lower climate sensitivity (reduced warming) than the driving GCMs, especially when using the CLM land surface scheme. While the broad patterns of precipitation change are consistent across the GCM and RegCM4 ensembles, greater differences are found at sub-regional scales over the various domains, evidently tied to the representation of local processes. This paper serves to provide a reference view of the behavior of the CREMA ensemble, while more detailed and process-based analysis of individual domains is left to companion papers of this special issue.

Projection-based disparity control for toed-in multiview images.

The multiview images captured by toed-in camera array can reproduce the 3D scene vividly with appropriate positive, negative, and zero disparities. However, it is a challenging task to adjust the depth of the scene according to requirements of visual effects. In this paper, we propose a novel disparity control method based on projection to solve this problem. With the relationship between the world coordinate system and camera coordinate system, the zero disparity point in reference view is projected into other views. Thus, disparities of different views are obtained through matched corresponding points and views are shifted with calculated disparities. The proposed method is easy to implement, and the depth of toed-in multiview images can be adjusted as requirements. Experiment results show that the proposed method is effective in comparison to the conventional method, and the processed multiview images present desirable stereoscopic visual quality.

Efficient multi-view video coding using inter-view information

Multi-view video coding (MVC) has been extended from H.264/AVC to improve the coding efficiency of multi-view video. This paper proposes a fast mode decision algorithm which can make an early decision on the correct mode partition to solve the issue of the enormous computational complexity. The best modes of the reference views are utilized to determine the complexity of the macroblock (MB) in the current view, the mode candidates needed to be calculated can then be obtained according to the complexity. If the complexity is low or medium, the search range can be reduced. The threshold of the rate-distortion cost for the current MB is calculated using the co-located and neighboring MBs in previously coded view and is utilized as the criterion for early termination. The motion vector difference in the reference view is applied to dynamically adjust the search range in the current MB. Experimental results prove that the proposed algorithm achieves a time saving of 81.05% for a fast TZ search and 87.85% for full search, and still maintains quality performance and bitrate.

Block-Based In-Loop View Synthesis for 3-D Video Coding

View synthesis prediction (VSP) employs a synthesized picture as a reference picture for current-view texture coding, which is an advanced disparity-compensated prediction. However, the picture-based view synthesis demands huge complexity, especially for decoders. Therefore, we propose a block-based in-loop view synthesis scheme which generates VSP samples only for blocks using VSP modes (called target blocks). For a target block, a window in reference view is estimated. Then, pixels within the window are warped to the current view, producing VSP samples for the target block. The proposed method turns the picture-level VSP sample generation into macroblock-level process, and significantly reduces complexity of the VSP module while maintaining coding efficiency.

Seamless View Synthesis Through Texture Optimization

In this paper, we present a novel view synthesis method named Visto, which uses a reference input view to generate synthesized views in nearby viewpoints. We formulate the problem as a joint optimization of inter-view texture and depth map similarity, a framework that is significantly different from other traditional approaches. As such, Visto tends to implicitly inherit the image characteristics from the reference view without the explicit use of image priors or texture modeling. Visto assumes that each patch is available in both the synthesized and reference views and thus can be applied to the common area between the two views but not the out-of-region area at the border of the synthesized view. Visto uses a Gauss–Seidel-like iterative approach to minimize the energy function. Simulation results suggest that Visto can generate seamless virtual views and outperform other state-of-the-art methods.