Reference View Research Articles

Foreground Removal Approach for Hole Filling in 3D Video and FVV Synthesis

The depth-image-based rendering is a key technique for 3D video and free viewpoint video synthesis. One of the critical problems in current synthesis methods is that the background (BG) occluded by the foreground objects might be exposed in the new view, and some holes are produced in the synthesized video. However, most of the traditional hole-filling approaches may bring some blurry effect or artifacts in the virtual view. In this paper, a foreground removal approach for hole filling is proposed, in which the foreground objects are removed from both the 2D video and its corresponding depth map, and then a BG video and its depth map are generated before the 3D warping and used to eliminate the holes in the synthesized video. Moreover, a BG extension method is applied in the reference view to prevent the large holes occurring along the border areas in the virtual view. Our analysis and experimental results have indicated that the proposed approach has better performance compared with the other methods in terms of the quality of synthesized video, computational complexity, and running time in multiview synthesis or multiframe synthesis.

Symmetry, outliers, and geodesics in coronary artery centerline reconstruction from rotational angiography.

The performance of many state-of-the-art coronary artery centerline reconstruction algorithms in rotational angiography heavily depends on accurate two-dimensional centerline information that, in practice, is not available due to segmentation errors. To alleviate the need for correct segmentations, we propose generic extensions to symbolic centerline reconstruction algorithms that target symmetrization, outlier rejection, and topology recovery on asymmetrically reconstructed point clouds. Epipolar geometry- and graph cut-based reconstruction algorithms are used to reconstruct three-dimensional point clouds from centerlines in reference views. These clouds are input to the proposed methods that consist of (a) merging of asymmetric reconstructions, (b) removal of inconsistent three-dimensional points using the reprojection error, and (c) projection domain-informed geodesic computation. We validate our extensions in a numerical phantom study and on two clinical datasets. In the phantom study, the overlap measure between the reconstructed point clouds and the three-dimensional ground truth increased from 68.4 ± 9.6% to 85.9 ± 3.3% when the proposed extensions were applied. In addition, the averaged mean and maximum reprojection error decreased from 4.32 ± 3.03 mm to 0.189 ± 0.182 mm and from 8.39 ± 6.08 mm to 0.392 ± 0.434 mm. For the clinical data, the mean and maximum reprojection error improved from 1.73 ± 0.97 mm to 0.882 ± 0.428 mm and from 3.83 ± 1.87 mm to 1.48 ± 0.61 mm, respectively. The application of the proposed extensions yielded superior reconstruction quality in all cases and effectively removed erroneously reconstructed points. Future work will investigate possibilities to integrate parts of the proposed extensions directly into reconstruction.

Wide-Baseline Foreground Object Interpolation Using Silhouette Shape Prior.

We consider the synthesis of intermediate views of an object captured by two widely spaced and calibrated cameras. This problem is challenging because foreshortening effects and occlusions induce significant differences between the reference images when the cameras are far apart. That makes the association or disappearance/appearance of their pixels difficult to estimate. Our main contribution lies in disambiguating this ill-posed problem by making the interpolated views consistent with a plausible transformation of the object silhouette between the reference views. This plausible transformation is derived from an object-specific prior that consists of a nonlinear shape manifold learned from multiple previous observations of this object by the two reference cameras. The prior is used to estimate the evolution of the epipolar silhouette segments between the reference views. This information directly supports the definition of epipolar silhouette segments in the intermediate views, as well as the synthesis of textures in those segments. It permits to reconstruct the epipolar plane images (EPIs) and the continuum of views associated with the EPI volume, obtained by aggregating the EPIs. Experiments on synthetic and natural images show that our method preserves the object topology in intermediate views and deals effectively with the self-occluded regions and the severe foreshortening effect associated with wide-baseline camera configurations.

Multi-view visual surveillance and phantom removal for effective pedestrian detection

To increase the robustness of detection in intelligent video surveillance systems, homography has been widely used to fuse foreground regions projected from multiple camera views to a reference view. However, the intersections of non-corresponding foreground regions can cause phantoms. This paper proposes an algorithm based on geometry and colour cues to cope with this problem, in which the homography between different camera views and the Mahalanobis distance between the colour distributions of every two associated foreground regions are considered. The integration of these two matching algorithms improves the robustness of the pedestrian and phantom classification. Experiments on real-world video sequences have shown the robustness of this algorithm.

The reference view for semantic interoperability in Integrated Product Development Process: The conceptual structure for injecting thin walled plastic products

Abstract The ongoing sophistication in costumer's needs have increased the complexity in the Integrated Product Development Process (IPDP). This remarkable growth demands that systems offer support to many different stages of IPDP, going across early planning and practical development phases. Concurrently, the level of detail and quantity of information are exponentially growing in this applications. However, there was also an increase in the number of semantic obstacles to sharing information across systems. These semantic obstacles are mainly related to the heterogenic nature of the information, which has its captured meaning interpreted in a divergent way, increasing the project costs and development time. In this context, this article contributes to the development of the core ontologies from the Reference View, to aid the semantic interoperability in the Product Design stage, further improving the exchange of information during different phases of the IPDP. The core ontologies research is focused on supporting the development of a plastic injected product, gathering and sharing information across the domains of part and mould design, mould manufacture and materials selection. The experimental system has demonstrated the potential to aid the exchange of information and inconsistency analysis in the Integrated Product Development Process.

Efficient multiview video plus depth coding for 3D-HEVC based on complexity classification of the treeblock

High efficiency video coding standard-based 3D video (3D-HEVC) has been extended from HEVC to improve the coding efficiency of multiview video plus depth (MVD). Similar to the joint model of HEVC, a computationally expensive exhaustive mode decision is performed to find the least rate-distortion cost for each treeblock in 3D-HEVC. Furthermore, additional coding tools have been added to 3D-HEVC for improving the coding efficiency of the dependent texture video and depth map. Those tools achieve the highest possible coding efficiency, but also bring a significant computational complexity which limits 3D-HEVC from real-time applications. In order to reduce computational complexity, we propose an efficient multiview video plus depth coding algorithm for 3D-HEVC that adaptively utilizes the complexity classification of the treeblock. The coding complexity model of a treeblock is first analyzed according to the prediction mode and coding mode from the corresponding treeblocks in the reference views. Based on the complexity classification model of the treeblock, we propose two efficient low-complexity approaches, including fast mode size decision and adaptive motion search range selection. Extensive experimental results demonstrate that the proposed MVD coding algorithm can achieve the average computational saving about 60.1% with negligible rate-distortion performance loss in comparison with the original 3D-HEVC encoder.

Analysis of Packet-Loss-Induced Distortion in View Synthesis Prediction-Based 3D Video Coding.

View synthesis prediction (VSP) is a crucial coding tool for improving compression efficiency in the next generation 3D video systems. However, VSP is susceptible to catastrophic error propagation when multi-view video plus depth (MVD) data are transmitted over lossy networks. This paper aims at accurately modeling the transmission errors propagated in the inter-view direction caused by VSP. Toward this end, we first study how channel errors gradually propagate along the VSP-based inter-view prediction path. Then, a new recursive model is formulated to estimate the expected end-to-end distortion caused by those channel losses. For the proposed model, the compound impact of the transmission distortions of both the texture video and depth map on the quality of the synthetic reference view is mathematically analyzed. Especially, the expected view synthesis distortion due to depth errors is characterized in the frequency domain using a new approach, which combines the energy densities of the reconstructed texture image and the channel errors. The proposed model also explicitly considers the disparity rounding operation invoked for the sub-pixel precision rendering of the synthesized reference view. Experimental results are presented to demonstrate that the proposed analytic model is capable of effectively modeling the channel-induced distortion for MVD-based 3D video transmission.

Hole Filling Method for Depth Image Based Rendering Based on Boundary Decision

In three-dimensional display systems, depth image based rendering is the most commonly used technique for generating images captured from a virtual viewpoint through reference views and depth maps. However, disoccluded hole filling remain a challenging issue as newly exposed area appears in the virtual view. Image inpainting based hole filling is a popular approach for filling in the hole, but the conventional methods generate annoying artifacts along the boundaries of the foreground objects. This letter presents a robust hole filling method that generates a virtual view with high quality. First, by using the local and neighboring property of the depth map, the background region is filled prior to the foreground region. Also, in the matching step to find the data to fill in, depth homogeneity is considered. Finally, based on the boundary decision using the depth property, the boundary regions are filled with background-relevant data to reduce boundary artifacts. Experimental results show that the views synthesized by the proposed method achieve higher visual comfort than the view synthesized by the existing methods.

Pseudo-Sequence-Based 2-D Hierarchical Coding Structure for Light-Field Image Compression

In this paper, we present a novel pseudo sequence based 2-D hierarchical reference structure for light-field image compression. In the proposed scheme, we first decompose the light-field image into multiple views and organize them into a 2-D coding structure according to the spatial coordinates of the corresponding microlens. Then we mainly develop three technologies to optimize the 2-D coding structure. First, we divide all the views into four quadrants, and all the views are encoded one quadrant after another to reduce the reference buffer size as much as possible. Inside each quadrant, all the views are encoded hierarchically to fully exploit the correlations between different views. Second, we propose to use the distance between the current view and its reference views as the criteria for selecting better reference frames for each inter view. Third, we propose to use the spatial relative positions between different views to achieve more accurate motion vector scaling. The whole scheme is implemented in the reference software of High Efficiency Video Coding. The experimental results demonstrate that the proposed novel pseudo-sequence based 2-D hierarchical structure can achieve maximum 14.2% bit-rate savings compared with the state-of-the-art light-field image compression method.

Reduced Reference Quality Metric for Synthesized Virtual Views in 3DTV

Multi-view video plus depth (MVD) has been widely used owing to its effectiveness in three-dimensional data representation. Using MVD, color videos with only a limited number of real viewpoints are compressed and transmitted along with captured or estimated depth videos. Because the synthesized views are generated from decoded real views, their original reference views do not exist at either the transmitter or receiver. Therefore, it is challenging to define an efficient metric to evaluate the quality of synthesized images. We propose a novel metric—the reduced-reference quality metric. First, the effects of depth distortion on the quality of synthesized images are analyzed. We then employ the high correlation between the local depth distortions and local color characteristics of the decoded depth and color images, respectively, to achieve an efficient depth quality metric for each real view. Finally, the objective quality metric of the synthesized views is obtained by combining all the depth quality metrics obtained from the decoded real views. The experimental results show that the proposed quality metric correlates very well with full reference image and video quality metrics.

Habitar, percibir y narrar el territorio. La construcción subjetiva de una tensión rural/urbana

<p>Para los habitantes nativos de Las Calles —una localidad del Valle de Traslasierra, en la provincia de Córdoba, Argentina— desde hace diez años ‘su’ pueblo comenzó a cambiar debido al impacto de la migración urbana. La sociedad receptora entiende las trasformaciones fisonómicas de la comunidad a partir de la referencia temporal ‘antes/ahora’ y percibe el arribo de citadinos al poblado como invasivo. El propósito de este trabajo es reflexionar sobre las transformaciones espaciales, identitarias<br />y relacionales de esta pequeña localidad a partir de los relatos identitarios de los habitantes autóctonos. Se indagan interpretaciones y percepciones que se nutren de imaginarios diversos en torno a nociones como desarrollo, progreso, pertenencia, bienestar, trabajo, espacio, tiempo y, sobre todo, en cuanto a la percepción de quién es el ‘otro’. En este contexto, la diversidad de procedencias, pertenencias, tradiciones y valores ponen en juego la configuración de la identidad local.</p>

Fronto‐parallel building facades stitching

Parallax handling is a challenging task for image stitching. Existing image stitching methods are quite sensitive to large parallax. One common phenomenon, e.g. is the ghosting. In this Letter, a flexible building facade stitching method that allows large parallax is proposed. Using the rigidity constraints of the man-made environment, each view can be aligned to a reference view. Excellent stitching performance demonstrates that the proposed method is superior to the state-of-the-art ones.

Heterogeneity in the Structure of the Left Ventricular Outflow Tract: A 3-Dimensional Transesophageal Echocardiographic Study.

The left ventricular outflow tract (LVOT) is a composite of adjoining structures; therefore, a circular or elliptical shape at one point may not represent its entire structure. The purpose of this study was to evaluate the presence of heterogeneity in the LVOT. Patients with normal valvular and ventricular function undergoing elective coronary revascularization surgery were included in the study. Intraoperative R-wave gated 3-dimensional (3D) transesophageal echocardiographic imaging of the LVOT was performed at end-systole, with the midesophageal long axis as the reference view. Acquired data were analyzed with the Philips Q-Lab software with multiplanar reformatting in the sagittal (minor axis), transverse (major axis), and coronal (cross-sectional area by planimetry) views of the LVOT. These measurements were made on the left ventricular side or proximal LVOT, aortic side, or distal LVOT and mid-LVOT. Fifty patients were included in the study. The LVOT minor (sagittal) axis dimension did not differ across the mid-LVOT, proximal LVOT, and distal LVOT (P = .11). The major axis diameter of LVOT differed among the 3 regions of the LVOT (P < .001). A difference in major axis diameter was observed between the proximal and the distal LVOT (median difference of 0.39 cm; Bonferroni-adjusted 95% confidence interval [CI] of the difference = 0.31-0.48 cm; Bonferroni-adjusted P < .001). Planimetry of the LVOT area differed significantly (P < .001) between the regions analyzed, and we found a difference between the distal and the proximal LVOT (median difference = 0.65 cm, Bonferroni-adjusted 95% CI of the difference = 0.44-0.88 cm, Bonferroni-adjusted P < .001). The LVOT area calculated from minor axis diameter differed significantly from the area obtained by planimetry (P < .001). There was heterogeneity in the major axis diameter and cross-sectional area for the different regions of the LVOT. The distal LVOT (aortic side) was more circular, whereas the proximal LVOT (left ventricular side) was more elliptical in shape. This change in shape from circular to elliptical was accounted for by a difference in the major axis diameter from proximal to distal LVOT and a relatively similar minor axis diameter. Although the clinical significance of this finding is unknown, the assumption of a uniform structure of LVOT is incorrect. Three-dimensional imaging may be useful for assessing the LVOT shape and size at a specific region of interest.

Hole Filling Algorithm for a Virtual-viewpoint Image by Using a Modified Exemplar Based In-painting

In this paper, a new algorithm by using 3D warping technique to effectively fill holes that are produced when creating a virtual-viewpoint image is proposed. A hole is defined as the region that cannot be seen in the reference view when a virtual view is created. In the proposed algorithm, to reduce the blurring effect that occurs on the hole region filled by conventional algorithms and to enhance the texture quality of the generated virtual view, Exemplar Based In-painting algorithm is used. The boundary noise which occurs in the initial virtual view obtained by 3D warping is also removed. After 3D warping, we estimate the relative location of the background to the holes and then pixels adjacent to the background are filled in priority to get better result by not using only adjacent objects information. Also, the temporal inconsistency between frames can be reduced by expanding the search region up to the previous frame when searching for most similar patch. The superiority of the proposed algorithm compared to the existing algorithms can be shown through the experimental results.

Multi-view and multi-plane data fusion for effective pedestrian detection in intelligent visual surveillance

For the robust detection of pedestrians in intelligent video surveillance, an approach to multi-view and multi-plane data fusion is proposed. Through the estimated homography, foreground regions are projected from multiple camera views to a reference view. To identify false-positive detections caused by foreground intersections of non-corresponding objects, the homographic transformations for a set of parallel planes, which are from the head plane to the ground, are applied. Multiple features including occupancy information and colour cues are extracted from such planes for joint decision-making. Experimental results on real world sequences have demonstrated the good performance of the proposed approach in pedestrian detection for intelligent visual surveillance.

In-Network View Synthesis for Interactive Multiview Video Systems

In multiview applications, camera views can be used as reference views to synthesize additional virtual viewpoints, allowing users to freely navigate within a 3D scene. However, bandwidth constraints may restrict the number of reference views sent to clients, limiting the quality of the synthesized viewpoints. In this work, we study the problem of in-network reference view synthesis aimed at improving the navigation quality at the clients. We consider a distributed cloud network architecture, where data stored in a main cloud is delivered to end users with the help of cloudlets, i.e., resource-rich proxies close to the users. We argue that, in case of limited bandwidth from the cloudlet to the users, re-sampling at the couldlet the viewpoints of the 3D scene (i.e., synthesizing novel virtual views in the cloudlets to be used as new references to the decoder) is beneficial compared to mere subsampling of the original set of camera views. We therefore cast a new reference view selection problem that seeks the subset of views minimizing the distortion over a view navigation window defined by the user under bandwidth constraints. We prove that the problem is NP-hard, and we propose an effective polynomial time algorithm using dynamic programming to solve the optimization problem under general assumptions that cover most of the multiview scenarios in practice. Simulation results confirm the performance gain offered by virtual view synthesis in the network.

Keeping Up WithWhichJoneses: Spatial Diffusion of Rule of Law Through Economic International Organizations

I develop a theory of spatial diffusion of the rule of law, where “space” is conceptualized as shared memberships in economic international organizations (IOs). I argue that the rule of law diffuses as a result of economic competition and socialization. Outside evaluators, such as international leaders, activists, and most importantly, international firms and investors, often assess states’ attractiveness as a business venue by comparing them to similar states. The natural reference group for such comparisons is not just geographical neighbors, but also states with shared memberships in economic IOs. Responding to this evaluation, states identify members of their own reference groups and view them as competition for investment. As a result, states within the same reference groups converge on issues related to lowering domestic economic risks, which are referred to by the umbrella term “domestic rule of law.” The resulting process of policy convergence is further aided by socialization. Socialization may involve emulation of policies of similar states or “active” learning through norm entrepreneurs, who use the reference groups for both evaluation and access to their target audiences. I capture the theorized spatial processes using a multiparametric spatiotemporal autoregressive model (m-STAR) and find support for the prediction.

Enhanced view random access ability for multiview video coding

Apart from the efficient compression, reducing the complexity of the view random access is one of the most important requirements that should be considered in multiview video coding. In order to obtain an efficient compression, both temporal and inter-view correlations are exploited in the multiview video coding schemes, introducing higher complexity in the temporal and view random access. We propose an inter-view prediction structure that aims to lower the cost of randomly accessing any picture at any position and instant, with respect to the multiview reference model JMVM and other recent relevant works. The proposed scheme is mainly based on the use of two base views (I-views) in the structure with selected positions instead of a single reference view as in the standard structures. This will, therefore, provide a direct inter-view prediction for all the remaining views and will ensure a low-delay view random access ability while maintaining a very competitive bit-rate performance with a similar video quality measured in peak signal-to-noise ratio. In addition to a new evaluation method of the random access ability, the obtained results show a significant improvement in the view random accessibility with respect to other reported works.

Egocentric Content

AbstractThe paper distinguishes two approaches to understanding the representational content of sentences and intentional states, and its role in describing people, predicting and explaining their behavior, and so forth. It sets forth the case for one of these approaches, the “egocentric” one, initially on the basis of its ability to explain the near‐indefeasibility of ascriptions of content to our own terms (“‘dogs’ as I use it means dogs”), but more generally on the basis of its providing an attractive overall picture of the descriptive and explanatory role of representational content. In doing this, the paper relates the egocentric view to an “immanent” or “deflationary” view of reference and truth conditions, and also to the view of reference‐talk and truth‐talk as anaphoric devices. It discusses the indeterminacy of content ascriptions to those in communities with radically different theories, a phenomenon that is unsurprising on the egocentric approach, and connects this to the thesis of the normativity of meaning. (It does all this in rather broad brush: many strands of the egocentric account will be familiar, and are the subject of familiar controversies; the point of the paper is less to address these controversies individually than to tie the strands together into what is hoped to be an appealing package.)

New Antimicrobial Approaches: Reuse of Old Drugs.

The global situation of antibiotic resistance and the reduction of investments in antibiotics research by the pharmaceutical industry suggest the need for specific cost-effective approaches in order to identify drugs for the therapy of many microbial infections. Among the viable alternative anti-infective compounds, drug repurposing, i.e. to find new uses for previously approved medicines, revealed some encouraging in vitro and in vivo results. In this article the reader has a panoramic view of the updated references on the strategies encountered during the repositioning process. New findings are reported about the anti-microbial efficacy of antipsychotic, cardiovascular, anti-inflammatory and anti-neoplastic drugs. This approach may enhance the portfolio of pharmaceutical companies reducing the need for pharmacokinetic and toxicity studies; the development of new uses of old drugs for different infectious diseases, leading to better health for patients, also in poor, tropical countries, appears to be having better results.