Scene Distortion Research Articles

A three-dimensional feature-based fusion strategy for infrared and visible image fusion

Due to the lacking of attention to the scene’s essential characteristics, the existing fusion methods suffer from the deficiency of scene distortion. In addition, the lack of groundtruth can cause an inadequate representation of vital information. To this end, we propose a novel infrared and visible image fusion network based on three-dimensional feature fusion strategy (D3Fuse). In our method, we consider the scene semantic information in the source images and extract the commonality contents of the two images as the third-dimensional feature to extend the feature space for fusion tasks. Specifically, a commonality feature extraction module (CFEM) is designed to extract the scene commonality features. Subsequently, the scene commonality features are utilized together with modality features to construct the fusion image. Moreover, to ensure the independence and diversity of distinct features, we employ a contrastive learning strategy with multiscale PCA coding, which stretches the feature distance in an unsupervised manner, prompting the encoder to extract more discriminative information without incurring additional parameters and computational costs. Furthermore, a contrastive enhancement strategy is utilized to ensure adequate representation of modality information. The results of the qualitative and quantitative evaluations on the three datasets show that the proposed method has better visual performance and higher objective metrics with lower computational cost. The object detection experiments show that our results have superior performance on high-level semantic tasks.

Stereoscopic Image Enabled Animation Art Design Exploration

Abstract With the emergence of movie art, many related art forms were born. Animation art has gained a large number of audiences due to its exaggerated expression. In order to improve the problems of character image overlapping and scene distortion that may occur in the original two-dimensional space, this paper constructs a semi-global stereo image matching algorithm based on the use of point cloud matching in the animation screen for the determination of the screen position and the morphology of the animated character. To compensate for the radiation difference in the stereo image matching algorithm, object-square geometric constraints are proposed to improve the matching cost calculation process. After calculating the matching cost, the original animation model is optimized or rebuilt by creating a depth space image. Taking Disney animation characters as an example, the number of joints required for the reconstruction of the “Snow White” character is 2.9 times more than that of the “Mickey” character, but the reconstruction time of the skeleton is only 12 times longer than that of the “Mickey” character. However, the skeleton reconstruction time is only 12.1 seconds longer than that of the “Mickey” character, which indicates that the algorithm proposed in this paper does not produce a large difference in reconstruction time due to the difference in characters. For the construction of animated indoor scenes, in the eight experimental scenes, the maximum number of grids can be divided into 42365412, and the processing time required is 276.413 seconds. It shows that the algorithm in this paper is capable of subregionally refining the indoor space of the animation. The proposed semi-global stereo image matching-based pan can quickly reconstruct animated character models and animated scenes, as can be seen.

Client-Oriented Blind Quality Metric for High Dynamic Range Stereoscopic Omnidirectional Vision Systems

A high dynamic range (HDR) stereoscopic omnidirectional vision system can provide users with more realistic binocular and immersive perception, where the HDR stereoscopic omnidirectional image (HSOI) suffers distortions during its encoding and visualization, making its quality evaluation more challenging. To solve the problem, this paper proposes a client-oriented blind HSOI quality metric based on visual perception. The proposed metric mainly consists of a monocular perception module (MPM) and binocular perception module (BPM), which combine monocular/binocular, omnidirectional and HDR/tone-mapping perception. The MPM extracts features from three aspects: global color distortion, symmetric/asymmetric distortion and scene distortion. In the BPM, the binocular fusion map and binocular difference map are generated by joint image filtering. Then, brightness segmentation is performed on the binocular fusion image, and distinctive features are extracted on the segmented high/low/middle brightness regions. For the binocular difference map, natural scene statistical features are extracted by multi-coefficient derivative maps. Finally, feature screening is used to remove the redundancy between the extracted features. Experimental results on the HSOID database show that the proposed metric is generally better than the representative quality metric, and is more consistent with the subjective perception.

Deep Feature Based Siamese Network for Visual Object Tracking

One of the most important and challenging research subjects in computer vision is visual object tracking. The information obtained from the first frame consists of limited and insufficient information to represent an object. If prior information about robust representation that can represent an object well is not sufficient, object tracking fails when not robustly responding to changes in features of the target object according to various factors, namely shape, illumination variation, and scene distortion. In this paper, a real-time single object tracking algorithm is proposed based on a Siamese network to solve this problem. For the object feature extraction, we designed a fully convolutional neural network that removes a fully connected layer and configured a convolution block consisting of a bottleneck structure that preserves the information in a previous layer. This network was designed as a Siamese network, while a regional proposal network was combined at the end of the network for object tracking. The ImageNet Large-Scale Visual Recognition Challenge 2017 dataset was used to train the network in the pre-training phase. Then, in the experimental phase, the object tracking benchmark dataset was used to quantitatively evaluate the network. The experimental results revealed that the proposed tracking algorithm produced more competitive results compared to other tracking algorithms.

Scene distortion detection algorithm using multitemporal remote sensing images

Multitemporal remote sensing images of a particular territory might include accidental scene distortions. Scene distortion is a significant local brightness change caused by the scene overlap with some opaque object or a natural phenomenon coincident with the moment of image capture, for example, clouds and shadows. The fact that different images of the scene are obtained at different instants of time makes the appearance, location and shape of scene distortions accidental. In this article we propose an algorithm for detecting accidental scene distortions using a dataset of multitemporal remote sensing images. The algorithm applies superpixel segmentation and anomaly detection methods to get binary images of scene distortion location for each image in the dataset. The algorithm is adapted to handle images with different spectral and spatial sampling parameters, which makes it more multipurpose than the existing solutions. The algorithm's quality was assessed using model images with scene distortions for two remote sensing systems. The experiments showed that the proposed algorithm with the optimal settings can reach a detection accuracy of about 90% and a false detection error of about 10%.

A Three-Stage Progressive Transmission Scheme for Virtual Environments Over Lossy Networks

The rapid development of Virtual Environments (VEs) gives rise to its prominent performance in a myriad of multimedia applications. Accompanied by this is the increasingly massive data used for ascending user's visual experience. Recently, 3D streaming makes it viable to put VEs into real-time use through progressive transmission. Nonetheless, downloading VEs over inevitable lossy networks still remains a major bottleneck that urgently needs to be solved. In this paper, we propose a three-stage progressive transmission scheme for VEs over lossy networks, which aims to minimize distortion even when encountering poor network condition. The first stage is the Pre-coding Stage that offers the most feasible transmission scheme for subsequent transmission, in this stage we combine the advanced interest management algorithm and our proposed scene distortion estimation algorithm to bring up a scene update strategy which takes the user's visual features, quality of the scene and bandwidth estimation into account, so as to determine the download priority comprehensively. In the following Transmitting Stage, we apply Transmission Control Protocol (TCP) to make sure that we convey the essential information. More importantly, we adopt expanding window fountain (EWF) codes to provide an unequal error protection (UEP) for data with different importance. Eventually in the third After-process Stage, we employ a geometry prediction algorithm for the scenario with packet loss. The final results of our experiments show that the three-stage scheme can offer preeminent visual experience especially when it comes to a relatively inferior network condition.

Full Reference Quality Assessment for Image Retargeting Based on Natural Scene Statistics Modeling and Bi-Directional Saliency Similarity

Image retargeting technology has been widely studied to adapt images for the devices with heterogeneous screen resolutions. Meanwhile effective objective retargeting quality assessment algorithms are also very important for optimizing and selecting favorable retargeting methods. Unlike previous assessment algorithms which rely on image local structure features and unidirectional prediction of information loss, we propose a bi-directional natural salient scene distortion model (BNSSD) including image natural scene statistics (NSS) measurement, salient global structure distortion measurement, and bi-directional salient information loss measurement. First, we propose a new NSS model in log-Gabor domain and verify its effectiveness in reflecting nature scene statistical distortions introduced during the retargeting process. Second, the concept of salient global structure distortion is proposed to measure the global structure uniformity in the corresponding salient regions between original and retargeted images. Finally, we propose a bidirectional salient information loss metric to measure the information loss between salient areas in original image and retargeted image. The effectiveness of the BNSSD model is verified on two widely recognized public databases, and the experimental results show that our method outperforms the state-of-the-art algorithms under different statistical assessment criteria.

Detection of chromatic and luminance distortions in natural scenes.

A number of studies have measured visual thresholds for detecting spatial distortions applied to images of natural scenes. In one study, Bex [J. Vis.10(2), 1 (2010)10.1167/10.2.231534-7362] measured sensitivity to sinusoidal spatial modulations of image scale. Here, we measure sensitivity to sinusoidal scale distortions applied to the chromatic, luminance, or both layers of natural scene images. We first established that sensitivity does not depend on whether the undistorted comparison image was of the same or of a different scene. Next, we found that, when the luminance but not chromatic layer was distorted, performance was the same regardless of whether the chromatic layer was present, absent, or phase-scrambled; in other words, the chromatic layer, in whatever form, did not affect sensitivity to the luminance layer distortion. However, when the chromatic layer was distorted, sensitivity was higher when the luminance layer was intact compared to when absent or phase-scrambled. These detection threshold results complement the appearance of periodic distortions of the image scale: when the luminance layer is distorted visibly, the scene appears distorted, but when the chromatic layer is distorted visibly, there is little apparent scene distortion. We conclude that (a)observers have a built-in sense of how a normal image of a natural scene should appear, and (b)the detection of distortion in, as well as the apparent distortion of, natural scene images is mediated predominantly by the luminance layer and not chromatic layer.

Optimized Packet Scheduling in Multiview Video Navigation Systems

We study coding and transmission strategies in multicamera systems, where correlated sources send data through a bottleneck channel to a central server, which eventually transmits views to different interactive users. We propose a dynamic navigation -path aware packet scheduling optimization under delay, bandwidth, and interactivity constraints aimed at optimizing the quality-of-experience of interactive users. In particular , the scene distortion is minimized jointly with the distortion variations along most likely navigation paths. The optimization relies both on a novel rate-distortion model, which captures the importance of each view in the scene reconstruction , and on an objective function that optimizes resources based on a client navigation model. The latter takes into account the distortion experienced by interactive clients as well as the distortion variations that might be observed by clients during multiview navigation. We solve the scheduling problem with a novel trellis-based solution, which permits to formally decompose the multivariate optimization problem, thereby significantly reducing the computation complexity. Simulation results show the PSNR quality gain offered by the proposed algorithm compared to baseline scheduling policies. Finally, we show that the best scheduling policy consistently adapts to the most likely user navigation path and that it minimizes distortion variations that can be very disturbing for users in traditional navigation systems.

Perceptual Issues in Stereoscopic Signal Processing

Perceiving three-dimensional video imagery appropriately in a display requires matching parameters throughout the imaging pathway, such as inter-aperture distance at the stereoscopic camera side with parallax shifting at the display side. In addition, many tradeoffs and compromises are often made at different points in the imaging pathway, leading to common perceptual distortions. Some of these may be simple two-dimensional image distortions such as display surface noise, while others are three-dimensional distortions, such as global geometric scene distortions and localized depth errors around edges. There is an increasing use of various forms of signal processing to modify the images, either for compensation of distortions due to system limitations, display constraints, formatting and compression for efficient transmission, or making depth range adjustments dependent on the display viewing conditions. Perceptual issues are critical to the design of the entire imaging pathway and this paper will highlight some of those due to stereoscopic signal processing.

Joint estimation of motion and radiometry of rotating camera for HDR global mosaic

In this paper, we present a global approach for constructing high dynamic range mosaics from multiple images with large exposure differences. To minimize registration errors caused by intensity mismatches in the image intensity space with low dynamic range, we propose the use of a scene radiance space with high dynamic range. By relating image intensities to scene radiances with a convenient distortion model, we robustly estimate registration parameters for the high dynamic range global mosaic, simultaneously estimating scene radiances and distortion parameters in a single framework using a computationally optimized Levenberg–Marquardt approach.

Illusory scene distortion occurs during perceived self-rotation in roll

We report a novel illusory distortion of the visual scene, which became apparent during both: (i) observer rotation inside a furnished stationary room; and (ii) room rotation about the stationary observer. While this distortion had several manifestations, the most common experience was that scenery near fixation appeared to sometimes lead and other times lag more peripheral scenery. Across a series of experiments, we eliminated explanations based on eye-movements, distance misperception, peripheral aliasing, differential motion sensitivity and adaptation. We found that these illusory scene distortions occurred only when the observer perceived (real or illusory) changes in self-tilt and maintained a stable fixation.