Pixel-wise Weights Research Articles

A Novel Fast Single Image Dehazing Algorithm Based on Artificial Multiexposure Image Fusion

Poor weather conditions, such as fog, haze, and mist, cause visibility degradation in captured images. Existing imaging devices lack the ability to effectively and efficiently mitigate the visibility degradation caused by poor weather conditions in real time. Image depth information is used to eliminate hazy effects by using existing physical model-based approaches. However, the imprecise depth information always affects dehazing performance. This article proposes an image fusion-based algorithm to enhance the performance and robustness of image dehazing. Based on a set of gamma-corrected underexposed images, pixelwise weight maps are constructed by analyzing both global and local exposedness to guide the fusion process. The spatial-dependence of luminance of the fused image is reduced, and its color saturation is balanced in the dehazing process. The performance of the proposed solution is confirmed in both theoretical analysis and comparative experiments.

Pixel-Aware Deep Function-Mixture Network for Spectral Super-Resolution

Spectral super-resolution (SSR) aims at generating a hyperspectral image (HSI) from a given RGB image. Recently, a promising direction is to learn a complicated mapping function from the RGB image to the HSI counterpart using a deep convolutional neural network. This essentially involves mapping the RGB context within a size-specific receptive field centered at each pixel to its spectrum in the HSI. The focus thereon is to appropriately determine the receptive field size and establish the mapping function from RGB context to the corresponding spectrum. Due to their differences in category or spatial position, pixels in HSIs often require different-sized receptive fields and distinct mapping functions. However, few efforts have been invested to explicitly exploit this prior.To address this problem, we propose a pixel-aware deep function-mixture network for SSR, which is composed of a new class of modules, termed function-mixture (FM) blocks. Each FM block is equipped with some basis functions, i.e., parallel subnets of different-sized receptive fields. Besides, it incorporates an extra subnet as a mixing function to generate pixel-wise weights, and then linearly mixes the outputs of all basis functions with those generated weights. This enables us to pixel-wisely determine the receptive field size and the mapping function. Moreover, we stack several such FM blocks to further increase the flexibility of the network in learning the pixel-wise mapping. To encourage feature reuse, intermediate features generated by the FM blocks are fused in late stage, which proves to be effective for boosting the SSR performance. Experimental results on three benchmark HSI datasets demonstrate the superiority of the proposed method.

Adaptive weighted log subtraction based on neural networks for markerless tumor tracking using dual-energy fluoroscopy.

To present a novel method, based on convolutional neural networks (CNN), to automate weighted log subtraction (WLS) for dual-energy (DE) fluoroscopy to be used in conjunction with markerless tumor tracking (MTT). A CNN was developed to automate WLS (aWLS) of DE fluoroscopy to enhance soft tissue visibility. Briefly, this algorithm consists of two phases: training a CNN architecture to predict pixel-wise weighting factors followed by application of WLS subtraction to reduce anatomical noise. To train the CNN, a custom phantom was built consisting of aluminum (Al) and acrylic (PMMA) step wedges. Per-pixel ground truth (GT) weighting factors were calculated by minimizing the contrast of Al in the step wedge phantom to train the CNN. The pretrained model was then utilized to predict pixel-wise weighting factors for use in WLS. For comparison, the weighting factor was manually determined in each projection (mWLS). A thorax phantom with five simulated spherical targets (5-25mm) embedded in a lung cavity, was utilized to assess aWLS performance. The phantom was imaged with fast-kV dual-energy (120 and 60kVp) fluoroscopy using the on-board imager of a commercial linear accelerator. DE images were processed offline to produce soft tissue images using both WLS methods. MTT was compared using soft tissue images produced with both mWLS and aWLS techniques. Qualitative evaluation demonstrated that both methods achieved soft tissue images with similar quality. The use of aWLS increased the number of tracked frames by 1-5% compared to mWLS, with the largest increase observed for the smallest simulated tumors. The tracking errors for both methods produced agreement to within 0.1 mm. A novel method to perform automated WLS for DE fluoroscopy was developed. Having similar soft tissue quality as well as bone suppression capability as mWLS, this method allows for real-time processing of DE images for MTT.

Occlusion-aware Region-based 3D Pose Tracking of Objects with Temporally Consistent Polar-based Local Partitioning.

Region-based methods have become the state-of-art solution for monocular 6-DOF object pose tracking in recent years. However, two main challenges still remain: the robustness to heterogeneous configurations (both foreground and background), and the robustness to partial occlusions. In this paper, we propose a novel region-based monocular 3D object pose tracking method to tackle these problems. Firstly, we design a new strategy to define local regions, which is simple yet efficient in constructing discriminative local color histograms. Contrary to previous methods which define multiple circular regions around the object contour, we propose to define multiple overlapped, fan-shaped regions according to polar coordinates. This local region partitioning strategy produces much less number of local regions that need to be maintained and updated, while still being temporally consistent. Secondly, we propose to detect occluded pixels using edge distance and color cues. The proposed occlusion detection strategy is seamlessly integrated into the region-based pose optimization pipeline via a pixel-wise weight function, which significantly alleviates the interferences caused by partial occlusions. We demonstrate the effectiveness of the proposed two new strategies with a careful ablation study. Furthermore, we compare the performance of our method with the most recent state-of-art region-based methods in a recently released large dataset, in which the proposed method achieves competitive results with a higher average tracking success rate. Evaluations on two real-world datasets also show that our method is capable of handling realistic tracking scenarios.

Dynamic pixel-wise weighting-based fully convolutional neural networks for left ventricle segmentation in short-axis MRI

Left ventricle (LV) segmentation in cardiac MRI is an essential procedure for quantitative diagnosis of various cardiovascular diseases. In this paper, we present a novel fully automatic left ventricle segmentation approach based on convolutional neural networks. The proposed network fully takes advantages of the hierarchical architecture and integrate the multi-scale feature together for segmenting the myocardial region of LV. Moreover, we put forward a dynamic pixel-wise weighting strategy, which can dynamically adjust the weight of each pixel according to the segmentation accuracy of upper layer and force the pixel classifier to take more attention on the misclassified ones. By this way, the LV segmentation performance of our method can be improved a lot especially for the apical and basal slices in cine MR images. The experiments on the CAP database demonstrate that our method achieves a substantial improvement compared with other well-know deep learning methods. Beside these, we discussed two major limitations in convolutional neural networks-based semantic segmentation methods for LV segmentation.

Learnable Gated Convolutional Neural Network for Semantic Segmentation in Remote-Sensing Images

Semantic segmentation in high-resolution remote-sensing (RS) images is a fundamental task for RS-based urban understanding and planning. However, various types of artificial objects in urban areas make this task quite challenging. Recently, the use of Deep Convolutional Neural Networks (DCNNs) with multiscale information fusion has demonstrated great potential in enhancing performance. Technically, however, existing fusions are usually implemented by summing or concatenating feature maps in a straightforward way. Seldom do works consider the spatial importance for global-to-local context-information aggregation. This paper proposes a Learnable-Gated CNN (L-GCNN) to address this issue. Methodologically, the Taylor expression of the information-entropy function is first parameterized to design the gate function, which is employed to generate pixelwise weights for coarse-to-fine refinement in the L-GCNN. Accordingly, a Parameterized Gate Module (PGM) was designed to achieve this goal. Then, the single PGM and its densely connected extension were embedded into different levels of the encoder in the L-GCNN to help identify the discriminative feature maps at different scales. With the above designs, the L-GCNN is finally organized as a self-cascaded end-to-end architecture that is able to sequentially aggregate context information for fine segmentation. The proposed model was evaluated on two public challenging benchmarks, the ISPRS 2Dsemantic segmentation challenge Potsdam dataset and the Massachusetts building dataset. The experiment results demonstrate that the proposed method exhibited significant improvement compared with several related segmentation networks, including the FCN, SegNet, RefineNet, PSPNet, DeepLab and GSN.For example, on the Potsdam dataset, our method achieved a 93.65% F 1 score and 88.06% I o U score for the segmentation of tiny cars in high-resolution RS images. As a conclusion, the proposed model showed potential for object segmentation from the RS images of buildings, impervious surfaces, low vegetation, trees and cars in urban settings, which largely varies in size and have confusing appearances.

SAR Image De-Noising based on GNL-Means with Optimized Pixel-Wise Weighting in Non-Subsample Shearlet Domain

SAR images have been widely used in many fields such as military and remote sensing. So the suppression of the speckle has been an important research issues. To improve the visual effect of non-local means, generalized non-local (GNL) means with optimized pixel-wise weighting is applied to shrink the coefficients of non-subsample Shearlet transform (NSST) of SAR image. The new method can optimize the weight of GNL, which not only improve the PSNR of de-noised image, but also can significantly enhance the visual effect of de-noising image.

A Computing Framework of Adaptive Support-Window Multi-Lateral Filter for Image and Depth Processing

In this paper, a computing framework for adaptive-support-window-based multi-lateral filters is proposed. The so-called multi-lateral filters, extended from the well-known bilateral filter, have found their broaden applications in noise/high-frequency suppression for 2-D image and 3-D depth processing. Our filters rely on binarizing the traditional pixel-wise weights to be only 0 or 1, resulting in an adaptive support window whose shape depends on the local image structure of the central anchor pixel. A cross-subwindow-based algorithm is devised to compute the adaptive support window. A fast algorithm based on integral images is also devised for data aggregation within such an irregularly shaped support window. Taking advantage of the integral images, our scheme presents a near constant-time complexity regardless of the size and shape of the support window. Experiments show that both noise suppression and edge-preserving can be simultaneously achieved by using our proposed framework. The average speedup ratios of our scheme are 14X~49X and 1.3X~5.6X against the traditional and the O(1) implementations, respectively. Our scheme also has the advantage of easy extension to tri-lateral and quadri-lateral filters, whereas other O(1) algorithms might not.

Region-of-Interest Based Conversational HEVC Coding with Hierarchical Perception Model of Face

In this paper, we propose a region-of-interest (ROI) based HEVC coding approach for conversational videos, with a novel hierarchical perception model of face (HP model), to improve the perceived visual quality of state-of-the-art HEVC standard. In contrast to the previous ROI-based video coding approaches, this novel HP model allows the unequal importance of facial features (e.g., the eyes and mouth) within the facial region, by generating a pixel-wise weight map. Benefitting from such a perception model, the adaptive coding tree unit (CTU) partition structure is developed to alleviate the encoding complexity of HEVC, without any degradation of the visual quality in facial regions, especially in the regions of facial features. Subsequently, for the rate control in HEVC a weight-based unified rate-quantization (URQ) scheme, instead of the conventional pixel-based URQ scheme, is proposed to adaptively adjust the value of quantization parameter (QP). Such an adaptive adjustment of QPs is capable of allocating more bits to the face/facial features with respect to our HP model, and as a result, the visual quality of face, in particular facial features, can be enhanced for conversational HEVC coding. Finally, the experimental results show that the perceived visual quality of our approach is greatly improved, with even less encoding time, for conversational video coding on the HEVC platform.