Weight Map Research Articles

Shared imaging biomarkers across Alzheimer’s and Parkinson’s disease

AbstractBackgroundAlzheimer's disease (AD) and Parkinson's disease (PD) are the two of the most common neurodegenerative diseases [1]. In previous studies, AD and PD demonstrate overlap in clinical syndromes, including presence of cognitive impairment, dementia in PD, and post‐mortem studies with regional atrophy and amyloid and tau deposition [2][3]. Here, we propose a novel morphometric‐feature based machine learning model to explore shared degenerative patterns in AD and PD on neuroimaging.MethodWe used T1‐weighted (T1w) MRI data from (PPMI) 166 controls/356 de novo PD patients, and (ADNI‐1) 242 controls/199 AD patients. As a baseline model we trained sparse spatially regularized (TV‐L1) logistic regression [6][7] using subcortical shape and cortical thickness [4][5] from T1w MRI. To train the two models jointly, we constrain the optimization with a cross‐disorder (CD) agreement term: the L2 model difference, weighted by the models’ local normalized mutual information (MI). The CD‐MI term allows the models to guide each other’s optimization, dynamically amplifying information sharing where degenerative patterns are most similar and minimizing sharing where they are dissimilar. We evaluate all models’ ROC area‐under‐the‐curve (ROC AUC) using nested cross‐validation and a shuffle‐split bootstrap test to assess model stability. For subcortical regions, we trained models using all regions, as well as individual models for each region (L/R thalamus, putamen, caudate, pallidum, hippocampus, amygdala, accumbens).ResultInformation sharing significantly improved predictive power for cortical thickness‐based, L amygdala, L caudate, and bilateral putamen models (Table 1, Figure 4). Weight maps of AD and PD models (Figures 1a,b,2) show areas of similar degenerative patterns in frontal and parietal association areas, thalamus caudate and putamen, confirmed by the mutual information maps (Figures 1c,3).ConclusionWe designed a novel machine learning model to explore joint imaging biomarkers between Alzheimer's and Parkinson’s disease. Our findings on neuroimaging suggest shared topography of frontal, parietal, temporal, and cingulate changes for AD and PD. This may signify overlap of underlying pathologies with co‐existent amyloid and tau burden in both AD and PD.

Two‐scale fusion method of infrared and visible images via parallel saliency features

An efficient fusion method of infrared and visible images is proposed based on the parallel saliency features. The method integrates significant feature information from the source images of multiple imaging modalities into a single fused image in a multi-scale domain, while suppressing visual artefacts and retaining more detail and texture information. First, the input images are decomposed into two-scale image representations, namely the base and detail layers, using a Gaussian filter. Second, the parallel saliency features of the high contrast and detail textures are captured to acquire the saliency maps. The contrast saliency weight map of the base layers based on the weighted local intensity energy aims to highlight the salient targets in infrared images and preserve the high-intensity regions in visible images, while the detail saliency weight map of the detail layers using the structure tensor to extract the detail texture information. Finally, the final image is reconstructed by the fused base and detail layers. Sufficient experimental results convincingly demonstrate that the presented method can achieve a comparable or superior performance compared with several state-of-the-art fusion methods via subjective assessments and objective evaluations, and it is more suitable for the practical applications due to the high computing efficiency.

A Novel Fast Single Image Dehazing Algorithm Based on Artificial Multiexposure Image Fusion

Poor weather conditions, such as fog, haze, and mist, cause visibility degradation in captured images. Existing imaging devices lack the ability to effectively and efficiently mitigate the visibility degradation caused by poor weather conditions in real time. Image depth information is used to eliminate hazy effects by using existing physical model-based approaches. However, the imprecise depth information always affects dehazing performance. This article proposes an image fusion-based algorithm to enhance the performance and robustness of image dehazing. Based on a set of gamma-corrected underexposed images, pixelwise weight maps are constructed by analyzing both global and local exposedness to guide the fusion process. The spatial-dependence of luminance of the fused image is reduced, and its color saturation is balanced in the dehazing process. The performance of the proposed solution is confirmed in both theoretical analysis and comparative experiments.

SAR Target Recognition Based on Efficient Fully Convolutional Attention Block CNN

Attention mechanisms have recently shown strong potential in improving the performance of convolutional neural networks (CNNs). This letter proposes a fully convolutional attention block (FCAB) that can be combined with a CNN to refine important features and suppress unnecessary ones in synthetic aperture radar (SAR) images. The FCAB consists of a channel attention module and a spatial attention module. For the channel attention module, we use average-pooling and max-pooling to learn complementary features, and apply group convolution to aggregate the information of the two types of channels. Global average-pooling is then used to encode the channel-wise importance. For the spatial attention module, the average-pooling and max-pooling along the channel axis are used to generate two spatial feature maps, and then two very lightweight convolutional layers are used to encode the spatial weight map. Experimental results on SAR images demonstrate that our FCAB can focus on important channels and object regions. It uses relatively few parameters and is computationally efficient, while bringing about significant performance gain for SAR recognition.

Structural Similarity Loss for Learning to Fuse Multi-Focus Images.

Convolutional neural networks have recently been used for multi-focus image fusion. However, some existing methods have resorted to adding Gaussian blur to focused images, to simulate defocus, thereby generating data (with ground-truth) for supervised learning. Moreover, they classify pixels as ‘focused’ or ‘defocused’, and use the classified results to construct the fusion weight maps. This then necessitates a series of post-processing steps. In this paper, we present an end-to-end learning approach for directly predicting the fully focused output image from multi-focus input image pairs. The suggested approach uses a CNN architecture trained to perform fusion, without the need for ground truth fused images. The CNN exploits the image structural similarity (SSIM) to calculate the loss, a metric that is widely accepted for fused image quality evaluation. What is more, we also use the standard deviation of a local window of the image to automatically estimate the importance of the source images in the final fused image when designing the loss function. Our network can accept images of variable sizes and hence, we are able to utilize real benchmark datasets, instead of simulated ones, to train our network. The model is a feed-forward, fully convolutional neural network that can process images of variable sizes during test time. Extensive evaluation on benchmark datasets show that our method outperforms, or is comparable with, existing state-of-the-art techniques on both objective and subjective benchmarks.

Fault diagnosis based on deep learning for current-carrying ring of catenary system in sustainable railway transportation

In the intelligent traffic transportation, the security and stability are vital for the sustainable transportation and efficient logistics. The fault diagnosis on the catenary system is crucial for the railway transportation. For purpose of improving the detection capability for the faulted current-carrying ring and maintaining the efficiency of the railway system, a fault diagnosis method for the current-carrying ring based on an improved RetinaNet model with the spatial attention map and channel weight map is proposed. The local and global features are utilized respectively. The spatial attention maps are embedded into the original convolutional neural network to emphasize the interested local features and weaken the influence of other objects and background. The channel weight maps are introduced into the feature pyramid network of detection network to weight the multi-scale feature maps in channels. The representative global features are focused and unnecessary features are suppressed. The experimental results indicate that the proposed method has increased fault diagnosis accuracy for faulted current-carrying rings compared with the original detection network based on different backbones. It can be used in improving efficiency and safety of railway transport system.

Fourier Domain Anomaly Detection and Spectral Fusion for Stripe Noise Removal of TIR Imagery

Stripe noise is a common and unwelcome noise pattern in various thermal infrared (TIR) image data including conventional TIR images and remote sensing TIR spectral images. Most existing stripe noise removal (destriping) methods are often difficult to keep a good and robust efficacy in dealing with the real-life complex noise cases. In this paper, based on the intrinsic spectral properties of TIR images and stripe noise, we propose a novel two-stage transform domain destriping method called Fourier domain anomaly detection and spectral fusion (ADSF). Considering the principal frequencies polluted by stripe noise as outliers in the statistical spectrum of TIR images, our naive idea is first to detect the potential anomalies and then correct them effectively in the Fourier domain to reconstruct a desired destriping result. More specifically, anomaly detection for stripe frequencies is achieved through a regional comparison between the original spectrum and the expected spectrum that statistically follows a generalized Laplacian regression model, and then an anomaly weight map is generated accordingly. In the correction stage, we propose a guidance-image-based spectrum fusion strategy, which integrates the original spectrum and the spectrum of a guidance image via the anomaly weight map. The final reconstruction result not only has no stripe noise but also maintains image structures and details well. Extensive real experiments are performed on conventional TIR images and remote sensing spectral images, respectively. The qualitative and quantitative assessment results demonstrate the superior effectiveness and strong robustness of the proposed method.

Hard Pixel Mining for Depth Privileged Semantic Segmentation

Semantic segmentation has achieved remarkable progress but remains challenging due to the complex scene, object occlusion, and so on. Some research works have attempted to use extra information such as a depth map to help RGB based semantic segmentation because the depth map could provide complementary geometric cues. However, due to the inaccessibility of depth sensors, depth information is usually unavailable for the test images. In this paper, we leverage only the depth of training images as the privileged information to mine the hard pixels in semantic segmentation, in which depth information is only available for training images but not available for test images. Specifically, we propose a novel Loss Weight Module, which outputs a loss weight map by employing two depth-related measurements of hard pixels: Depth Prediction Error and Depth-aware Segmentation Error. The loss weight map is then applied to segmentation loss, with the goal of learning a more robust model by paying more attention to the hard pixels. Besides, we also explore a curriculum learning strategy based on the loss weight map. Meanwhile, to fully mine the hard pixels on different scales, we apply our loss weight module to multi-scale side outputs. Our hard pixels mining method achieves the state-of-the-art results on three benchmark datasets, and even outperforms the methods which need depth input during testing.

WITHDRAWN: Low light image contrast enhancement using advanced perfusion technique

Abstract The images captured tend to degrade due to the scattering and absorption of light. This may be due to the diminished colors, low light and undistinguishable objects in the image. For the improvement of quality of such degraded images, Advanced Perfusion technique is employed to enhance the contrast of the low light Image. In this technique, the input image in RGB color space is converted to LAB color spaces. Two versions of input image are considered, in which, the first image is processed for brightness contrast enhancement and the second image is processed for color contrast enhancement. For the enhancement of color contrast, the second image is passed through a Recursive Separated and Weighted Histogram Equalization, (RSWHE) followed by power law transformation. Two main features are extracted from both the images, namely, Laplacian weight map and Saliency weight map. These two features of the two LAB images are fused together to produce a final enhanced image, which has improved brightness contrast and color contrast. The proposed image enhancement technique is evaluated with various performance parameters such as Structural Similarity Index (SSIM), Peak Signal-to-Noise Ratio (PSNR), and Lightness Order Error (LOE). The performance of the proposed method is then compared with the existing methods for validation.

On the Hilbert eigenvariety at exotic and CM classical weight 1 points

Let F be a totally real number field and let f be a classical cuspidal p-regular Hilbert modular eigenform over F of parallel weight 1. Let x be the point on the p-adic Hilbert eigenvariety $${\mathcal {E}}$$ corresponding to an ordinary p-stabilization of f. We show that if the p-adic Schanuel conjecture is true, then $${\mathcal {E}}$$ is smooth at x if f has CM. If we additionally assume that $$F/\mathbb {Q}$$ is Galois, we show that the weight map is etale at x if f has either CM or exotic projective image (which is the case for almost all cuspidal Hilbert modular eigenforms of parallel weight 1). We prove these results by showing that the completed local ring of the eigenvariety at x is isomorphic to a universal nearly ordinary Galois deformation ring.

Fusion of visible and infrared images via saliency detection using two-scale image decomposition

As it isn’t sufficient to inspect the scene in several applications to think about just the noticeable articles, route and object identification require distinctive imaging modalities. In this paper, we propose another picture combination technique dependent on saliency discovery and two-scale picture disintegration. This technique is gainful on the grounds that saliency-based strategies have been broadly utilized in the combination of infrared (IR) and visible (VIS) images, which can feature the notable article locale and save the point by point foundation data at the same time. Another weight map development process dependent on visual saliency is proposed. Thus, it is quick, proficient and skilful. Our strategy is evaluated on a few datasets and is assessed subjectively by visual examination and quantitatively utilizing metrics. Results that are achieved by Matlab-2019A version through the comparison of entropy, standard deviation, PSNR and SSIM values of VIS and IR image datasets of several fusion methodologies uncover that the proposed technique execution is practically identical or better than the current strategies.

Fine-Grained Motion Estimation for Video Frame Interpolation

Recent advances in video frame interpolation have shown that convolutional neural networks combined with optical flow are capable of producing a high-quality intermediate frame between two consecutive input frames in most scenes. However, existing methods have difficulties dealing with large and non-uniform motions that widely exist in real-world scenes because they often adopt the same strategy to deal with different motions, which easily results in unsatisfactory results. In this article, we propose a novel fine-grained motion estimation approach (FGME) for video frame interpolation. It mainly contains two strategies: multi-scale coarse-to-fine optimization and multiple motion features estimation. The first strategy is to gradually refine optical flows and weight maps, both of which are used to synthesize the target frame. The second strategy aims to provide fine-grained motion features by generating multiple optical flows and weight maps. To demonstrate its effectiveness, we propose a fully convolutional neural network with three refinement scales and four motion features. Surprisingly, this simple network produces state-of-the-art results on three standard benchmark datasets and real-world examples, with advantages in terms of effectiveness, simplicity, and network size over other existing approaches. Furthermore, we demonstrate that the FGME approach has good generality and can significantly improve the synthesis quality of other methods.

Robust multi-focus image fusion using lazy random walks with multiscale focus measures

We present a robust multi-focus image fusion (MFIF) method based on lazy random walks (LRW) and multiscale focus measures in this paper. Firstly, the multiscale sum of the weighted modified Laplacian (MSWML) is used to extract the focus information of source images at different scales. Then, we compute the scale-consistent focus maps by collaboratively employing the local focus information as well as the global focus information. After that, the scale-consistent focus maps are fed into LRW-based estimation framework to produce the fusion weight maps. Finally, a weighted-sum strategy is used to generate the fused image. Experimental results show that our method consistently produces high quality fused images with few artifacts and achieves better performance than some state-of-the-art methods.

A novel multi-scale fusion framework for detail-preserving low-light image enhancement

In this paper, we propose a novel multi-scale fusion framework for low-illumination image enhancement, which effectively enhances images taken under various low-light conditions. Based on the high dynamic range imaging technique, we first employ a novel remapping function to generate a sequence of artificial multi-exposure images. The generated sequence of images ensures that the contrast of each intensity interval of the input image is enhanced at least once. Then three fusion-relevant features, namely, exposure, global contrast and local contrast, are selected as the weight maps. Combined with the weight maps, a pyramid fusion scheme is introduced to do a layer-by-layer integration of the different frequency bands of the image layer by layer. In addition, a strategy for extracting details from the original image is designed, which effectively maintains the detail information without causing colour distortions. The framework is very efficient and suitable for mobile devices because most of the calculations are at the pixel-level. Extensive experiments have shown that the proposed approach yields comparable and better performances in comparisons with the state-of-the-art competing techniques in both qualitative and quantitative evaluations.

Multi‐exposure image fusion via a pyramidal integration of the phase congruency of input images with the intensity‐based maps

The most important part of the common algorithms for multi-exposure image fusion (MEF) is the selection of features and metrics that are appropriate for weight map extraction. This study presents a structure-based multi-exposure image fusion by employing the phase congruency (PC) of the input image. The main idea behind PC-based analysis is that the locations of image key attributes are at points where frequency components are maximally in phase. PC detects the details of an image invariant to its contrast and also emphasises on the texture- or structure-based features. In this work, alongside intensity-based maps, the extracted PC-based map is utilised for MEF in a pyramidal manner. Several experiments conducted on the benchmark dataset including a variety of natural multi-exposed image sequences to evaluate the proposed algorithm. Quantitative evaluations in terms of MEF structural similarity index and visual quality assessments show that the proposed method achieves better performance and produces comparable fused images in comparison to other approaches.

Fast color balance and multi-path fusion for sandstorm image enhancement

Outdoor images captured during sandstorm weather condition frequently yield color cast and poor visibility, which causes some applications to fail in computer vision, such as video surveillance and object recognition systems. In this paper, a fast color balance method followed by an effective fusion model is proposed to enhance the sandstorm-degraded images. Firstly, color channel compensation and piece-wise affine transform balance the aberrant pixels obtained by camera and remove color cast. Then, multi-path fusion comprising of underexposure and contrast-enhanced inputs is used for visibility enhancement, where saturation and Laplacian contrast are measured as weight maps and constructed for Gaussian pyramids. In order to reduce block effects and artifacts of the reconstructed image, we introduce the multi-scale strategy for each path. Experimental results of both synthetic and real-world sandstorm images demonstrate that the proposed color balance method features high computational efficiency and performs much better than comparative methods in terms of image quality. In addition, our fusion-based method also outperforms existing enhancement algorithms of sandstorm image via qualitative and quantitative evaluations.

Adaptive QP offset selection algorithm for virtual reality 360-degree video based on CTU complexity

Virtual reality 360-degree video requires ultra-high resolution to provide realistic feeling and dynamic perspective. Huge data volume brings new challenges to coding and transmission. Quantization parameter (QP) is one of the key parameters to control output bitrate and reconstruction quality during coding process. Many QP offset selection algorithms designed for this kind of video are based on latitude or Equirectangular Projection (ERP) weight maps, which cannot adapt to the situation of the flat block in tropical area or the complex block in polar area. In this paper, a new metric to measure complexity of Coding Tree Unit (CTU) is designed, and an adaptive QP offset selection algorithm is proposed based on CTU complexity to improve the quantization process. Each CTU is classified into one of the five categories according to its complexity, and then different QP offset value is determined for each category. By improving the quality of the visually sensitive area and reducing the bitrate of the flat one, the efficiency of the encoder is improved. The experimental results show that, compared with the HM16.20, the WS-PSNR increases by 0.40 dB, the BD-rate reduces by 1.99%, and the quality of visually sensitive areas has improved significantly.

Navigating Non-Playable Characters Based on User Trajectories with Accumulation Map and Path Similarity

In this paper, we propose a method to efficiently control the path of non-playable characters (NPC) in an interactive virtual environment such as a game or virtual reality (VR) by calculating a weight map and path similarity based on the user’s path. Our method automatically constructs a navigation mesh that provides a new route to the NPC by referring to the user’s trajectory. Our method finds more paths that users usually go through as time passes, and the number of users increases. Accordingly, the paths that NPCs can traverse automatically are updated adaptively to the virtual environment. In addition, NPC agents can move smartly by assigning high weights to the user’s preferred paths. We tested the usefulness of the proposed method through several example scenarios in an interactive environment such as a video game or VR, and this method can be easily applied to various types of navigation based on the interactive environment.

An improved algorithm of multi-exposure image fusion by detail enhancement

Multi-exposure image fusion is an effective method for depicting high dynamic range of the target scene in a single image. However, there are still some problems remaining: the preserving of global contrast, the preserving of the local details in saturated regions, and the existence of halo artifacts. To solve these problems, this paper proposes a new multi-exposure image fusion algorithm with detail enhancement. Firstly, the well-exposedness evaluation function, the chromatic information evaluation function and the local detail preserved function are used to measure weight maps. Then, an improved multi-exposure fusion framework based on pyramid decomposition is proposed to further enhance the details. The experimental results demonstrate that the proposed algorithm can preserve more details than the state-of-the-art. In the view of appreciation, our approach could produce a more realistic brightness distribution of target scene as well as avoid halo artifacts.

Multi-exposure image fusion based on linear embeddings and watershed masking

High dynamic range imaging (HDRI) is a challenging technology but yet demanding for modern imaging applications. Low-cost image sensors have limited dynamic range, and it is not always possible to capture and display natural scenes with high contrast and information loss in any exposure is inevitable. Three solutions for HDRI are using expensive high dynamic range (HDR) cameras with HDR-compatible displays, tone mapping operators for low dynamic range (LDR) screens, and capturing and fusing multiple exposures of the same LDR scene via image fusion algorithms. Companies that produce user grade devices prefer multi-exposure fusion (MEF) approaches to obtain HDR-like images for LDR screens due to its low cost. Hence, merging a stack of images containing different exposures of the same scene into a single informative image is an attractive research field. In this study, a novel, simple yet effective method is proposed for static image exposure fusion. The developed technique is based on weight map extraction via linear embeddings and watershed masking. The main advantage lies in watershed masking-based adjustment for obtaining accurate weights for image fusion. The comprehensive experimental comparisons demonstrate very strong visual and statistical results, and this approach should facilitate future MEF studies.