Spatial Smoothness Constraint Research Articles

Automatic Subretinal Fluid Segmentation of Retinal SD-OCT Images With Neurosensory Retinal Detachment Guided by Enface Fundus Imaging.

Accurate segmentation of neurosensory retinal detachment (NRD) associated subretinal fluid in spectral domain optical coherence tomography (SD-OCT) is vital for the assessment of central serous chorioretinopathy (CSC). A novel two-stage segmentation algorithm was proposed, guided by Enface fundus imaging. In the first stage, Enface fundus image was segmented using thickness map prior to detecting the fluid-associated abnormalities with diffuse boundaries. In the second stage, the locations of the abnormalities were used to restrict the spatial extent of the fluid region, and a fuzzy level set method with a spatial smoothness constraint was applied to subretinal fluid segmentation in the SD-OCT scans. Experimental results from 31 retinal SD-OCT volumes with CSC demonstrate that our method can achieve a true positive volume fraction (TPVF), false positive volume fraction (FPVF), and positive predicative value (PPV) of 94.3%, 0.97%, and 93.6%, respectively, for NRD regions. Our approach can also discriminate NRD-associated subretinal fluid from subretinal pigment epithelium fluid associated with pigment epithelial detachment with a TPVF, FPVF, and PPV of 93.8%, 0.40%, and 90.5%, respectively. We report a fully automatic method for the segmentation of subretinal fluid. Our method shows the potential to improve clinical therapy for CSC.

Constrained Superpixel Tracking.

In this paper, we propose a constrained graph labeling algorithm for visual tracking where nodes denote superpixels and edges encode the underlying spatial, temporal, and appearance fitness constraints. First, the spatial smoothness constraint, based on a transductive learning method, is enforced to leverage the latent manifold structure in feature space by investigating unlabeled superpixels in the current frame. Second, the appearance fitness constraint, which measures the probability of a superpixel being contained in the target region, is developed to incrementally induce a long-term appearance model. Third, the temporal smoothness constraint is proposed to construct a short-term appearance model of the target, which handles the drastic appearance change between consecutive frames. All these three constraints are incorporated in the proposed graph labeling algorithm such that induction and transduction, short- and long-term appearance models are combined, respectively. The foreground regions inferred by the proposed graph labeling method are used to guide the tracking process. Tracking results, in turn, facilitate more accurate online update by filtering out potential contaminated training samples. Both quantitative and qualitative evaluations on challenging tracking data sets show that the proposed constrained tracking algorithm performs favorably against the state-of-the-art methods.

Reducing uncertainty of dynamic heterogeneous information networks: a fusing reconstructing approach

In real world, a heterogeneous information network (HIN) is often dynamic due to the time varying features of the nodes, and uncertain due to missing values and noise. In this paper, we investigate the problem of reducing the uncertainty of a dynamic HIN, which is an important task for HIN analysis. The challenges are three-fold, the heterogeneity of features, the heterogeneity of constraints, and the dynamic uncertainty. We propose a novel approach, called fusing reconstruction (FRec), which reconstructs the uncertain snapshots of a dynamic HIN in a homogeneous feature space combining two fusions, the fusion of heterogeneous features and the fusion of heterogeneous constraints. To address the challenge of the heterogeneity of features, we propose an invertible fusing transformation (IFT) as the first part of FRec. IFT is a bidirectional transformation, which is able to learn unified latent homogeneous feature representations for heterogeneous nodes and transform them back to the raw heterogeneous feature space by its invertibility. To address the challenge of the heterogeneity of constraints and the challenge of dynamic uncertainty, we propose a heterogeneous constraints fusion based tensor reconstruction model (HCF-TRM) as the second part of FRec. HCF-TRM is able to denoise the uncertain snapshots of a dynamic HIN and recovers the missing values by fusing the spatial smoothness constraint and the temporal smoothness constraint into the tensor reconstruction. At last, the extensive experiments conducted on real datasets and synthetic datasets verify the effectiveness and scalability of FRec.

1D Current Source Density (CSD) Estimation in Inverse Theory: A Unified Framework for Higher-Order Spectral Regularization of Quadrature and Expansion-Type CSD Methods.

Estimation of current source density (CSD) from the low-frequency part of extracellular electric potential recordings is an unstable linear inverse problem. To make the estimation possible in an experimental setting where recordings are contaminated with noise, it is necessary to stabilize the inversion. Here we present a unified framework for zero- and higher-order singular-value-decomposition (SVD)-based spectral regularization of 1D (linear) CSD estimation from local field potentials. The framework is based on two general approaches commonly employed for solving inverse problems: quadrature and basis function expansion. We first show that both inverse CSD (iCSD) and kernel CSD (kCSD) fall into the category of basis function expansion methods. We then use these general categories to introduce two new estimation methods, quadrature CSD (qCSD), based on discretizing the CSD integral equation with a chosen quadrature rule, and representer CSD (rCSD), an even-determined basis function expansion method that uses the problem's data kernels (representers) as basis functions. To determine the best candidate methods to use in the analysis of experimental data, we compared the different methods on simulations under three regularization schemes (Tikhonov, tSVD, and dSVD), three regularization parameter selection methods (NCP, L-curve, and GCV), and seven different a priori spatial smoothness constraints on the CSD distribution. This resulted in a comparison of 531 estimation schemes. We evaluated the estimation schemes according to their source reconstruction accuracy by testing them using different simulated noise levels, lateral source diameters, and CSD depth profiles. We found that ranking schemes according to the average error over all tested conditions results in a reproducible ranking, where the top schemes are found to perform well in the majority of tested conditions. However, there is no single best estimation scheme that outperforms all others under all tested conditions. The unified framework we propose expands the set of available estimation methods, provides increased flexibility for 1D CSD estimation in noisy experimental conditions, and allows for a meaningful comparison between estimation schemes.

Robust geometric ℓp-norm feature pooling for image classification and action recognition

Feature pooling is a key component in modern visual classification system. However, the conventional two prevailing pooling techniques, namely average and max poolings, are not theoretically optimal, due to the unrecoverable loss of the spatial information during the statistical summarization and the underlying over-simplified assumption about the feature distribution. Addressing these issues, this paper proposes to generalize previous pooling methods toward a weighted ℓp-norm spatial pooling function tailored for class-specific feature spatial distribution. Optimizing such a pooling function toward discriminative class separability that is subject to a spatial smoothness constraint yields a so-called geometric ℓp-norm pooling (GLP) method. Furthermore, to handle the variation of object scale/position, which would affect not only the learning of discriminative pooling weights but also the applicability of the learned weights, we propose a simple yet effective self-alignment step during both learning and testing to adaptively adjust the pooling weights for individual images. Image segmentation and visual saliency map are utilized to construct a directed pixel adjacency graph. The discriminative pooling weights are diffused using random walk on the constructed graph and therefore the discriminative pooling weights are propagated onto the salient and foreground region. This leads to a robust version of GLP (RGLP) which can cope with the misalignment of object position and scale in images. Comprehensive experiments validate the effectiveness of the proposed GLP feature pooling framework. The proposed random walk based self-alignment step can effectively alleviate the image misalignment issue and further boost classification accuracy. State-of-the-art image classification and action recognition performances are attained on several benchmarks.

Video Face Editing Using Temporal-Spatial-Smooth Warping

Editing faces in videos is a popular yet challenging task in computer vision and graphics that encompasses various applications, including facial attractiveness enhancement, makeup transfer, face replacement, and expression manipulation. Directly applying the existing warping methods to video face editing has the major problem of temporal incoherence in the synthesized videos, which cannot be addressed by simply employing face tracking techniques or manual interventions, as it is difficult to eliminate the subtly temporal incoherence of the facial feature point localizations in a video sequence. In this article, we propose a temporal-spatial-smooth warping (TSSW) method to achieve a high temporal coherence for video face editing. TSSW is based on two observations: (1) the control lattices are critical for generating warping surfaces and achieving the temporal coherence between consecutive video frames, and (2) the temporal coherence and spatial smoothness of the control lattices can be simultaneously and effectively preserved. Based upon these observations, we impose the temporal coherence constraint on the control lattices on two consecutive frames, as well as the spatial smoothness constraint on the control lattice on the current frame. TSSW calculates the control lattice (in either the horizontal or vertical direction) by updating the control lattice (in the corresponding direction) on its preceding frame, i.e., minimizing a novel energy function that unifies a data-driven term, a smoothness term, and feature point constraints. The contributions of this article are twofold: (1) we develop TSSW, which is robust to the subtly temporal incoherence of the facial feature point localizations and is effective to preserve the temporal coherence and spatial smoothness of the control lattices for editing faces in videos, and (2) we present a new unified video face editing framework that is capable for improving the performances of facial attractiveness enhancement, makeup transfer, face replacement, and expression manipulation.

Automated segmentation of optic disc in SD-OCT images and cup-to-disc ratios quantification by patch searching-based neural canal opening detection.

Glaucoma is one of the most common causes of blindness worldwide. Early detection of glaucoma is traditionally based on assessment of the cup-to-disc (C/D) ratio, an important indicator of structural changes to the optic nerve head. Here, we present an automated optic disc segmentation algorithm in 3-D spectral domain optical coherence tomography (SD-OCT) volumes to quantify this ratio. The proposed algorithm utilizes a two-stage strategy. First, it detects the neural canal opening (NCO) by finding the points with maximum curvature on the retinal pigment epithelium (RPE) boundary with a spatial correlation smoothness constraint on consecutive B-scans, and it approximately locates the coarse disc margin in the projection image using convex hull fitting. Then, a patch searching procedure using a probabilistic support vector machine (SVM) classifier finds the most likely patch with the NCO in its center in order to refine the segmentation result. Thus, a reference plane can be determined to calculate the C/D radio. Experimental results on 42 SD-OCT volumes from 17 glaucoma patients demonstrate that the proposed algorithm can achieve high segmentation accuracy and a low C/D ratio evaluation error. The unsigned border error for optic disc segmentation and the evaluation error for C/D ratio comparing with manual segmentation are 2.216 ± 1.406 pixels (0.067 ± 0.042 mm) and 0.045 ± 0.033, respectively.

Noise robust spatially regularized myelin water fraction mapping with the intrinsic B1-error correction based on the linearized version of the extended phase graph model.

To improve the quantification accuracy of transverse relaxometry by accounting for B1 -error, after minimizing slice profile imperfections. The slice profile of refocusing pulses was optimized by setting refocusing slice thicknesses three times that of the excitation pulse. The first step of data processing combined the L-curve approach with the linearized version of the extended phase graph model to jointly estimate the temporal regularization constant map and the flip angle error (FAE)-map. The second step improved the noise robustness of the reconstruction by imposing a spatial smoothness constraint on T2 -distributions. The proposed method (spatial-regularization-with-FAE-correction) was evaluated against methods without FAE-correction (conventional-regularization-without-FAE-correction, spatial-regularization-without-FAE-correction) and conventional-regularization-with-FAE-correction using relevant statistics (simulated data: mean square myelin reconstruction error [MSMRE] and averaged-symmetric-Kullbeck-Leibler score [SKL] between returned distributions and ground truths; experimental data: median of mean square error [MMSE] of fitting across entire data-set and coefficient of variation [COV] in white-matter [WM] regions of interest [ROIs]). In simulation, our method resulted in reduced MSMRE (at signal-to-noise ratio [SNR] = 200: MSMRESpatial-regularization-without-FAEC = 0.057; MSMRESpatial-regularization-with-FAEC = 0.0107) and reduced SKL scores (at SNR = 200: SKLSpatial-regularization-without-FAEC = 0.061; SKLSpatial-regularization-with-FAEC = 0.0143). In human volunteers, our method yielded a reduced MSE of fitting (MMSESpatial-regularization-without-FAEC = (2.26 ± 0.60) × 10(-3) ; MMSESpatial-regularization-with-FAEC = (1.57 ± 0.44) × 10(-4) )and also resulted in reduced COV (COVSpatial-regularization-without-FAEC = 0.08-0.19; COVSpatial-regularization-with-FAEC = 0.09-0.12). In a water-phantom, a good correlation between the absolute value of measured B1 -map and FAE-map was found (regression analysis: slope = 1.04; R(2) = 0.66). The proposed method resulted in more accurate and noise robust myelin water fraction maps with improved depiction of subcortical WM structures.

The performance of the spatiotemporal Kalman filter and LORETA in seizure onset localization.

The assumption of spatial-smoothness is often used to solve the bioelectric inverse problem during electroencephalographic (EEG) source imaging, e.g., in low resolution electromagnetic tomography (LORETA). Since the EEG data show a temporal structure, the combination of the temporal-smoothness and the spatial-smoothness constraints may improve the solution of the EEG inverse problem. This study investigates the performance of the spatiotemporal Kalman filter (STKF) method, which is based on spatial and temporal smoothness, in the localization of a focal seizure's onset and compares its results to those of LORETA. The main finding of the study was that the STKF with an autoregressive model of order two significantly outperformed LORETA in the accuracy and consistency of the localization, provided that the source space consists of a whole-brain volumetric grid. In the future, these promising results will be confirmed using data from more patients and performing statistical analyses on the results. Furthermore, the effects of the temporal smoothness constraint will be studied using different types of focal seizures.

GMM based Image Segmentation and Analysis of Image Restoration Tecniques

Now days cauterization of images and video is easy with the advanced technologies in camera. But this images and videos are get easily contaminated by noise due to the characteristics of the image sensors due to this they are mostly blurred so we can loss important data. To avoid this problem we proposed an algorithm for segmentation based on Gaussian mixture model (GMM) and restoration technique with spatial smoothness constraints. The researchers worked on single type of image but the different environmental images are may be affected due to different noises so that researched work is not suitable for all environmental conditions. The proposed algorithm is works on the all type of images, which can remove noises from different diverse field of images with calculating different image parameter. From all of this we can get the optimum solution of suitable filters for combination of image and noise for reduction of noise by comparing of all of that. Here we also present the algorithm for video segmentation & restoration. General Terms Segmentation algorithm, image parameters, image restoration techniques.

Toward naturalistic 2D-to-3D conversion.

Natural scene statistics (NSSs) models have been developed that make it possible to impose useful perceptually relevant priors on the luminance, colors, and depth maps of natural scenes. We show that these models can be used to develop 3D content creation algorithms that can convert monocular 2D videos into statistically natural 3D-viewable videos. First, accurate depth information on key frames is obtained via human annotation. Then, both forward and backward motion vectors are estimated and compared to decide the initial depth values, and a compensation process is applied to further improve the depth initialization. Then, the luminance/chrominance and initial depth map are decomposed by a Gabor filter bank. Each subband of depth is modeled to produce a NSS prior term. The statistical color-depth priors are combined with the spatial smoothness constraint in the depth propagation target function as a prior regularizing term. The final depth map associated with each frame of the input 2D video is optimized by minimizing the target function over all subbands. In the end, stereoscopic frames are rendered from the color frames and their associated depth maps. We evaluated the quality of the generated 3D videos using both subjective and objective quality assessment methods. The experimental results obtained on various sequences show that the presented method outperforms several state-of-the-art 2D-to-3D conversion methods.

Automated retinal layers segmentation in SD-OCT images using dual-gradient and spatial correlation smoothness constraint

Automatic segmentation of retinal layers in spectral domain optical coherence tomography (SD-OCT) images plays a vital role in the quantitative assessment of retinal disease, because it provides detailed information which is hard to process manually. A number of algorithms to automatically segment retinal layers have been developed; however, accurate edge detection is challenging. We developed an automatic algorithm for segmenting retinal layers based on dual-gradient and spatial correlation smoothness constraint. The proposed algorithm utilizes a customized edge flow to produce the edge map and a convolution operator to obtain local gradient map in the axial direction. A valid search region is then defined to identify layer boundaries. Finally, a spatial correlation smoothness constraint is applied to remove anomalous points at the layer boundaries. Our approach was tested on two datasets including 10 cubes from 10 healthy eyes and 15 cubes from 6 patients with age-related macular degeneration. A quantitative evaluation of our method was performed on more than 600 images from cubes obtained in five healthy eyes. Experimental results demonstrated that the proposed method can estimate six layer boundaries accurately. Mean absolute boundary positioning differences and mean absolute thickness differences (mean±SD) were 4.43±3.32μm and 0.22±0.24μm, respectively.

The Role of Spatial Information in Disentangling the Irradiance–Reflectance–Transmittance Ambiguity

In the satellite hyperspectral measures, the contributions of light, surface, and atmosphere are mixed. Applications need separate access to the sources. Conventional inversion techniques usually take a pixelwise spectral-only approach. However, recent improvements in retrieving surface and atmosphere characteristics use heuristic spatial smoothness constraints. In this paper, we theoretically justify such heuristics by analyzing the impact of spatial information on the uncertainty of the solution. The proposed analysis allows to assess in advance the uniqueness (or robustness) of the solution depending on the curvature of a likelihood surface. In situations where pixel-based approaches become unreliable, it turns out that the consideration of spatial information always makes the problem to be better conditioned. With the proposed analysis, this is easily understood since the curvature is consistent with the complexity of the sources measured in terms of the number of significant eigenvalues (or free parameters in the problem). In agreement with recent results in hyperspectral image coding, spatial correlations in the sources imply that the intrinsic complexity of the spatio-spectral representation of the signal is always lower than its spectral-only counterpart. According to this, the number of free parameters in the spatio-spectral inverse problem is smaller, so the spatio-spectral approaches are always better than spectral-only approaches. Experiments using ensembles of actual reflectance values and realistic MODTRAN irradiance and atmosphere radiance and transmittance values show that the proposed analysis successfully predicts the practical difficulty of the problem and the improved quality of spatio-spectral retrieval.

Segmentation and estimation of spatially varying illumination.

In this paper, we present an unsupervised method for segmenting the illuminant regions and estimating the illumination power spectrum from a single image of a scene lit by multiple light sources. Here, illuminant region segmentation is cast as a probabilistic clustering problem in the image spectral radiance space. We formulate the problem in an optimization setting, which aims to maximize the likelihood of the image radiance with respect to a mixture model while enforcing a spatial smoothness constraint on the illuminant spectrum. We initialize the sample pixel set under each illuminant via a projection of the image radiance spectra onto a low-dimensional subspace spanned by a randomly chosen subset of spectra. Subsequently, we optimize the objective function in a coordinate-ascent manner by updating the weights of the mixture components, sample pixel set under each illuminant, and illuminant posterior probabilities. We then estimate the illuminant power spectrum per pixel making use of these posterior probabilities. We compare our method with a number of alternatives for the tasks of illumination region segmentation, illumination color estimation, and color correction. Our experiments show the effectiveness of our method as applied to one hyperspectral and three trichromatic image data sets.

Piecewise Smoothed Value Picking Regularization Applied to 2-D TM and TE Inverse Scattering

The Stepwise Relaxed Value Picking (SRVP) regularization technique, proposed earlier for the iterative reconstruction of piecewise (quasi-)homogeneous objects, is a non-spatial technique, whereby the reconstruction unknowns are clustered around a limited number of-a-priori unknown-reference values. Artifacts have been observed in some 2-D and 3D complex permittivity reconstructions. This paper therefore combines the non-spatial SRVP technique with a spatial smoothing technique, whereby the reference values provided by the former-in each iteration-are employed by the latter to define separate smoothing regions. This way edges are preserved, since the spatial smoothing constraints in the cost function are active within but not across the region boundaries. This combined technique, denoted as Stepwise Relaxed Piecewise Smoothed Value Picking (SRPSVP) regularization, is formulated for the 2.5D microwave inverse scattering problem and is illustrated with reconstructions from the Institut Fresnel 2-D scattering database.

Measurement of left ventricular torsion using block-matching-based speckle tracking for two-dimensional echocardiography

Left ventricular (LV) torsion is a sensitive and global index of LV systolic and diastolic function, but how to noninvasively measure it is challenging. Two-dimensional echocardiography and the block-matching based speckle tracking method were used to measure LV torsion. Main advantages of the proposed method over the previous ones are summarized as follows: (1) The method is automatic, except for manually selecting some endocardium points on the end-diastolic frame in initialization step. (2) The diamond search strategy is applied, with a spatial smoothness constraint introduced into the sum of absolute differences matching criterion; and the reference frame during the search is determined adaptively. (3) The method is capable of removing abnormal measurement data automatically. The proposed method was validated against that using Doppler tissue imaging and some preliminary clinical experimental studies were presented to illustrate clinical values of the proposed method.

Simultaneous image interpolation for stereo images

Simultaneous interpolation of stereo images aims to use a pair of stereo images to reconstruct another pair of images with higher resolutions. To tackle this problem, a simultaneous approach is proposed in this paper. Since the prior image model is important for solving the ill-posed numerical issues encountered in the image interpolation computation, the proposed approach exploits a prior image model that considers both the spatial local smoothness constraint within each reconstructed high-resolution image and the disparity-compensated local smoothness constraint between the pair of reconstructed high-resolution images. The proposed approach requires the disparity to be known in advance or has been accurately estimated by the existing stereo matching algorithm. Experiments are conducted using both artificially generated images and real-life images to demonstrate the superior performance of the proposed approach.

Voxel-level functional connectivity using spatial regularization

Discovering functional connectivity between and within brain regions is a key concern in neuroscience. Due to the noise inherent in fMRI data, it is challenging to characterize the properties of individual voxels, and current methods are unable to flexibly analyze voxel-level connectivity differences. We propose a new functional connectivity method which incorporates a spatial smoothness constraint using regularized optimization, enabling the discovery of voxel-level interactions between brain regions from the small datasets characteristic of fMRI experiments. We validate our method in two separate experiments, demonstrating that we can learn coherent connectivity maps that are consistent with known results. First, we examine the functional connectivity between early visual areas V1 and VP, confirming that this connectivity structure preserves retinotopic mapping. Then, we show that two category-selective regions in ventral cortex - the Parahippocampal Place Area (PPA) and the Fusiform Face Area (FFA) - exhibit an expected peripheral versus foveal bias in their connectivity with visual area hV4. These results show that our approach is powerful, widely applicable, and capable of uncovering complex connectivity patterns with only a small amount of input data.

Slip distribution of the 2007 Bengkulu earthquake inferred from tsunami waveforms and InSAR data

Joint inversion study using tsunami waveforms and InSAR data provides a new way to understand the magnitude and spatial extent of subduction zone earthquakes. A great earthquake (Mw 8.5) occurred on 12 September 2007 off the west coast of Bengkulu, Indonesia. The tsunami generated by the event was recorded by tide gauge stations around the Indian Ocean and by two bottom‐pressure sensors in the deep sea. The ground surface displacements produced by the earthquake on Pagai Islands and on Sumatra Island were observed by Interferometric Synthetic Aperture Radar (InSAR). We estimated the slip distribution of the earthquake by joint inversion incorporating a spatial smoothness constraint, using tsunami waveforms and InSAR data. The total released seismic moment calculated from the slip distribution is 6.7 × 1021 N m (Mw 8.5), consistent with the seismic moment of the Global CMT solution, 6.71 × 1021 N m. The maximum observed tsunami heights along the coast of Bengkulu agree with those computed from the slip distribution. The slip amount of the 2007 earthquake is smaller than the amount of potential slip that has been accumulated since after the 1797 and 1833 events. The premise is that, averaged over long periods of time, the entire fault must slip equal amounts. Therefore the 2007 great earthquake could be followed by several great earthquakes that will rupture the plate interface until the potential slip that has been accumulated is completely released.

Localized anisotropic tomography with well information in VTI media

We develop a concept of localized seismic grid tomography constrained by well information and apply it to building vertically transversely isotropic (VTI) velocity models in depth. The goal is to use a highly automated migration velocity analysis to build anisotropic models that combine optimal image focusing with accurate depth positioning in one step. We localize tomography to a limited volume around the well and jointly invert the surface seismic and well data. Well information is propagated into the local volume by using the method of preconditioning, whereby model updates are shaped to follow geologic layers with spatial smoothing constraints. We analyze our concept with a synthetic data example of anisotropic tomography applied to a 1D VTI model. We demonstrate four cases of introducing additionalinformation. In the first case, vertical velocity is assumed to be known, and the tomography inverts only for Thomsen’s [Formula: see text] and [Formula: see text] profiles using surface seismic data alone. In the second case, tomography simultaneously inverts for all three VTI parameters, including vertical velocity, using a joint data set that consists of surface seismic data and vertical check-shot traveltimes. In the third and fourth cases, sparse depth markers and walkaway vertical seismic profiling (VSP) are used, respectively, to supplement the seismic data. For all four examples, tomography reliably recovers the anisotropic velocity field up to a vertical resolution comparable to that of the well data. Even though walkaway VSP has the additional dimension of angle or offset, it offers no further increase in this resolution limit. Anisotropic tomography with well constraints has multiple advantages over other approaches and deserves a place in the portfolio of model-building tools.