Regularized Cost Function Research Articles

Nonrigid Registration of Ultrasound and MRI Using Contextual Conditioned Mutual Information

Mutual information (MI) quantifies the information that is shared between two random variables and has been widely used as a similarity metric for multi-modal and uni-modal image registration. A drawback of MI is that it only takes into account the intensity values of corresponding pixels and not of neighborhoods. Therefore, it treats images as "bag of words" and the contextual information is lost. In this work, we present Contextual Conditioned Mutual Information (CoCoMI), which conditions MI estimation on similar structures. Our rationale is that it is more likely for similar structures to undergo similar intensity transformations. The contextual analysis is performed on one of the images offline. Therefore, CoCoMI does not significantly change the registration time. We use CoCoMI as the similarity measure in a regularized cost function with a B-spline deformation field and efficiently optimize the cost function using a stochastic gradient descent method. We show that compared to the state of the art local MI based similarity metrics, CoCoMI does not distort images to enforce erroneous identical intensity transformations for different image structures. We further present the results on nonrigid registration of ultrasound (US) and magnetic resonance (MR) patient data from image-guided neurosurgery trials performed in our institute and publicly available in the BITE dataset. We show that CoCoMI performs significantly better than the state of the art similarity metrics in US to MR registration. It reduces the average mTRE over 13 patients from 4.12 mm to 2.35 mm, and the maximum mTRE from 9.38 mm to 3.22 mm.

Linear Temporal Logic for Regular Cost Functions

Regular cost functions have been introduced recently as an extension to the notion of regular languages with counting capabilities, which retains strong closure, equivalence, and decidability properties. The specificity of cost functions is that exact values are not considered, but only estimated. In this paper, we define an extension of Linear Temporal Logic (LTL) over finite words to describe cost functions. We give an explicit translation from this new logic to two dual form of cost automata, and we show that the natural decision problems for this logic are PSPACE-complete, as it is the case in the classical setting. We then algebraically characterize the expressive power of this logic, using a new syntactic congruence for cost functions introduced in this paper.

Corrections to the displacement estimation based on analytic minimization of adaptive regularized cost functions for ultrasound elastography.

Ultrasound elastography has been widely applied in clinical diagnosis. To produce high-quality elastograms, displacement estimation is important to generate ne displacement map from the original ratio-frequency signals. Traditional displacement estimation methods are based on the local information of signals pair, such as cross-correlation method, phase zero estimation. However, the tissue movement is nonlocal during realistic elasticity process due to the compression coming from the surface. Recently, regularized cost functions have been broadly used in ultrasound elastography. In this paper, we tested the using of analytic minimization of adaptive regularized cost function, a combination of different regularized cost functions, to correct the displacement estimation calculated by cross-correlation method directly or by lateral displacement guidance. We have demonstrated that the proposed method exhibit obvious advantages in terms of imaging quality with higher levels of elastographic signal-to-noise ratio and elastographic contrast-to-noise ratio in the simulation and phantom experiments respectively.

Self-similarity weighted mutual information: A new nonrigid image registration metric

Mutual information (MI) has been widely used as a similarity measure for rigid registration of multi-modal and uni-modal medical images. However, robust application of MI to deformable registration is challenging mainly because rich structural information, which are critical cues for successful deformable registration, are not incorporated into MI. We propose a self-similarity weighted graph-based implementation of α-mutual information (α-MI) for nonrigid image registration. We use a self-similarity measure that uses local structural information and is invariant to rotation and to local affine intensity distortions, and therefore the new Self Similarity α-MI (SeSaMI) metric inherits these properties and is robust against signal nonstationarity and intensity distortions. We have used SeSaMI as the similarity measure in a regularized cost function with B-spline deformation field to achieve nonrigid registration. Since the gradient of SeSaMI can be derived analytically, the cost function can be efficiently optimized using stochastic gradient descent methods. We show that SeSaMI produces a robust and smooth cost function and outperforms the state of the art statistical based similarity metrics in simulation and using data from image-guided neurosurgery.

Ultrasound elastography using multiple images

Displacement estimation is an essential step for ultrasound elastography and numerous techniques have been proposed to improve its quality using two frames of ultrasound RF data. This paper introduces a technique for calculating a displacement field from three (or multiple) frames of ultrasound RF data. To calculate a displacement field using three images, we first derive constraints on variations of the displacement field with time using mechanics of materials. These constraints are then used to generate a regularized cost function that incorporates amplitude similarity of three ultrasound images and displacement continuity. We optimize the cost function in an expectation maximization (EM) framework. Iteratively reweighted least squares (IRLS) is used to minimize the effect of outliers. An alternative approach for utilizing multiple images is to only consider two frames at any time and sequentially calculate the strains, which are then accumulated. We formally show that, compared to using two images or accumulating strains, the new algorithm reduces the noise and eliminates ambiguities in displacement estimation. The displacement field is used to generate strain images for quasi-static elastography. Simulation, phantom experiments and in vivo patient trials of imaging liver tumors and monitoring ablation therapy of liver cancer are presented for validation. We show that even with the challenging patient data, where it is likely to have one frame among the three that is not optimal for strain estimation, the introduction of physics-based prior as well as the simultaneous consideration of three images significantly improves the quality of strain images. Average values for strain images of two frames versus ElastMI are: 43 versus 73 for SNR (signal to noise ratio) in simulation data, 11 versus 15 for CNR (contrast to noise ratio) in phantom data, and 5.7 versus 7.3 for CNR in patient data. In addition, the improvement of ElastMI over both utilizing two images and accumulating strains is statistically significant in the patient data, with p-values of respectively 0.006 and 0.012.

Regular Cost Functions, Part I: Logic and Algebra over Words

The theory of regular cost functions is a quantitative extension to the classical notion of regularity. A cost function associates to each input a non-negative integer value (or infinity), as opposed to languages which only associate to each input the two values "inside" and "outside". This theory is a continuation of the works on distance automata and similar models. These models of automata have been successfully used for solving the star-height problem, the finite power property, the finite substitution problem, the relative inclusion star-height problem and the boundedness problem for monadic-second order logic over words. Our notion of regularity can be -- as in the classical theory of regular languages -- equivalently defined in terms of automata, expressions, algebraic recognisability, and by a variant of the monadic second-order logic. These equivalences are strict extensions of the corresponding classical results. The present paper introduces the cost monadic logic, the quantitative extension to the notion of monadic second-order logic we use, and show that some problems of existence of bounds are decidable for this logic. This is achieved by introducing the corresponding algebraic formalism: stabilisation monoids.

Structured Low-Rank Approximation with Missing Data

We consider low-rank approximation of affinely structured matrices with missing elements. The method proposed is based on reformulation of the problem as inner and outer optimization. The inner minimization is a singular linear least-norm problem and admits an analytic solution. The outer problem is a nonlinear least-squares problem and is solved by local optimization methods: minimization subject to quadratic equality constraints and unconstrained minimization with regularized cost function. The method is generalized to weighted low-rank approximation with missing values and is illustrated on approximate low-rank matrix completion, system identification, and data-driven simulation problems. An extended version of this paper is a literate program, implementing the method and reproducing the presented results.

Subband Adaptive Filtering withl1-Norm Constraint for Sparse System Identification

This paper presents a new approach of the normalized subband adaptive filter (NSAF) which directly exploits the sparsity condition of an underlying system for sparse system identification. The proposed NSAF integrates a weightedl1-norm constraint into the cost function of the NSAF algorithm. To get the optimum solution of the weightedl1-norm regularized cost function, a subgradient calculus is employed, resulting in a stochastic gradient based update recursion of the weightedl1-norm regularized NSAF. The choice of distinct weightedl1-norm regularization leads to two versions of thel1-norm regularized NSAF. Numerical results clearly indicate the superior convergence of thel1-norm regularized NSAFs over the classical NSAF especially when identifying a sparse system.

Regularized nonnegative matrix factorization using Gaussian mixture priors for supervised single channel source separation

We introduce a new regularized nonnegative matrix factorization (NMF) method for supervised single-channel source separation (SCSS). We propose a new multi-objective cost function which includes the conventional divergence term for the NMF together with a prior likelihood term. The first term measures the divergence between the observed data and the multiplication of basis and gains matrices. The novel second term encourages the log-normalized gain vectors of the NMF solution to increase their likelihood under a prior Gaussian mixture model (GMM) which is used to encourage the gains to follow certain patterns. In this model, the parameters to be estimated are the basis vectors, the gain vectors and the parameters of the GMM prior. We introduce two different ways to train the model parameters, sequential training and joint training. In sequential training, after finding the basis and gains matrices, the gains matrix is then used to train the prior GMM in a separate step. In joint training, within each NMF iteration the basis matrix, the gains matrix and the prior GMM parameters are updated jointly using the proposed regularized NMF. The normalization of the gains makes the prior models energy independent, which is an advantage as compared to earlier proposals. In addition, GMM is a much richer prior than the previously considered alternatives such as conjugate priors which may not represent the distribution of the gains in the best possible way. In the separation stage after observing the mixed signal, we use the proposed regularized cost function with a combined basis and the GMM priors for all sources that were learned from training data for each source. Only the gain vectors are estimated from the mixed data by minimizing the joint cost function. We introduce novel update rules that solve the optimization problem efficiently for the new regularized NMF problem. This optimization is challenging due to using energy normalization and GMM for prior modeling, which makes the problem highly nonlinear and non-convex. The experimental results show that the introduced methods improve the performance of single channel source separation for speech separation and speech–music separation with different NMF divergence functions. The experimental results also show that, using the GMM prior gives better separation results than using the conjugate prior.

Scheduling an unbounded batching machine with family jobs and setup times

We consider the problem of scheduling n jobs on an unbounded batching machine that can process any number of jobs belonging to the same family simultaneously in the same batch. All jobs in the same batch complete at the same time. Jobs belonging to different families cannot be processed in the same batch, and setup times are required to switch between batches that process jobs from different families. For a fixed number of families m, we present a generic forward dynamic programming algorithm that solves the problem of minimizing an arbitrary regular cost function in pseudopolynomial time. We also present a polynomial-time backward dynamic programming algorithm with time complexity O (mn(n/m+1)m) for minimizing any additive regular minsum objective function and any incremental regular minmax objective function. The effectiveness of our dynamic programming algorithm is shown by computational experiments based on the scheduling of the heat-treating process in a steel manufacturing plant.

Least squares support vector machines with tuning based on chaotic differential evolution approach applied to the identification of a thermal process

In the past decade, support vector machines (SVMs) have gained the attention of many researchers. SVMs are non-parametric supervised learning schemes that rely on statistical learning theory which enables learning machines to generalize well to unseen data. SVMs refer to kernel-based methods that have been introduced as a robust approach to classification and regression problems, lately has handled nonlinear identification problems, the so called support vector regression. In SVMs designs for nonlinear identification, a nonlinear model is represented by an expansion in terms of nonlinear mappings of the model input. The nonlinear mappings define a feature space, which may have infinite dimension. In this context, a relevant identification approach is the least squares support vector machines (LS-SVMs). Compared to the other identification method, LS-SVMs possess prominent advantages: its generalization performance (i.e. error rates on test sets) either matches or is significantly better than that of the competing methods, and more importantly, the performance does not depend on the dimensionality of the input data. Consider a constrained optimization problem of quadratic programing with a regularized cost function, the training process of LS-SVM involves the selection of kernel parameters and the regularization parameter of the objective function. A good choice of these parameters is crucial for the performance of the estimator. In this paper, the LS-SVMs design proposed is the combination of LS-SVM and a new chaotic differential evolution optimization approach based on Ikeda map (CDEK). The CDEK is adopted in tuning of regularization parameter and the radial basis function bandwith. Simulations using LS-SVMs on NARX (Nonlinear AutoRegressive with eXogenous inputs) for the identification of a thermal process show the effectiveness and practicality of the proposed CDEK algorithm when compared with the classical DE approach.

Erratum to: Scheduling jobs on parallel machines to minimize a regular step total cost function

Our paper entitled “Scheduling jobs on parallel machines to minimize a regular step total cost function” appeared online in Journal of Scheduling. It is an extension from the conference article (Detienne et al. 2009). In these works, we missed two studies considering stepwise cost functions in scheduling problems, in which the authors refer to the optimization criterion as stepwise tardiness. Curry and Peters (2005) investigates a rescheduling problem in a parallel machine environment. The authors examine the tradeoff between schedule stability and tardiness cost, using simulation. A job can be on time or late for 1, 2, 3 or 4 days, thus yielding a stepwise cost function. They also consider machine reassignment costs. Within the simulation, the rescheduling problem is solved using a branch-andprice algorithm. They propose an Integer Linear Program-

RLS Algorithm With Convex Regularization

In this letter, the RLS adaptive algorithm is considered in the system identification setting. The RLS algorithm is regularized using a general convex function of the system impulse response estimate. The normal equations corresponding to the convex regularized cost function are derived, and a recursive algorithm for the update of the tap estimates is established. We also introduce a closed-form expression for selecting the regularization parameter. With this selection of the regularization parameter, we show that the convex regularized RLS algorithm performs as well as, and possibly better than, the regular RLS when there is a constraint on the value of the convex function evaluated at the true weight vector. Simulations demonstrate the superiority of the convex regularized RLS with automatic parameter selection over regular RLS for the sparse system identification setting.

An exact approach for scheduling jobs with regular step cost functions on a single machine

This paper studies a single-machine scheduling problem whose objective is to minimize a regular step total cost function. Lower and upper bounds, obtained from linear and Lagrangian relaxations of different Integer Linear Programming formulations, are compared. A dedicated exact approach is presented, based on a Lagrangian relaxation. It consists of finding a Constrained Shortest Path in a specific graph designed to embed a dominance property. Filtering rules are developed for this approach in order to reduce the size of the graph, and the problem is solved by successively removing infeasible paths from the graph. Numerical experiments are conducted to evaluate the lower and upper bounds. Moreover, the exact approach is compared with a standard solver and a naive branch-and-bound algorithm.

Boundary data completion: the method of boundary value problem factorization

We consider the following data completion problem for the Laplace equation in the cylindrical domain: ( is a smooth bounded open set and a > 0), limited by the faces and . The Neumann and Dirichlet boundary conditions are given on Γ0 while no condition is given on Γa. The completion data problem consists in recovering a boundary condition on Γa. This problem has been known to be ill-posed since Hadamard [12]. The problem is set as an optimal control problem with a regularized cost function. To obtain directly an approximation of the missing data on Γa we use the method of factorization of elliptic boundary value problems. This method allows us to factorize a boundary value problem in the product of two parabolic problems. Here it is applied to the optimality system (i.e. jointly on the state and adjoint state equations).

Phase pattern denoising using a regularized cost function with complex-valued Markov random fields based on a discrete model

We present a simple and effective method for denoising phase patterns based on a discrete model. The proposed filtering method transforms the image denoising problem to solving the energy diffusion problem of a system with complex-valued fields. We establish an appropriate cost function that uses the discrete form of complex-valued Markov random fields. The attractiveness of the proposed filtering method includes three points: the first is that the filtering process can be easily implemented using an iterative method, the second is that 2π phase jumps are well preserved, and the third is its little computational effort. The performance of the proposed method is demonstrated by simulated and experimentally obtained phase patterns.

Real-Time Regularized Ultrasound Elastography

This paper introduces two real-time elastography techniques based on analytic minimization (AM) of regularized cost functions. The first method (1D AM) produces axial strain and integer lateral displacement, while the second method (2D AM) produces both axial and lateral strains. The cost functions incorporate similarity of radio-frequency (RF) data intensity and displacement continuity, making both AM methods robust to small decorrelations present throughout the image. We also exploit techniques from robust statistics to make the methods resistant to large local decorrelations. We further introduce Kalman filtering for calculating the strain field from the displacement field given by the AM methods. Simulation and phantom experiments show that both methods generate strain images with high SNR, CNR and resolution. Both methods work for strains as high as 10% and run in real-time. We also present in vivo patient trials of ablation monitoring. An implementation of the 2D AM method as well as phantom and clinical RF-data can be downloaded.

Scheduling jobs on parallel machines to minimize a regular step total cost function

This paper deals with a parallel machine scheduling problem whose objective is to minimize a regular step total cost function. A real world application of the problem is presented, and Mixed Integer Linear Programming models are described for the cases with and without release dates, as well as a dedicated constraint generation procedure. Experimental results are reported and discussed to evaluate the relevance of the different approaches on well-known special cases and the general case.

Far-field acoustic source localization and bearing estimation using Σ Δ learners

Localization of acoustic sources using miniature microphone arrays poses a significant challenge due to fundamental limitations imposed by the physics of sound propagation. With sub-wavelength distances between the microphones, resolving acute localization cues become difficult due to precision artifacts. In this paper we propose a framework which overcomes this limitation by integrating signal-measurement (analog-to-digital conversion) with statistical learning (bearing estimation). At the core of the proposed approach is a min-max stochastic optimization of a regularized cost function that embeds manifold learning within ¿¿ modulation. As a result, the algorithm directly produces a quantized sequence of the bearing estimates whose precision can be improved asymptotically similar to a conventional ¿¿ modulators. In this paper we present a hardware implementation of a miniature acoustic source localizer which comprises of: (a) a common-mode canceling microphone array and (b) a ¿¿ integrated circuit which produces bearing parameters. The parameters are then combined in an estimation procedure that can achieve a linear range from 0°-90°. Measured results from a prototype fabricated in a 0.5 ¿m CMOS process demonstrate that the proposed localizer can reliably estimate the bearing of an acoustic source with a resolution less than 2° while consuming less than 75 ¿W of power.

A Min–Max Optimization Framework for Designing $\Sigma\Delta$ Learners: Theory and Hardware

In this paper, we present a framework for constructing ΣΔ learning algorithms and hardware that can identify and track low-dimensional manifolds embedded in a high-dimensional analog signal space. At the core of the proposed approach is a min-max stochastic optimization of a regularized cost function that combines machine learning with ΣΔ modulation. As a result, the algorithm not only produces a quantized sequence of the transformed analog signals but also a quantized representation of the transform itself. The framework is generic and can be extended to higher order ΣΔ modulators and for different signal transformations. In this paper, the ΣΔ learning is demonstrated for identifying linear compression manifolds, which can eliminate redundant AD conversion (ADC) paths. This improves the energy efficiency of the proposed architecture compared to a conventional multichannel data acquisition system. Measured results from a four channel prototype fabricated in a 0.5 μm CMOS process has been used to verify the energy efficiency of the ΣΔ learner and to demonstrate its real-time adaptation capabilities that are consistent with the theoretical and simulated results. One of the salient features of ΣΔ learning is its self-calibration property, whereby the performance remains unchanged even in the presence of computational artifacts (mismatch and nonlinearities). This property makes the proposed architecture ideal for implementing practical high-dimensional AD converters.