Automatic Parameter Selection Research Articles

Ambipolar diffusion in the Bifrost code

Context. Ambipolar diffusion is a physical mechanism related to the drift between charged and neutral particles in a partially ionized plasma that is key to many different astrophysical systems. However, understanding its effects is challenging due to basic uncertainties concerning relevant microphysical aspects and the strong constraints it imposes on the numerical modeling. Aims. Our aim is to introduce a numerical tool that allows us to address complex problems involving ambipolar diffusion in which, additionally, departures from ionization equilibrium are important or high resolution is needed. The primary application of this tool is for solar atmosphere calculations, but the methods and results presented here may also have a potential impact on other astrophysical systems. Methods. We have developed a new module for the stellar atmosphere Bifrost code that improves its computational capabilities of the ambipolar diffusion term in the generalized Ohm’s law. This module includes, among other things, collision terms adequate to processes in the coolest regions in the solar chromosphere. As the main feature of the module, we have implemented the super time stepping (STS) technique, which allows an important acceleration of the calculations. We have also introduced hyperdiffusion terms to guarantee the stability of the code. Results. We show that to have an accurate value for the ambipolar diffusion coefficient in the solar atmosphere it is necessary to include as atomic elements in the equation of state not only hydrogen and helium, but also the main electron donors like sodium, silicon, and potassium. In addition, we establish a range of criteria to set up an automatic selection of the free parameters of the STS method that guarantees the best performance, optimizing the stability and speed for the ambipolar diffusion calculations. We validate the STS implementation by comparison with a self-similar analytical solution.

Tomographic reconstruction with spatially varying parameter selection

In this paper we propose a new approach for tomographic reconstruction with spatially varying regularization parameter. Our work is based on the SA-TV image restoration model proposed by Dong et al (2011 J. Math. Imag. Vis. 40 82–104) where an automated parameter selection rule for spatially varying parameters has been proposed. Their parameter selection rule, however, only applies if measured imaging data are defined in the image domain, e.g. for image denoising and image deblurring problems. By introducing an auxiliary variable in their model, we show here that this idea can indeed by extended to general inverse imaging problems such as tomographic reconstruction where measurements are not in the image domain. With a spatially varying regularization parameter, the new method can suppress artifacts due to limited data and noise while preserving more details. Using numerical simulations on synthetic and real data, we demonstrate the validity of the proposed approach and its effectiveness for computed tomography reconstruction, delivering reconstruction results that are significantly improved compared to the state-of-the-art.

A BAYESIAN APPROACH BASED ON ACQUISITION FUNCTION FOR OPTIMAL SELECTION OF DEEP LEARNING HYPERPARAMETERS: A CASE STUDY WITH ENERGY MANAGEMENT DATA

With the recent rollout of smart meters, huge amount of data can be generated on hourly and daily basis. Researchers and industry persons can leverage from this big data to make intelligent decisions via deep learning (DL) algorithms. However, the performance of DL algorithms are heavily dependent on the proper selection of parameters. If the hyperparameters are poorly selected, they usually lead to suboptimal results. Traditional approaches include a manual setting of parameters by trial and error methods which is time consuming and difficult process. In this paper, a Bayesian approach based on acquisition is presented to automatic selection of optimal parameters based on provided data. The acquisition function was established to search for the best parameter from the input space and evaluate the next points based on past observations. The tuning process identifies the best model parameters by iterating the objective function and minimizing the loss for optimizable variables such as learning rate and Hidden layersize. To validate the presented approach, we conducted a case study on real-life energy management datasets while constructing a deep learning model on MATLAB platform. A performance comparison was drawn with random parameters and optimal parameters selected by presented approach. The comparison results illustrate that the presented approach is effective as it brings a notable improvement in the performance of learning algorithm.

AutoTuner: High Fidelity and Robust Parameter Selection for Metabolomics Data Processing.

Untargeted metabolomics experiments provide a snapshot of cellular metabolism but remain challenging to interpret due to the computational complexity involved in data processing and analysis. Prior to any interpretation, raw data must be processed to remove noise and to align mass-spectral peaks across samples. This step requires selection of dataset-specific parameters, as erroneous parameters can result in noise inflation. While several algorithms exist to automate parameter selection, each depends on gradient descent optimization functions. In contrast, our new parameter optimization algorithm, AutoTuner, obtains parameter estimates from raw data in a single step as opposed to many iterations. Here, we tested the accuracy and the run-time of AutoTuner in comparison to isotopologue parameter optimization (IPO), the most commonly used parameter selection tool, and compared the resulting parameters’ influence on the properties of feature tables after processing. We performed a Monte Carlo experiment to test the robustness of AutoTuner parameter selection and found that AutoTuner generated similar parameter estimates from random subsets of samples. We conclude that AutoTuner is a desirable alternative to existing tools, because it is scalable, highly robust, and very fast (∼100–1000× speed improvement from other algorithms going from days to minutes). AutoTuner is freely available as an R package through BioConductor.

Generalized Link-Based Additive Survival Models with Informative Censoring

Time to event data differ from other types of data because they are censored. Most of the related estimation techniques assume that the censoring mechanism is noninformative while in many applications it can actually be informative. The aim of this work is to introduce a class of flexible survival models which account for the information provided by the censoring times. The baseline functions are estimated non-parametrically by monotonic P-splines, whereas covariate effects are flexibly determined using additive predictors. Parameter estimation is reliably carried out within a penalized maximum likelihood framework with integrated automatic multiple smoothing parameter selection. We derive the -consistency and asymptotic normality of the noninformative and informative estimators, and shed light on the efficiency gains produced by the newly introduced informative estimator when compared to its non-informative counterpart. The finite sample properties of the estimators are investigated via a Monte Carlo simulation study which highlights the good empirical performance of the proposal. The modeling framework is illustrated on data about infants hospitalized for pneumonia. The models and methods discussed in the article have been implemented in the R package GJRM to allow for transparent and reproducible research. Supplementary materials for this article are available online.

Adaptive parameter selection for weighted-TV image reconstruction problems

We propose an efficient estimation technique for the automatic selection of locally-adaptive Total Variation regularisation parameters based on an hybrid strategy which combines a local maximum-likelihood approach estimating space-variant image scales with a global discrepancy principle related to noise statistics. We verify the effectiveness of the proposed approach solving some exemplar image reconstruction problems and show its outperformance in comparison to state-of-the-art parameter estimation strategies, the former weighting locally the fit with the data [4], the latter relying on a bilevel learning paradigm [8, 9].

Time domain force localization and reconstruction based on hierarchical Bayesian method

In engineering practice, localization and reconstruction of external excitations is a tough problem in absence of prior knowledge. Traditional regularization approaches require assigning regularization parameters and shape parameters before implementing the reconstruction. However, a poor selection of these parameters generally leads to a poor reconstruction. In this paper a time domain hierarchical Bayesian method is proposed to locate and reconstruct the external excitations with automatic selection of parameters. The entire method is performed in two stages. First based on the property of posterior probability distribution function of shape parameters, a novel non-force criterion is proposed to determine the non-force locations quickly. Then based on the determined force locations, a reduced-dimension problem is constructed and the posterior distributions of parameters are sampled by a Metropolis-within-Gibbs sampler with nested blocking technique. Both numerical simulations and laboratory experiment of a cantilever beam under various load conditions are carried out to validate the proposed method.

LinCos-Softmax: Learning Angle-Discriminative Face Representations With Linearity-Enhanced Cosine Logits

In recent years, the angle-based softmax losses have significantly improved the performance of face recognition whereas these loss functions are all based on cosine logit. A potential weakness is that the nonlinearity of the cosine function may undesirably saturate the angular optimization between the features and the corresponding weight vectors, thereby preventing the network from fully learning to maximize the angular discriminability of features. As a result, the generalization of learned features may be compromised. To tackle this issue, we propose a Linear-Cosine Softmax Loss (LinCos-Softmax) to more effectively learn angle-discriminative facial features. The main characteristic of the loss function we propose is the use of an approximated linear logit. Compared with the conventional cosine logit, it has a stronger linear relationship with the angle on enhancing angular discrimination through Taylor expansion. We also propose an automatic scale parameter selection scheme, which can conveniently provide an appropriate scale for different logits without the need for exhaustive parameter search to improve performance. In addition, we propose a margin-enhanced Linear-Cosine Softmax Loss (m-LinCos-Softmax) to further enlarge inter-class distances and reduce intra-class variations. Experimental results on several face recognition benchmarks (LFW, AgeDB-30, CFP-FP, MegaFace Challenge 1) demonstrate the effectiveness of the proposed method and its superiority to existing angular softmax loss variants.

High-Contrast Reflection Tomography With Total-Variation Constraints

Inverse scattering is the process of estimating the spatial distribution of the scattering potential of an object by measuring the scattered wavefields around it. In this article, we consider reflection tomography of high contrast objects that commonly occurs in ground-penetrating radar, exploration geophysics, terahertz imaging, ultrasound, and electron microscopy. Unlike conventional transmission tomography, the reflection regime is severely ill-posed since the measured wavefields contain far less spatial frequency information of the target object. We propose a constrained incremental frequency inversion framework that requires no side information from a background model of the object. Our framework solves a sequence of regularized least-squares subproblems that ensure consistency with the measured scattered wavefield while imposing total-variation and non-negativity constraints. We propose a proximal Quasi-Newton method to solve the resulting subproblem and devise an automatic parameter selection routine to determine the constraint of each subproblem. We validate the performance of our approach on synthetic low-resolution phantoms and with a mismatched forward model test on a high-resolution phantom.

Parameter-Free Landweber Iteration Method for Robust Adaptive Beamforming

In this paper, we propose a novel parameter-free Landweber iteration (LI) method to improve the robustness of a beamformer based on the LI algorithm and the L-curve criterion. The critical aspect of the proposed method is the automatic parameter selection procedure, which determines the iteration stop number in a scenario-dependent manner. This parameter-free method can flexibly adapt to different communication environments. To further improve the computational efficiency, an accelerated parameter-free LI method is also proposed that incorporates the Newton iteration into the LI algorithm without compromising the robustness of the beamformer. Numerical experiments demonstrate the effectiveness of the proposed beamformer.

Analysis and automatic parameter selection of a variational model for mixed Gaussian and salt-and-pepper noise removal

We analyse a variational regularisation problem for mixed noise removal that has been recently proposed in Calatroni et al (2017 SIAM J. Imaging Sci. 10 1196–233). The data discrepancy term of the model combines L1 and L2 terms in an infimal convolution fashion and it is appropriate for the joint removal of Gaussian and Salt & Pepper noise. Here, we perform a fine analysis of the model that emphasises the balancing effect of the two parameters appearing in the discrepancy term. Namely, we study the asymptotic behaviour of the model for large and small values of these parameters and we compare its solutions to the ones of the corresponding variational models with L1 and L2 data fidelity. Extensions to the general linear inverse problems setting are also discussed. Furthermore, we compute exact solutions to the denoising problem, for simple data functions in the case of total variation regularisation. Using these theoretical results, we then analytically study a bilevel optimisation strategy for the automatic selection of model parameters by means of a training set. Finally, we report some numerical results which confirm the validity of our analysis and the use of popular data models in the case of ‘blind’ optimal noise model selection.

Stack transition artifact removal (STAR) for cardiac CT.

In cardiac computed tomography (CT), irregular motion may lead to unique artifacts for scanners with a longitudinal collimation that does not cover the entire heart. Given partial coverage, subvolumes, or stacks, may be reconstructed and used to assemble a final CT volume. Irregular motion, for example, due to cardiac arrhythmia or breathing, may cause mismatch between neighboring stacks and therefore discontinuities within the final CT volume. The aim of this work is the removal of the discontinuities that are hereafter referred to as stack transition artifacts. A stack transition artifact removal (STAR) is achieved using a symmetric deformable image registration. A symmetric Demons algorithm was implemented and applied to stacks to remove mismatch and therefore the stack transition artifacts. The registration can be controlled with one parameter that affects the smoothness of the deformation vector field (DVF). The latter is crucial for realistically transforming the stacks. Different smoothness settings as well as an entirely automatic parameter selection that considers the required deformation magnitude for each registration were tested with patient data. Thirteen datasets were evaluated. Simulations were performed on two additional datasets. STAR considerably improved image quality while computing realistic DVFs. Discontinuities, for example, appearing as breaks or cuts in coronary arteries or cardiac valves, were removed or considerably reduced. A constant smoothing parameter that ensured satisfactory results for all datasets was found. The automatic parameter selection was able to find a proper setting for each individual dataset. Consequently, no over regularization of the DVF occurred that would unnecessarily limit the registration accuracy for cases with small deformations. The automatic parameter selection yielded the best overall results and provided a registration method for cardiac data that does not require user input.

Denoising of cross-spectral matrices using canonical coherence.

Disturbing noise in microphone signals is an often occurring problem. The present paper has a main focus on measurements with microphone arrays in wind tunnels, where flow noise will be generated in the individual microphones. Considering stationary operational conditions, the cross-spectral matrix (CSM) is first averaged. Subsequently it is applied for aerodynamic noise-source mapping on a vehicle in the flow using one of several possible array processing algorithms. Several methods exist to suppress the flow-noise effects. Long-time averaging will gradually concentrate the flow-noise contributions on the CSM diagonal, and some array processing algorithms can avoid use of the diagonal. Other algorithms need the diagonal. Denoising methods that subtract a maximum of signal power from the diagonal, while retaining all off-diagonal elements unchanged and the matrix positive semidefinite, have been shown to be limited by remaining off-diagonal flow-noise contributions. A couple of algorithms exist that can overcome that limitation by supporting modifications outside the diagonal, but these methods seem to be limited by high computational cost or difficult parameter selection. The present paper introduces a unique canonical-coherence-based method, which is computationally very fast and with automated parameter selection. The performance is investigated through simulated and real measurements.

Arbitrary Screen-Aware Manga Reading Framework with Parameter-Optimized Panel Extraction

While reading Manga on smart devices, it is inconvenient to drag and zoom through contents. Therefore, this work proposes a user-friendly and intuitive Manga reading framework for arbitrary reading conditions. Our framework creates a reading flow by breaking pages into contextual panels, organizing panels as display units, grouping units as a display list according to reading conditions, and playing the list with designed eye-movement-simulated transitions. The core is parameter-optimized panel extraction, which uses the occupied area of extracted panels and the ratio of borders parallel to each other to automate parameter selection for matching up identified panel corners and borders. While applying to 70 Manga chapters, our extraction performs comparatively better against two state-of-the-art algorithms and successfully locates “break-outs.” Furthermore, a usability test on three chapters demonstrates that our framework is effective and visually pleasing for arbitrary Manga-reading scenarios while preserving contextual ideas through panel arrangement.

A multi-scale level set method based on local features for segmentation of images with intensity inhomogeneity

Images with intensity inhomogeneity pose significant challenges in image segmentation. Local region-based level set models have recently been recognized as promising methods to segment such images. In these models, local intensity information in a neighborhood of predetermined size is extracted and then embedded into the energy functional, guiding the evolution of deformable contour toward desired boundaries. The local neighborhood intensities are assumed to be rather constant; therefore, the selection of neighborhood size greatly influences effectiveness and robustness. Complex image characteristics, such as variation in degree of intensity inhomogeneity and noise levels among regions, can lead to severe challenges for accurate image segmentation when using only a fixed scale parameter for local regions. We propose a new multi-scale local feature-based level set method for image segmentation with an improved strategy based on previous studies of multi-scale image filtering methods, which allow for automatic selection of filtering scale parameters. Our novel method can adaptively determine the optimal scale parameter for each pixel during contour evolution, alleviating the challenges caused by severe intensity inhomogeneity. First, we define a Local Maximum Description Difference feature (LMDD), based on multi-scale local region descriptors. We incorporate the LMDD, associated with the maximum response of multi-scale high-pass filters for each pixel, into three local region based level set models with Chan-Vese (CV)-like structure to construct the energy functional. Finally, we complete the segmentation through minimization of this energy. Our experimental results illustrate the good performance of the proposed level set method for segmenting images with severe intensity inhomogeneity.

The Optimized Multi-Scale Permutation Entropy and Its Application in Compound Fault Diagnosis of Rotating Machinery.

Multi-scale permutation entropy (MPE) is a statistic indicator to detect nonlinear dynamic changes in time series, which has merits of high calculation efficiency, good robust ability, and independence from prior knowledge, etc. However, the performance of MPE is dependent on the parameter selection of embedding dimension and time delay. To complete the automatic parameter selection of MPE, a novel parameter optimization strategy of MPE is proposed, namely optimized multi-scale permutation entropy (OMPE). In the OMPE method, an improved Cao method is proposed to adaptively select the embedding dimension. Meanwhile, the time delay is determined based on mutual information. To verify the effectiveness of OMPE method, a simulated signal and two experimental signals are used for validation. Results demonstrate that the proposed OMPE method has a better feature extraction ability comparing with existing MPE methods.

Frequency-splitting dynamic MRI reconstruction using multi-scale 3D convolutional sparse coding and automatic parameter selection.

In this paper, we propose a novel image reconstruction algorithm using multi-scale 3D convolutional sparse coding and a spectral decomposition technique for highly undersampled dynamic Magnetic Resonance Imaging (MRI) data. The proposed method recovers high-frequency information using a shared 3D convolution-based dictionary built progressively during the reconstruction process in an unsupervised manner, while low-frequency information is recovered using a total variation-based energy minimization method that leverages temporal coherence in dynamic MRI. Additionally, the proposed 3D dictionary is built across three different scales to more efficiently adapt to various feature sizes, and elastic net regularization is employed to promote a better approximation to the sparse input data. We also propose an automatic parameter selection technique based on a genetic algorithm to find optimal parameters for our numerical solver which is a variant of the alternating direction method of multipliers (ADMM). We demonstrate the performance of our method by comparing it with state-of-the-art methods on 15 single-coil cardiac, 7 single-coil DCE, and a multi-coil brain MRI datasets at different sampling rates (12.5%, 25% and 50%). The results show that our method significantly outperforms the other state-of-the-art methods in reconstruction quality with a comparable running time and is resilient to noise.

Enhanced global optimization methods applied to complex fisheries stock assessment models

Statistical fisheries models are frequently used by researchers and agencies to understand the behavior of marine ecosystems or to estimate the maximum acceptable catch of different species of commercial interest. The parameters of these models are usually adjusted through the use of optimization algorithms. Unfortunately, the choice of the best optimization method is far from trivial. This work proposes the use of population-based algorithms to improve the optimization process of the Globally applicable Area Disaggregated General Ecosystem Toolbox (Gadget), a flexible framework that allows the development of complex statistical marine ecosystem models. Specifically, parallel versions of the Differential Evolution (DE) and the Particle Swarm Optimization (PSO) methods are proposed. The proposals include an automatic selection of the internal parameters to reduce the complexity of their usage, and a restart mechanism to avoid local minima. The resulting optimization algorithms were called PMA (Parallel Multirestart Adaptive) DE and PMA PSO respectively. Experimental results prove that the new algorithms are faster and produce more accurate solutions than the other parallel optimization methods already included in Gadget. Although the new proposals have been evaluated on fisheries models, there is nothing specific to the tested models in them, and thus they can be also applied to other optimization problems. Moreover, the PMA scheme proposed can be seen as a template that can be easily applied to other population-based heuristics.

DoFORC tool for calculating first-order reversal curve diagrams of noisy scattered data

The first-order reversal curve (FORC) diagram method is one of the most successful characterization techniques used to characterize complex hysteretic phenomena not only in magnetism but also in other areas of science like in ferroelectricity, geology, archeology, in spin-transition materials, etc. Because the definition of the FORC diagram involves a second-order derivative, the main problem in their numerical calculation is that the derivative of a function for which only discrete noise-contaminated data values are available magnifies the noise that is inevitably present in measurements. In this paper, we present the doFORC tool for calculating FORC diagrams of noise scattered data. It can provide both a smooth approximation of the measured magnetization and all its partial derivatives. Even if doFORC is mainly dedicated to FORC diagrams’ computation, it can process a general set of arbitrarily distributed two-dimensional points. doFORC is a free, portable application working on various operating systems, with an easy to use graphical interface, with four regression methods implemented to obtain a smooth approximation of the data which may then be differentiated to obtain approximations for derivatives. In order to perform the diagnostics and goodness of fit, doFORC computes residuals to characterize the difference between the observed and predicted values, generalized cross-validation to measure the predictive performance, two information criteria to quantify the information that is lost by using an approximate model, and three degrees of freedom to compare different amounts of smoothing being performed by different smoothing methods. Based on these, doFORC can perform automatic smoothing parameter selection.

Improving Classification Quality in Uncertain Graphs

In many real applications that use and analyze networked data, the links in the network graph may be erroneous or derived from probabilistic techniques. In such cases, the node classification problem can be challenging, since the unreliability of the links may affect the final results of the classification process. If the information about link reliability is not used explicitly, then the classification accuracy in the underlying network may be affected adversely. In this article, we focus on situations that require the analysis of the uncertainty that is present in the graph structure. We study the novel problem of node classification in uncertain graphs, by treating uncertainty as a first-class citizen. We propose two techniques based on a Bayes model and automatic parameter selection and show that the incorporation of uncertainty in the classification process as a first-class citizen is beneficial. We experimentally evaluate the proposed approach using different real data sets and study the behavior of the algorithms under different conditions. The results demonstrate the effectiveness and efficiency of our approach.