Ill-posed Research Articles

Improved adaptive regularization for simulated annealing inversion of transient electromagnetic

Geophysical inversion usually involves ill-posed problem. Regularization is the most commonly used method to mitigate this problem. There are many regularization parameter selection methods, among which the adaptive regularization method can automatically update parameters during iteration, reducing the difficulty of parameter selection. Therefore, it is widely used in linear inversion. However, there are very few studies on the use of adaptive regularization methods in stochastic optimization algorithms. The biggest difficulty is that in stochastic optimization algorithms, the search direction of any iteration is completely random. Data fitting term and stabilizing term vary in a wide range, making it difficult for traditional methods to work. In this paper, we consider the contributions of the data fitting term and the stabilizing term in the objective function and give an improved adaptive regularization method for very fast simulated annealing (VFSA) inversion for transient electromagnetic (TEM) data. The optimized method adjusts the two terms dynamically to make them in balance. We have designed several numerical experiments, and the experimental results demonstrate that the method in this paper not only accelerates the convergence, but also the inversion results are very little affected by the initial regularization parameter. Finally, we apply this method to field data, and the inversion results show very good agreements with nearby borehole data.

Physics-Informed Deep Learning Approach for Reintroducing Atomic Detail in Coarse-Grained Configurations of Multiple Poly(lactic acid) Stereoisomers.

Multiscale modeling of complex molecular systems, such as macromolecules, encompasses methods that combine information from fine and coarse representations of molecules to capture material properties over a wide range of spatiotemporal scales. Being able to exchange information between different levels of resolution is essential for the effective transfer of this information. The inverse problem of reintroducing atomistic degrees of freedom in coarse-grained (CG) molecular configurations is particularly challenging as, from a mathematical point of view, it is an ill-posed problem; the forward mapping from the atomistic to the CG description is typically defined via a deterministic operator ("one-to-one" problem), whereas the reversed mapping from the CG to the atomistic model refers to creating one representative configuration out of many possible ones ("one-to-many" problem). Most of the backmapping methods proposed so far balance accuracy, efficiency, and general applicability. This is particularly important for macromolecular systems with different types of isomers, i.e., molecules that have the same molecular formula and sequence of bonded atoms (constitution) but differ in the three-dimensional configurations of their atoms in space. Here, we introduce a versatile deep learning approach for backmapping multicomponent CG macromolecules with chiral centers, trained to learn structural correlations between polymer configurations at the atomistic level and their corresponding CG descriptions. This method is intended to be simple and flexible while presenting a generic solution for resolution transformation. In addition, the method is aimed to respect the structural features of the molecule, such as local packing, capturing therefore the physical properties of the material. As an illustrative example, we apply the model on linear poly(lactic acid) (PLA) in melt, which is one of the most popular biodegradable polymers. The framework is tested on a number of model systems starting from homopolymer stereoisomers of PLA to copolymers with randomly placed chiral centers. The results demonstrate the efficiency and efficacy of the new approach.

A multi-scale attention residual-based U-Net network for stroke electrical impedance tomography.

Electrical impedance tomography (EIT), a non-invasive, radiation-free, and convenient imaging technique, has been widely used in the diagnosis of stroke. However, due to soft-field nonlinearity and the ill-posed inverse problem, EIT images always suffer from low spatial resolution. Therefore, a multi-scale convolutional attention residual-based U-Net (MARU-Net) network is proposed for stroke reconstruction. Based on the U-Net network, a residual module and a multi-scale convolutional attention module are added to the concatenation layer. The multi-scale module extracts feature information of different sizes, the attention module strengthens the useful information, and the residual module improves the performance of the network. Based on the above advantages, the network is used in the EIT system for stroke imaging. Compared with convolutional neural networks and one-dimensional convolutional neural networks, the MARU-Net network has fewer artifacts, and the reconstructed image is clear. At the same time, the reduction of noisy artifacts in the MARU-Net network is verified. The results show that the image correlation coefficient of the reconstructed image with noise is greater than 0.87. Finally, the practicability of the network is verified by a model physics experiment.

Model-based deep learning for ultrasound localization microscopy

Ultrasound localization microscopy (ULM) effectively visualizes vasculature through localization of microbubbles using ultrasound. However, challenges arise from the domain gap between clean data from controlled lab environments and real-world scenarios. Factors, such as high bubble densities, reduced frame rates, and poor image quality (e.g., due to aberration), impede clinical adoption. Traditional methods face limitations in handling the challenging nature of ULM, which have spurred the integration of data-driven techniques. We specifically focus on model-based deep learning, emphasizing statistical inference techniques for increased robustness on out-of-distribution data. The localization task is expressed as an inverse problem y = Hx + n, where y, x, and n represent the observed ultrasound signal, underlying microbubble localizations, and noise. Inaccuracies in the measurement matrix H arise from oversimplified ultrasound acquisition models (i.e., aberration distorting the point-spread function), necessitating the application of spatial priors to regularize the forward model. The ISTA algorithm can enforce sparsity on x and help solve the ill-posed inverse problem. A deep learning extension, namely learned ISTA, can additionally mitigate errors in the forward model. Beyond spatial priors, imposing temporal priors on individual microbubbles enhances accuracy. KalmanNet, integrating Kalman filtering with deep learning, exemplifies this approach by effectively learning the temporal dynamics of microbubbles.

Solution of the inverse problem of estimating particle size distributions

In this work, we describe two alternative methods for solving the ill-conditioned inverse problem that allows estimating the particle size distribution (PSD) from turbidimetry measurements. The first method uses the inverse Penrose matrix to solve the inverse problem in its discrete form. The second method consists of replacing an ill-posed problem with a collection of well-posed problems, penalizing the norm of the solution, and it is known as the Tikhonov regularization. Both methods are used to solve a synthetic application of the inverse problem by solving the direct problem using a theoretical expression of the distribution of particles sizes function f(D) and considering soft industrial latex particles (NBR), with average particle diameters of: 80.4, 82.8, 83.6, and 84.5 nm; and three illumination wavelengths in the UV-Vis region: 300, 450, and 600 nm. The estimated solution obtained by the inverse Penrose matrix is different from the original solution due to the inverse problem is ill-conditioned. In contrast, when using Tikhonov’s regularization, the estimate obtained is close to the original solution, which proves that the particle size distribution is adequate.

An overview of physics-based data science techniques in ocean acoustics

Scientific applications in ocean acoustics include inversion, signal detection, seabed classification, source localization, and environmental assessment. Traditionally, addressing these problems commonly rely on modeling, deterministic setups, or optimization methods. However, these solutions encounter challenges when faced with non-ideal data or experimental conditions, leading to ill-posed problems or assumptions that deviate from reality. Through a new approach in recent years the integration of data science and machine learning techniques has emerged as a promising solution across various research fields. Only recently have these methodologies been introduced to the field of ocean acoustics, offering efficient and alternative for addressing problems through data-driven approaches. This study presents a collection of techniques developed by the University of Delaware, focusing on acoustic and environmental assessment. The methodologies employed include statistical approaches like maximum entropy and data-driven methods, ranging from simpler strategies such as dictionary learning and image segmentation, to more sophisticated structures like convolutional neural networks and graph neural networks, the latter can exploit spatial information in the data. The results obtained from these diverse methods offer valuable insight, paving the way for innovative alternatives to physics-based problem-solving in ocean acoustics.

Estimate of the Spectrum of Discrete Sequences in Ill-Posed Problems Based on the Study of the Numerical Rank of the Trajectory Matrix

Estimate of the Spectrum of Discrete Sequences in Ill-Posed Problems Based on the Study of the Numerical Rank of the Trajectory Matrix

Structural Tectonic Scheme Creation Based on Seismic-Gravity Models and Isostasy Usage: Ural Case

Process of Earth’s density models creation leads to the solution of direct and inverse gravimetry problems. The inverse problem of gravimetry is a classic example of an ill-posed problem: in the common statement, its solution is not unique and unstably depends on input data. Therefore, it is necessary to determine solutions belonging substantial sets of correctness, choosing reasonable models of an initial approximation. In this paper the application of complex interpretation methods of seismic and gravitational data for the creation of three-dimensional models of crust and the upper mantle are presented. Original algorithms and programs were developed for implementation of these methods. They contain solution of non-linear (structural) inverse problem and the solution of the linear three-dimensional inverse problem taking note of the side sources. Coefficients of the “density-velocity” correlation formulas for a number of geo-traverses were defined. Also, we suggest a technique of tectonic maps construction, which is based on the lithostatic pressure calculation. Its idea can be applied to both two- and three-dimensional cases. In the 2D case we show the way to split the mantle to blocks with vertical boundaries. If lithostatic compensation hypothesis is adopted, the method also allows one to calculate density value for each block. Such separation of the mantle helps to diminish discrepancy between model and observed fields. In 3D case we suggest a method, which can be used to construct tectonic structure maps with information about approximate depth and height of each tectonic block.

A modified algebraic method of mathematical signal processing in radar problems

A new method for approximate solution of ill-posed inverse problems of reconstructing images of objects with angular resolution exceeding the Rayleigh criterion is proposed and justified, i.e. with super resolution. Angular super-resolution allows you to obtain images of objects with increased clarity and distinguish previously invisible details of images of complex objects. In addition, on this basis, the probability of correct solutions to problems of object recognition and identification increases. Mathematically, the problem is reduced to solving the linear Fredholm integral equation of the first kind of convolution type. Solutions are sought with additional conditions in the form of restrictions on the location and size of the desired radiation source, which makes it possible to regularize the problem. The method is a development of one of the parameterization methods the algebraic method. The solution is sought in the form of a representation of the desired function in the area where the source is located in the form of a series expansion over the input sequence of orthogonal functions with unknown coefficients. Thus, the inverse problem is parameterized and reduced to searching for expansion coefficients. The presented method is based on the use of a priori information about the localization area of the radiation source, or on an estimate of the location and size of this area obtained by scanning the viewing sector with a goniometer system. Using zero values of the function describing the source outside this region, for systems based on antenna arrays it is possible to find tens and even hundreds of expansion coefficients of the desired function in a Fourier series. The solution is constructed in the form of an iterative process with a consistent increase in the number of functions used in the expansion until the solution remains stable. The adequacy and stability of the solutions was verified during numerical experiments using a mathematical model. The results of numerical studies show that the presented methods of digital processing of received signals make it possible to achieve an effective angular resolution 3–10 times higher than the Rayleigh criterion. The proposed method makes it possible to miniaturize the antenna system without degrading its characteristics. Compared to known ones, it is relatively simple, which allows it to be used by systems in real time.

Color constancy in real-world settings.

Color constancy denotes the ability to assign a particular and stable color percept to an object, irrespective of its surroundings and illumination. The light reaching the eye confounds illumination and spectral reflectance of the object, making the recovery of constant object color an ill-posed problem. How good the visual system is at accomplishing this task is still a matter of heated debate, despite more than a 100 years of research. Depending on the laboratory task and the specific cues available to observers, color constancy was found to be at levels ranging between 15% and 80%, which seems incompatible with the relatively stable color appearance of objects around us and the consistent usage of color names in real life. Here, we show close-to-perfect color constancy using real objects in a natural task and natural environmental conditions, chosen to mimic the role of color constancy in everyday life. Participants had to identify the color of a (non-present) item familiar to them in an office room under five different experimental illuminations. They mostly selected the same colored Munsell chip as their match to the absent object, even though the light reaching the eye in each case differed substantially. Our results demonstrate that color constancy under ideal conditions in the real world can indeed be exceptionally good. We found it to be as good as visual memory permits and not generally compromised by sensory uncertainty.

Linearized maximum rank correlation estimation when covariates are functional

This paper extends the linearized maximum rank correlation (LMRC) estimation proposed by Shen et al. (2023) to the setting where the covariate is a function. However, this extension is nontrivial due to the difficulty of inverting the covariance operator, which may raise the ill-posed inverse problem, for which we integrate the functional principal component analysis to the LMRC procedure. The proposed estimator is robust to outliers in response and computationally efficient. We establish the rate of convergence of the proposed estimator, which is minimax optimal under certain smoothness assumptions. Furthermore, we extend the proposed estimation procedure to handle discretely observed functional covariates, including both sparse and dense sampling designs, and establish the corresponding rate of convergence. Simulation studies demonstrate that the proposed estimators outperform the other existing methods for some examples. Finally, we apply our method to a real data to illustrate its usefulness.

Untrained Metamaterial-Based Coded Aperture Imaging Optimization Model Based on Modified U-Net

Metamaterial-based coded aperture imaging (MCAI) is a forward-looking radar imaging technique based on wavefront modulation. The scattering coefficients of the target can resolve as an ill-posed inverse problem. Data-based deep-learning methods provide an efficient, but expensive, way for target reconstruction. To address the difficulty in collecting paired training data, an untrained deep radar-echo-prior-based MCAI (DMCAI) optimization model is proposed. DMCAI combines the MCAI model with a modified U-Net for predicting radar echo. A joint loss function based on deep-radar echo prior and total variation is utilized to optimize network weights through back-propagation. A target reconstruction strategy by alternatively using the imaginary and real part of the radar echo signal (STAIR) is proposed to solve the DMCAI. It makes the target reconstruction task turn into an estimation from an input image by the U-Net. Then, the optimized weights serve as a parametrization that bridges the input image and the target. The simulation and experimental results demonstrate the effectiveness of the proposed approach under different SNRs or compression measurements.

Adaptive anisotropic Bayesian meshing for inverse problems

We consider inverse problems estimating distributed parameters from indirect noisy observations through discretization of continuum models described by partial differential or integral equations. It is well understood that errors arising from the discretization can be detrimental for ill-posed inverse problems, as discretization error behaves as correlated noise. While this problem can be avoided with a discretization fine enough to decrease the modeling error level below that of the exogenous noise that is addressed, e.g. by regularization, the computational resources needed to deal with the additional degrees of freedom may increase so much as to require high performance computing environments. Following an earlier idea, we advocate the notion of the discretization as one of the unknowns of the inverse problem, which is updated iteratively together with the solution. In this approach, the discretization, defined in terms of an underlying metric, is refined selectively only where the representation power of the current mesh is insufficient. In this paper we allow the metrics and meshes to be anisotropic, and we show that this leads to significant reduction of memory allocation and computing time.

Convergent Data-Driven Regularizations for CT Reconstruction

The reconstruction of images from their corresponding noisy Radon transform is a typical example of an ill-posed linear inverse problem as arising in the application of computerized tomography (CT). As the (naïve) solution does not depend on the measured data continuously, regularization is needed to reestablish a continuous dependence. In this work, we investigate simple, but yet still provably convergent approaches to learning linear regularization methods from data. More specifically, we analyze two approaches: one generic linear regularization that learns how to manipulate the singular values of the linear operator in an extension of our previous work, and one tailored approach in the Fourier domain that is specific to CT-reconstruction. We prove that such approaches become convergent regularization methods as well as the fact that the reconstructions they provide are typically much smoother than the training data they were trained on. Finally, we compare the spectral as well as the Fourier-based approaches for CT-reconstruction numerically, discuss their advantages and disadvantages and investigate the effect of discretization errors at different resolutions.

Single image super-resolution with denoising diffusion GANS

Single image super-resolution (SISR) refers to the reconstruction from the corresponding low-resolution (LR) image input to a high-resolution (HR) image. However, since a single low-resolution image corresponds to multiple high-resolution images, this is an ill-posed problem. In recent years, generative model-based SISR methods have outperformed conventional SISR methods in performance. However, the SISR methods based on GAN, VAE, and Flow have the problems of unstable training, low sampling quality, and expensive computational cost. These models also struggle to achieve the trifecta of diverse, high-quality, and fast sampling. In particular, denoising diffusion probabilistic models have shown impressive variety and high quality of samples, but their expensive sampling cost prevents them from being well applied in the real world. In this paper, we investigate the fundamental reason for the slow sampling speed of the SISR method based on the diffusion model lies in the Gaussian assumption used in the previous diffusion model, which is only applicable for small step sizes. We propose a new Single Image Super-Resolution with Denoising Diffusion GANS (SRDDGAN) to achieve large-step denoising, sample diversity, and training stability. Our approach combines denoising diffusion models with GANs to generate images conditionally, using a multimodal conditional GAN to model each denoising step. SRDDGAN outperforms existing diffusion model-based methods regarding PSNR and perceptual quality metrics, while the added latent variable Z solution explores the diversity of likely HR spatial domain. Notably, the SRDDGAN model infers nearly 11 times faster than diffusion-based SR3, making it a more practical solution for real-world applications.

Non-orthogonal optical multiplexing empowered by deep learning

Orthogonality among channels is a canonical basis for optical multiplexing featured with division multiplexing, which substantially reduce the complexity of signal post-processing in demultiplexing. However, it inevitably imposes an upper limit of capacity for multiplexing. Herein, we report on non-orthogonal optical multiplexing over a multimode fiber (MMF) leveraged by a deep neural network, termed speckle light field retrieval network (SLRnet), where it can learn the complicated mapping relation between multiple non-orthogonal input light field encoded with information and their corresponding single intensity output. As a proof-of-principle experimental demonstration, it is shown that the SLRnet can effectively solve the ill-posed problem of non-orthogonal optical multiplexing over an MMF, where multiple non-orthogonal input signals mediated by the same polarization, wavelength and spatial position can be explicitly retrieved utilizing a single-shot speckle output with fidelity as high as ~ 98%. Our results resemble an important step for harnessing non-orthogonal channels for high capacity optical multiplexing.

Spherical collapse in scalar-Gauss-Bonnet gravity: Taming ill-posedness with a Ricci coupling

We study spherical collapse of a scalar cloud in scalar-Gauss-Bonnet gravity—a theory in which black holes can develop scalar hair if they are in a certain mass range. We show that an additional quadratic coupling of the scalar field to the Ricci scalar can mitigate loss of hyperbolicity problems that have plagued previous numerical collapse studies and instead lead to well-posed evolution. This suggests that including specific additional interactions can be a successful strategy for tackling well-posedness problems in effective field theories of gravity with nonminimally coupled scalars. Our simulations also show that spherical collapse leads to black holes with scalar hair when their mass is below a mass threshold and above a minimum mass bound and that above the mass threshold, the collapse leads to black holes without hair, in line with results in the static case and perturbative analyses. For masses below the minimum mass bound, we find that the scalar cloud smoothly dissipates, leaving behind flat space. Published by the American Physical Society 2024

Semantic and Geometric-Aware Day-to-Night Image Translation Network.

Autonomous driving systems heavily depend on perception tasks for optimal performance. However, the prevailing datasets are primarily focused on scenarios with clear visibility (i.e., sunny and daytime). This concentration poses challenges in training deep-learning-based perception models for environments with adverse conditions (e.g., rainy and nighttime). In this paper, we propose an unsupervised network designed for the translation of images from day-to-night to solve the ill-posed problem of learning the mapping between domains with unpaired data. The proposed method involves extracting both semantic and geometric information from input images in the form of attention maps. We assume that the multi-task network can extract semantic and geometric information during the estimation of semantic segmentation and depth maps, respectively. The image-to-image translation network integrates the two distinct types of extracted information, employing them as spatial attention maps. We compare our method with related works both qualitatively and quantitatively. The proposed method shows both qualitative and qualitative improvements in visual presentation over related work.

Integrating L1 and weighted L2 regularization for moving force identification from combined response measurements

Existing regularization methods have been proven effective in the ill-posed problem of moving force identification (MFI). However, both the constant and time-varying components of moving forces cannot be accurately identified at the same time, especially under higher measurement noises. To address this issue, a new MFI method is proposed by integrating the L1 and weighted L2 regularization methods in this study, where the presented weighted L2 regularization strategy achieves a more nuanced balance of moving force components through the incorporation of weight coefficients. Furthermore, the inherent solving characteristics of L1 and weighted L2 regularization methods are fully utilized to solve different components in moving forces separately. Numerical simulations and experimental verifications are carried out to assess the effectiveness of the proposed method. The results show that the proposed method can accurately identify moving forces even under 25% higher noise levels and it can effectively evaluate the axle loads and gross vehicle weight of the experimental vehicle. Besides, comparative studies further show that the proposed method outperforms the existing L1 and L2 regularization methods in the identification of both constant and time-varying forces, which provides a potential MFI tool for a higher accuracy and noise robustness to fix the MFI problem in practice.

Sparse Regularization Based on Orthogonal Tensor Dictionary Learning for Inverse Problems

In seismic data processing, data recovery including reconstruction of the missing trace and removal of noise from the recorded data are the key steps in improving the signal-to-noise ratio (SNR). The reconstruction of seismic data and removal of noise becomes a sparse optimization problem that can be solved by using sparse regularization. Sparse regularization is a key tool in the solution of inverse problems. They are used to introduce prior knowledge and make the approximation of ill-posed inverses feasible. It deals with ill-posedness by replacing an ill-posed inverse problem with a well-posed problem that has a solution close to the true solution. In the last 2 decades, interest has shifted from linear toward nonlinear regularization methods even for linear inverse problems. In inverse problems, regularizations serve as stabilizing the solution of ill-posed inverse problems and give a solution that adequately fits measurements without producing unjustifiably complex artifacts. In this paper, we present a novel sparse regularization based on a tensor-based dictionary method for inverse problems (seismic data interpolation and denoising). This regularization avoids the vectorization step for sparse representation of seismic data during the reconstruction process. The key step in sparsifying signals is the choice of sparsity-promoting dictionary learning. The learning-based approach can adaptively sparsify datasets but has high computational complexity and involves no prior-constraint pattern information for the dataset. Many existing dictionary learning methods would be computationally infeasible for the high dimensional seismic data processing. These methods also destroy the essential information as well as it reduces the discriminability and expressibility of the signal, since they deal with vectorization. In this paper, the orthogonal tensor dictionary learning that learns a dictionary from the input data by employing orthogonality and separability is proposed as sparse regularization for the inverse problems. The performance of the proposed method was validated in seismic data interpolation and denoising individually as well as simultaneously. Numerical examples of synthetic and real seismic datasets demonstrate the validity of the proposed method. The SNR of the recovered data confirms that the proposed method is the most effective method than K-singular value decomposition and orthogonal dictionary learning methods.