Forward-backward Algorithm Research Articles

A two-metric variable scaled forward-backward algorithm for $$\ell _0$$ optimization problem and its applications

A two-metric variable scaled forward-backward algorithm for $$\ell _0$$ optimization problem and its applications

Stochastic Precipitation Generation for the Xilingol League Using Hidden Markov Models with Variational Bayes Parameter Estimation

Precipitation modeling holds significant importance in various fields such as agriculture, animal husbandry, weather derivatives, hydrology, and risk and disaster preparedness. Stochastic precipitation generators (SPGs) represent a class of statistical models designed to generate synthetic data capable of simulating dry and wet precipitation stretches for a long duration. The construction of Hidden Markov Models (HMMs), which treat latent meteorological circumstances as hidden states, is an efficient technique for simulating precipitation. Considering that there are many choices of emission distributions used to generate positive precipitation, the characteristics of different distributions for simulating positive precipitation have not been fully explored. The paper includes a simulation study that demonstrates how the Pareto distribution, when used as the distribution for generating positive precipitation, addresses the limitations of the exponential and gamma distributions in predicting heavy precipitation events. Additionally, the Pareto distribution offers flexibility through adjustable parameters, making it a promising option for precipitation modeling. We can estimate parameters in HMMs using forward–backward algorithms, Variational Bayes Expectation-Maximization (VBEM), and Stochastic Variational Bayes (SVB). In the Xilingol League, located in the central part of the Inner Mongolia Autonomous Region, China, our study involved data analysis to identify crucial locations demonstrating a robust correlation and notable partial correlation between the Normalized Difference Vegetation Index (NDVI) and annual precipitation. We performed fitting of monthly dry days ratios and monthly precipitation using seasonal precipitation and year-round precipitation data at these crucial locations. Subsequently, we conducted precipitation predictions for the daily, monthly, and annual time frames using the new test dataset observations. The study concludes that the SPG fits the monthly dry-day ratio better for annual daily precipitation data than for seasonal daily precipitation data. The fitting error for the monthly dry day ratio corresponding to annual daily precipitation data is 0.053 (exponential distribution) and 0.066 (Pareto distribution), while for seasonal daily precipitation data, the fitting error is 0.14 (exponential distribution) and 0.15 (Pareto distribution). The exponential distribution exhibits the poorest performance as a model for predicting future precipitation, with average errors of 2.49 (daily precipitation), 40.62 (monthly precipitation), and 130.40 (annual precipitation). On the other hand, the Pareto distribution demonstrates the best overall predictive performance, with average errors of 0.69 (daily precipitation), 34.69 (monthly precipitation), and 66.42 (annual precipitation). The results of this paper can provide decision support for future grazing strategies in the Xilingol League.

On the definitions of hidden Markov models

At least three probabilistic definitions of hidden Markov models (HMMs) have been used frequently in the literature. Unfortunately, one of these definitions shows fatal flaws, however nowadays a lot of literature still uses this definition. The aim of this paper is on one hand to specifically point out one such fatal flaw (in terms of deriving the well-known forward-backward algorithm), and on the other hand to list key properties of HMMs under the other two plausible (and equivalent) probabilistic definitions for further developments. As applications, we rigorously layout forward-backward algorithms for inhomogeneous HMMs and hidden reciprocal models, and fully present connections between HMMs and undirected graphical models which are not mentioned anywhere in the literature, to the best of our knowledge.

A Hidden Semi-Markov Model for Predicting Production Cycle Time Using Bluetooth Low Energy Data

This study proposes a statistical model to characterize the temporal characteristics of an entire production process. The model utilizes received signal strength indicator (RSSI) data obtained from a Bluetooth low energy (BLE) network. A hidden semi-Markov model (HSMM) is formulated based on the characteristics of the production process, and the forward-backward algorithm is employed to re-estimate the probability distribution of state durations. The proposed method is validated through numerical, simulation, and real-world experiments, yielding promising results. The results show that the Kullback-Leibler divergence (KLD) score of 0.1843, while the simulation achieves an average vector distance score of 0.9740. The real-time experiment also shows a reasonable accuracy, with an average HSMM estimated throughput time of 30.48 epochs, compared to the average real throughput time of 33.99 epochs. Overall, the model serves as a valuable tool for predicting the cycle time and throughput time of a production line.

Convergence analysis of the generalized viscosity explicit method for solving variational inclusion problems in Banach spaces with applications

In this paper, we develop the viscosity explicit method for finding a solution of the variational inclusion problem, which is a zero of the sum of two accretive operators, in Banach spaces which integrates the algorithm defined by Cholamjiak, Pholasa, Suantai, and Sunthrayuth [The generalized viscosity explicit rules for solving variational inclusion problems in Banach spaces. Optimization. 2021;70(12):2607–2633. doi: 10.1080/02331934.2020.1789131] and the algorithm defined by Wang, Wang, and Zhang [Strong convergence of viscosity forward-backward algorithm to the sum of two accretive operators in Banach space. Optimization. 2021;70(1):169–190. doi: 10.1080/02331934.2019.1705299]. Also, we provide a new suitable assumption to prove the strong convergence theorem. We apply our main result to the variational inequality problem, the convex minimization problem, and the split feasibility problem. Numerical examples are given to illustrate the performance of the proposed method in the comparison with the related methods.

Optimal control of a spatiotemporal [formula omitted] model with reaction–diffusion involving [formula omitted]-Laplacian operator

The present paper’s primary goal is to make an analysis and study a reaction–diffusion SIR epidemic mathematical model expressed as a parabolic system of partial differential equations using the p-Laplacian operator. Immunity is compelled through vaccination distribution, which is seen as a control variable. Our main goal is to define an optimal control, which reduces the spread of infection and the cost of vaccination over a limited period of time and space. Existence and uniqueness of a positive solution and existence of an optimal control for the proposed model are proved. Then a description and characterization of the optimal control is provided in terms of state and adjoint functions. Optimality system is numerically resolved by a discrete iterative scheme pertained to and the forward–backward algorithm. Furthermore, using various p-values for the p-Laplacian operator, numerical results demonstrate the effectiveness of the suggested control strategy, which yields meaningful outcomes.

Deriving Vocal Fold Oscillation Information from Recorded Voice Signals Using Models of Phonation.

During phonation, the vocal folds exhibit a self-sustained oscillatory motion, which is influenced by the physical properties of the speaker's vocal folds and driven by the balance of bio-mechanical and aerodynamic forces across the glottis. Subtle changes in the speaker's physical state can affect voice production and alter these oscillatory patterns. Measuring these can be valuable in developing computational tools that analyze voice to infer the speaker's state. Traditionally, vocal fold oscillations (VFOs) are measured directly using physical devices in clinical settings. In this paper, we propose a novel analysis-by-synthesis approach that allows us to infer the VFOs directly from recorded speech signals on an individualized, speaker-by-speaker basis. The approach, called the ADLES-VFT algorithm, is proposed in the context of a joint model that combines a phonation model (with a glottal flow waveform as the output) and a vocal tract acoustic wave propagation model such that the output of the joint model is an estimated waveform. The ADLES-VFT algorithm is a forward-backward algorithm which minimizes the error between the recorded waveform and the output of this joint model to estimate its parameters. Once estimated, these parameter values are used in conjunction with a phonation model to obtain its solutions. Since the parameters correlate with the physical properties of the vocal folds of the speaker, model solutions obtained using them represent the individualized VFOs for each speaker. The approach is flexible and can be applied to various phonation models. In addition to presenting the methodology, we show how the VFOs can be quantified from a dynamical systems perspective for classification purposes. Mathematical derivations are provided in an appendix for better readability.

Improving the Thresholds of Generalized LDPC Codes With Convolutional Code Constraints

CC-GLPDC codes are a class of generalized low-density parity-check (GLDPC) codes where the constraint nodes (CNs) represent convolutional codes. This allows for efficient decoding in the trellis with the forward-backward algorithm, and the strength of the component codes easily can be controlled by the encoder memory without changing the graph structure. In this letter, we extend the class of CC-GLDPC codes by introducing different types of irregularity at the CNs and investigating their effect on the BP and MAP decoding thresholds for the binary erasure channel (BEC). For the considered class of codes, an exhaustive grid search is performed to find the BP-optimized and MAP-optimized ensembles and compare their thresholds with the regular ensemble of the same design rate. The results show that irregularity can significantly improve the BP thresholds, whereas the thresholds of the MAP-optimized ensembles are only slightly different from the regular ensembles. Simulation results for the AWGN channel are presented as well and compared to the corresponding thresholds.

Massive Parallelization of Massive Sample-Size Survival Analysis

Large-scale observational health databases are increasingly popular for conducting comparative effectiveness and safety studies of medical products. However, increasing number of patients poses computational challenges when fitting survival regression models in such studies. In this article, we use Graphics Processing Units (GPUs) to parallelize the computational bottlenecks of massive sample-size survival analyses. Specifically, we develop and apply time- and memory-efficient single-pass parallel scan algorithms for Cox proportional hazards models and forward-backward parallel scan algorithms for Fine-Gray models for analysis with and without a competing risk using a cyclic coordinate descent optimization approach. We demonstrate that GPUs accelerate the computation of fitting these complex models in large databases by orders of magnitude as compared to traditional multi-core CPU parallelism. Our implementation enables efficient large-scale observational studies involving millions of patients and thousands of patient characteristics. The above implementation is available in the open-source R package Cyclops.

Hierarchical nuclear norm penalization for multi-view data integration.

The prevalence of data collected on the same set of samples from multiple sources (i.e., multi-view data) has prompted significant development of data integration methods based on low-rank matrix factorizations. These methods decompose signal matrices from each view into the sum of shared and individual structures, which are further used for dimension reduction, exploratory analyses, and quantifying associations across views. However, existing methods have limitations in modeling partially-shared structures due to either too restrictive models, or restrictive identifiability conditions. To address these challenges, we propose a new formulation for signal structures that include partially-shared signals based on grouping the views into so-called hierarchical levels with identifiable guarantees under suitable conditions. The proposed hierarchy leads us to introduce a new penalty, hierarchical nuclear norm (HNN), for signal estimation. In contrast to existing methods, HNN penalization avoids scores and loadings factorization of the signals and leads to a convex optimization problem, which we solve using a dual forward-backward algorithm. We propose a simple refitting procedure to adjust the penalization bias and develop an adapted version of bi-cross-validation for selecting tuning parameters. Extensive simulation studies and analysis of the genotype-tissue expression data demonstrate the advantages of our method over existingalternatives.

Single-particle reconstruction in cryo-EM based on three-dimensional weighted nuclear norm minimization

Single-particle reconstruction (SPR) in cryogenic electron microscopy (cryo-EM) aims at aligning and averaging two-dimensional micrographs to reconstruct a three-dimensional particle.How to reconstruct micrographs from heavy noise is a crucial point for achieving better micrograph quality, and thus many methods focus on noise removal. However, new problems such as over-smoothing often occur in their results due to failure in handling heavy noise well. This paper proposes a three-dimensional weighted nuclear norm minimization (3DWNNM) model for SPR in the cryo-EM task to address these issues. Specifically, we design a minimization solver based on the forward-backward splitting algorithm to tackle our model efficiently. Under certain conditions, this solution has an energy-decaying feature and performs exceptionally well in reconstruction. Numerical experiments fully demonstrate the effectiveness and the robustness of the proposed method.

Collaborative passenger flow control for an urban rail transit network

AbstractAmid worsening passenger congestion, passenger flow control has become a major need in urban rail transit. However, existing passenger flow control strategies are fixed and do not consider the impacts of the spatial and temporal variations in passenger flow. To address the problems mentioned, a method for collaborative passenger flow control in urban rail transit is proposed. First, the temporal and spatial distribution of passengers in the network is obtained. Then, a model of collaborative passenger flow control for urban rail transit is built to reduce the number of controlled source points and controlled passengers and minimize the difference between the expected number of passengers on bottleneck links and the number of passengers with control. Next, a multilevel network that considers the time dimension is established based on the model. A forward–backward algorithm is introduced to make full use of the adaptive learning rate to solve the model. A case study based on a small‐scale network shows that the forward–backward algorithm has good convergence. To verify the effectiveness of the method, the passenger flow of Chengdu Metro during COVID‐19 is analyzed. The objective function of the forward–backward algorithm is 44% lower than that of the gradient descent method and 36% lower than that of adaptive moment estimate (ADAM). Its computational speed is also acceptable. The results show that compared with the fixed passenger flow control strategy of metro operators, the obtained strategy is more effective in reducing the number of controlled passengers and source points and alleviating congestion of bottleneck links.

Convergence rate of inertial forward–backward algorithms based on the local error bound condition

Abstract The ‘inertial forward–backward algorithm’ (IFB) is a powerful algorithm for solving a class of convex non-smooth minimization problems, IFB relies on an inertial parameter $\gamma _{k}$ whose tuning is crucial for achieving accelerated convergence speeds as compared to the classical forward–backward algorithm. Under the local error bound condition, it is known that IFB converges R-linearly as soon as the inertial parameter satisfies ${\sup _{k}}{\gamma _{k}} \leqslant \tilde{\gamma } <1.$ On the contrary, we are not aware of any convergence result for the case ${\sup _{k}}{\gamma _{k}} = 1.$ In this paper, we consider six different choices of inertial parameters satisfying this last condition, and show convergence of the corresponding IFB algorithms under the local error bound condition. Finally, we propose a class of inertial forward–backward algorithm with an adaptive modification (IFB_AdapM) and show that it enjoys the same convergence results.

Convergence of an asynchronous block-coordinate forward-backward algorithm for convex composite optimization

AbstractIn this paper, we study the convergence properties of a randomized block-coordinate descent algorithm for the minimization of a composite convex objective function, where the block-coordinates are updated asynchronously and randomly according to an arbitrary probability distribution. We prove that the iterates generated by the algorithm form a stochastic quasi-Fejér sequence and thus converge almost surely to a minimizer of the objective function. Moreover, we prove a general sublinear rate of convergence in expectation for the function values and a linear rate of convergence in expectation under an error bound condition of Tseng type. Under the same condition strong convergence of the iterates is provided as well as their linear convergence rate.

Scalable precision wide-field imaging in radio interferometry – II. AIRI validated on ASKAP data

ABSTRACT Accompanying Part I, this sequel delineates a validation of the recently proposed AI for Regularization in radio-interferometric Imaging (AIRI) algorithm on observations from the Australian Square Kilometre Array Pathfinder (ASKAP). The monochromatic AIRI-ASKAP images showcased in this work are formed using the same parallelized and automated imaging framework described in Part I: ‘uSARA validated on ASKAP data’. Using a Plug-and-Play approach, AIRI differs from uSARA by substituting a trained denoising deep neural network (DNN) for the proximal operator in the regularization step of the forward–backward algorithm during deconvolution. We build a trained shelf of DNN denoisers that target the estimated image dynamic ranges of our selected data. Furthermore, we quantify variations of AIRI reconstructions when selecting the nearest DNN on the shelf versus using a universal DNN with the highest dynamic range, opening the door to a more complete framework that not only delivers image estimation but also quantifies epistemic model uncertainty. We continue our comparative analysis of source structure, diffuse flux measurements, and spectral index maps of selected target sources as imaged by AIRI and the algorithms in Part I – uSARA and WSClean. Overall, we see an improvement over uSARA and WSClean in the reconstruction of diffuse components in AIRI images. The scientific potential delivered by AIRI is evident in further imaging precision, more accurate spectral index maps, and a significant acceleration in deconvolution time, whereby AIRI is four times faster than its subiterative sparsity-based counterpart uSARA.

Fast Track Smoothing Algorithm for Underwater Target Tracking Based on KD Tree

A fast-track smoothing algorithm based on the KD Tree is proposed to address the problem of state estimation in underwater multi-target tracking. For multi-target tracking, the multi-Bernoulli filter becomes intractable due to the high-dimensional nature of the posterior. This problem can be overcome by the particle filter (PF), which approximates the posterior by a set of weighted samples (“particles”). The probability distribution of the multi-Bernoulli algorithm based on this algorithm contains a series of particles with weights. Therefore, while smoothing it, the state transfer probability of each particle must be computed, and the state transfer function must be calculated, which is related to the difference in states between the k-1 moment and the k moment, we refer to this difference as the state distance. However, the KD Tree can be used to divide the particle set into different state spaces according to the particle states, and thus calculate the state distances more efficiently. The remarkable feature of this algorithm is that the time complexity is greatly reduced from that O(N2) of the traditional algorithm to O(NlogN). This paper implements the proposed method using particles, the improved algorithm based on KD tree effectively mitigates the disadvantages of previous forward-backward smoothing algorithms, and significantly reduces the computation time while improving the system state estimation precision. Experimental results of simulation analysis and the optimal sub-patten assignment distance is employed as an assessment criterion to validate the algorithm’s performance.

A Forward–Backward Algorithm With Different Inertial Terms for Structured Non-Convex Minimization Problems

A Forward–Backward Algorithm With Different Inertial Terms for Structured Non-Convex Minimization Problems

Apply Graph Signal Processing on NILM: An Unsupervised Approach Featuring Power Sequences.

As a low-cost demand-side management application, non-intrusive load monitoring (NILM) offers feedback on appliance-level electricity usage without extra sensors. NILM is defined as disaggregating loads only from aggregate power measurements through analytical tools. Although low-rate NILM tasks have been conducted by unsupervised approaches based on graph signal processing (GSP) concepts, enhancing feature selection can still contribute to performance improvement. Therefore, a novel unsupervised GSP-based NILM approach with power sequence feature (STS-UGSP) is proposed in this paper. First, state transition sequences (STS) are extracted from power readings and featured in clustering and matching, instead of power changes and steady-state power sequences featured in other GSP-based NILM works. When generating graph in clustering, dynamic time warping distances between STSs are calculated for similarity quantification. After clustering, a forward-backward power STS matching algorithm is proposed for searching each STS pair of an operational cycle, utilizing both power and time information. Finally, load disaggregation results are obtained based on STS clustering and matching results. STS-UGSP is validated on three publicly accessible datasets from various regions, generally outperforming four benchmarks in two evaluation metrics. Besides, STS-UGSP estimates closer energy consumption of appliances to the ground truth than benchmarks.

Combining mixed effects hidden Markov models with latent alternating recurrent event processes to model diurnal active-rest cycles.

Data collected from wearable devices can shed light on an individual's pattern of behavioral and circadian routine. Phone use can be modeled as alternating processes, between the state of active use and the state of being idle. Markov chains and alternating recurrent event models are commonly used to model state transitions in cases such as these, and the incorporation of random effects can be used to introduce diurnal effects. While state labels can be derived prior to modeling dynamics, this approach omits informative regression covariates that can influence state memberships. We instead propose an alternating recurrent event proportional hazards (PH) regression to model the transitions between latent states. We propose an expectation-maximization algorithm for imputing latent state labels and estimating parameters. We show that our E-step simplifies to the hidden Markov model (HMM) forward-backward algorithm, allowing us to recover an HMM with logistic regression transition probabilities. In addition, we show that PH modeling of discrete-time transitions implicitly penalizes the logistic regression likelihood and results in shrinkage estimators for the relative risk. This new estimator favors an extended stay in a state and is useful for modeling diurnal rhythms. We derive asymptotic distributions for our parameter estimates and compare our approach against competing methods through simulation as well as in a digital phenotyping study that followed smartphone use in a cohort of adolescents with mooddisorders.

Stochastic variable metric proximal gradient with variance reduction for non-convex composite optimization

This paper introduces a novel algorithm, the Perturbed Proximal Preconditioned SPIDER algorithm (3P-SPIDER), designed to solve finite sum non-convex composite optimization. It is a stochastic Variable Metric Forward–Backward algorithm, which allows approximate preconditioned forward operator and uses a variable metric proximity operator as the backward operator; it also proposes a mini-batch strategy with variance reduction to address the finite sum setting. We show that 3P-SPIDER extends some Stochastic preconditioned Gradient Descent-based algorithms and some Incremental Expectation Maximization algorithms to composite optimization and to the case the forward operator can not be computed in closed form. We also provide an explicit control of convergence in expectation of 3P-SPIDER, and study its complexity in order to satisfy the approximate epsilon-stationary condition. Our results are the first to combine the non-convex composite optimization setting, a variance reduction technique to tackle the finite sum setting by using a minibatch strategy and, to allow deterministic or random approximations of the preconditioned forward operator. Finally, through an application to inference in a logistic regression model with random effects, we numerically compare 3P-SPIDER to other stochastic forward–backward algorithms and discuss the role of some design parameters of 3P-SPIDER.