Discrete Stochastic Approximation Research Articles

Idiographic Dynamic Modeling for Behavioral Interventions with Mixed Data Partitioning and Discrete Simultaneous Perturbation Stochastic Approximation.

This paper presents the use of discrete simultaneous perturbation stochastic approximation (DSPSA) as a routine method to efficiently determine features and parameters of idiographic (i.e. single subject) dynamic models for personalized behavioral interventions using various partitions of estimation and validation data. DSPSA is demonstrated as a valuable method to search over model features and regressor orders of AutoRegressive with eXogenous input estimated models using participant data from Just Walk (a behavioral intervention to promote physical activity in sedentary adults); results of DSPSA are compared to those of exhaustive search. In Just Walk, DSPSA efficiently and quickly estimates models of walking behavior, which can then be used to develop control systems to optimize the impacts of behavioral interventions. The use of DSPSA to evaluate models using various partitions of individual data into estimation and validation data sets also highlights data partitioning as an important feature of idiographic modeling that should be carefully considered.

Discrete Stochastic Optimization for Public Health Interventions with Constraints

Many public health threats exist, motivating the need to find optimal intervention strategies. Given the stochastic nature of the threats (e.g., the spread of pandemic influenza, the occurrence of drug overdoses, and the prevalence of alcohol-related threats), deterministic optimization approaches may be inappropriate. In this paper, we implement a stochastic optimization method to address aspects of the 2009 H1N1 and the COVID-19 pandemics, with the spread of disease modeled by the open-source Monte Carlo simulations, FluTE, and Covasim, respectively. Without testing every possible option, the objective of the optimization is to determine the best combination of intervention strategies so as to result in minimal economic loss to society. To reach our objective, this application-oriented paper uses the discrete simultaneous perturbation stochastic approximation method (DSPSA), a recursive simulation-based optimization algorithm, to update the input parameters in the disease simulation software so that the output iteratively approaches minimal economic loss. Assuming that the simulation models for the spread of disease (FluTE for H1N1 and Covasim for COVID-19 in our case) are accurate representations for the population being studied, the simulation-based strategy we present provides decision makers a powerful tool to mitigate potential human and economic losses from any epidemic. The basic approach is also applicable in other public health problems, such as opioid abuse and drunk driving.

Model Personalization in Behavioral Interventions using Model-on-Demand Estimation and Discrete Simultaneous Perturbation Stochastic Approximation.

This paper presents the use of discrete Simultaneous Perturbation Stochastic Approximation (DSPSA) to optimize dynamical models meaningful for personalized interventions in behavioral medicine, with emphasis on physical activity. DSPSA is used to determine an optimal set of model features and parameter values which would otherwise be chosen either through exhaustive search or be specified a priori. The modeling technique examined in this study is Model-on-Demand (MoD) estimation, which synergistically manages local and global modeling, and represents an appealing alternative to traditional approaches such as ARX estimation. The combination of DSPSA and MoD in behavioral medicine can provide individualized models for participant-specific interventions. MoD estimation, enhanced with a DSPSA search, can be formulated to provide not only better explanatory information about a participant's physical behavior but also predictive power, providing greater insight into environmental and mental states that may be most conducive for participants to benefit from the actions of the intervention. A case study from data collected from a representative participant of the Just Walk intervention is presented in support of these conclusions.

Discrete Time Multi-servers Loss Systems and Stochastic Approximation with Delayed Groups of Customers

The stochastic approximation procedure with delayed groups of delayed customers is investigated. The Robbins-Monro stochastic approximation procedure is adjusted to be usable in the presence of delayed groups of delayed customers. Two loss systems are introduced to get an accurate description of the proposed procedure. Each customer comes after fixed time-intervals with the stage of the following customer is accurate according to the outcome of the preceding one, where the serving time of a customer is assumed to be discrete random variable. Some applications of the procedure are given where the analysis of their results is obtained. The analysis shows that efficiencies of the procedure can be increased by minimizing the number of customers of a group irrespective of their service times that may take maximum values. Efficiencies depend on the maximum service time of the customer and on the number of customers of the group. The most important result is that efficiencies of the procedure are increased by increasing the service time distributions as well as service times of customers .This new situation can be applied to increase the number of served customers where the number of served groups will also be increased. The results obtained seem to be acceptable. In general, our proposal can be utilized to other stochastic approximation procedures to increase the production in many fields such as medicine, computer sciences, industry, and applied sciences.

Learning to cluster for rendering with many lights

We present an unbiased online Monte Carlo method for rendering with many lights. Our method adapts both the hierarchical light clustering and the sampling distribution to our collected samples. Designing such a method requires us to make clustering decisions under noisy observation, and making sure that the sampling distribution adapts to our target. Our method is based on two key ideas: a coarse-to-fine clustering scheme that can find good clustering configurations even with noisy samples, and a discrete stochastic successive approximation method that starts from a prior distribution and provably converges to a target distribution. We compare to other state-of-the-art light sampling methods, and show better results both numerically and visually.

Joint Relay Selection and Adaptive Modulation and Coding for Wireless Cooperative Communications

This paper considers a wireless cooperative communication system with the adaptive modulation and coding (AMC). We take the joint relay and AMC mode selection as two time-scales of optimization problems, which aim at maximizing the achievable data rate. We treat the best relay selection as a long-term objective and the best AMC mode selection as a short-term objective, to limit the signaling overhead for the wireless communication system. However, the evaluation of the achievable data rate for any of relay and AMC mode combinations needs the estimation of the wireless channel. To this end, we propose an online optimization-while-estimation algorithm based on discrete stochastic approximation to jointly solve both the relay and AMC selection problems. The algorithm requires only per-frame channel sampling and estimation in each algorithm iteration, and approaches the optimal achievable data rate asymptotically. Simulation results show that our proposed algorithm can quickly converge to the maximum achievable data rate.

High‐resolution optical‐to‐SAR image registration using mutual information and SPSA optimisation

The high-resolution optical and synthetic aperture radar (SAR) images are widely used in many remote sensing application areas such as image fusion and change detection where image registration is a fundamental step. The latest high-resolution optical and SAR satellites and airborne systems provide geometrically corrected images which do not contain global deformations. Though the images do not have global differences, still, registration differences exist between these optical and SAR images. These registration differences should be minimised through an automatic registration method before using the images for the aforementioned applications. However, an automatic optical-to-SAR image registration is a challenging task due to the presence of significant nonlinear intensity differences as well as local geometric distortions between the images. In order to solve these problems, an automatic optical-to-SAR image registration method is proposed which can effectively handle the registration differences between the globally corrected high-resolution images. In the proposed method, initially, a coarse registration is performed by using a discrete simultaneous perturbation stochastic approximation (SPSA) optimisation. Then, a smooth continuous SPSA optimisation is utilised for the fine registration of the images. Experiments are performed on six sets of high-resolution optical-SAR image pairs and the results show the effectiveness of the proposed method.

Stochastic time-dependent enzyme kinetics: Closed-form solution and transient bimodality.

We derive an approximate closed-form solution to the chemical master equation describing the Michaelis-Menten reaction mechanism of enzyme action. In particular, assuming that the probability of a complex dissociating into an enzyme and substrate is significantly larger than the probability of a product formation event, we obtain expressions for the time-dependent marginal probability distributions of the number of substrate and enzyme molecules. For delta function initial conditions, we show that the substrate distribution is either unimodal at all times or else becomes bimodal at intermediate times. This transient bimodality, which has no deterministic counterpart, manifests when the initial number of substrate molecules is much larger than the total number of enzyme molecules and if the frequency of enzyme-substrate binding events is large enough. Furthermore, we show that our closed-form solution is different from the solution of the chemical master equation reduced by means of the widely used discrete stochastic Michaelis-Menten approximation, where the propensity for substrate decay has a hyperbolic dependence on the number of substrate molecules. The differences arise because the latter does not take into account enzyme number fluctuations, while our approach includes them. We confirm by means of a stochastic simulation of all the elementary reaction steps in the Michaelis-Menten mechanism that our closed-form solution is accurate over a larger region of parameter space than that obtained using the discrete stochastic Michaelis-Menten approximation.

Reliable and Opportunistic Transmissions for Underwater Acoustic Networks

Acoustic waves propagate slowly in water, and time-varying UACs result in inevitably high bit error rate and packet loss rate. The long propagation delay and the error-prone nature of UACs impose challenges on reliable transmissions in UANs. In this article, we identify the challenges for reliable acoustic transmissions and propose a CL-FEC scheme, which achieves opportunistic transmissions to overcome the frequent transmission failures in UACs. CL-FEC adopts fountain codes as a packet-level FEC and adopts channel codes as a bit-level FEC, to realize reliable transmissions over UACs without per-packet feedback. To further improve the throughput of CL-FEC, we formulate the transmissions over UACs into a stochastic throughput optimization problem. A discrete stochastic approximation based algorithm is then developed to achieve the optimal CL-FEC by online exploiting channel estimating and algorithm iterations. Simulation results show the asymptotic convergence and the iterative optimality of the algorithm.

Two-Way MIMO Relaying Systems Employing Layered Relay-and-Antenna Selection Strategies

In this paper, a variety of layered relay-and-antenna selection (LRAS) strategies are developed for the two-way multiple-input multiple-output (MIMO) amplify-and-forward (AF) relaying systems in the presence of spatial correlation between antennas over fading channels. On the basis of the achievable sum-rate (ASR) maximization, the condition number (CN), adaptive discrete stochastic approximation (A-DSA), Gram–Schmidt (GS), and squared Frobenius norm (SFN)-based LRAS algorithms are investigated under an equal power allocation. To reduce the computational complexity of the relay-and-antenna selection (RAS) selection strategies, the best single relay and the subset of transmit-and-receive (Tx/Rx) antenna pairs are selected and separated into a two-stage manner. Remarkably, the LRAS techniques enable to alleviate substantially the computational complexity incurred by an exhaustive-mode RAS search algorithm. Finally, simulation results demonstrate the excellent sum-rate capacity of the proposed LRAS methods over other existing selection ones for MIMO two-way multiple-AF-relay systems.

Optimal Redundancy Control Strategy for Fountain Code-Based Underwater Acoustic Communication

In this paper, a transmission redundancy optimization strategy for fountain codes that uses an adaptive filtering technique as well as recent results concerning the upper bound of the source packet erasure probability for fountain codes over finite fields is proposed. The performance of the proposed strategy was evaluated in static and simulated realistic underwater acoustic channel environments simulated by a recently proposed statistical underwater acoustic channel model. The goal of this investigation is to improve the communication efficiency of underwater acoustic sensor networks that use communication protocols based on fountain codes. The performance of the proposed strategy is compared with the recently proposed approaches of discrete stochastic approximation (DSA) and the joint power and rate control (JPR) schemes. The simulation results demonstrate that the strategy proposed in this paper performs better than the existing schemes in both static and simulated realistic environments. The proposed strategy is 4.75 times better than the DSA in the static convergence performance evaluation. Further analysis shows that in the presence of a simulated realistic environment, compared to DSA and JPR, the strategy proposed in this paper reduces the overall transmission redundancy by approximately 11.4% and 52.6%, respectively. It also improves the average communication efficiency by 1.3% and 12.5%, respectively. The average optimization error relative to optimal redundancy is improved by approximately 62.1% and 92.5%, respectively. These numerical results are obtained by employing the specific simulation parameters and the simulated environment mentioned in this paper.

Interference Alignment in Virtualized Heterogeneous Cellular Networks With Imperfect Channel State Information

Wireless network virtualization is a promising technique for future wireless networks. In this paper, different from traditional virtualization approaches by means of resource isolation at the subchannel or time-slot level, we propose a novel framework of heterogeneous cellular network virtualization combined with interference alignment (IA) technology, which utilizes IA to cancel the mutual interference, by aligning the interference from other transmitters into a lower dimensional subspace at each receiver. In this framework, we formulate the virtual resource allocation as a joint virtualization and IA problem, considering the gain not only from interference mitigation introduced by IA but from the sum-rate improvement brought by virtualization as well. In addition, to reduce the computational complexity, with the recent advances in discrete stochastic approximation (DSA), we propose a two-step algorithm to solve the formulated problem. The basic principle is to design IA schemes for each feasible association combination and then to traverse the association space to search for the optimal association combination with the maximum sum rate. Extensive simulations are conducted with different system parameters to show the effectiveness of the proposed scheme.

Virtual Resource Allocation in Software-Defined Information-Centric Cellular Networks With Device-to-Device Communications and Imperfect CSI

In this paper, we propose an architecture of software-defined information-centric network virtualization with device-to-device (D2D) communications, which facilitates dynamic virtual resource allocation and content caching via a software-defined networking (SDN) controller with a global view of the system. In our proposed framework, substrate physical resources can be virtualized and shared among multiple mobile virtual network operators (MVNOs). Meanwhile, by means of integrating D2D communications into information-centric wireless networks, content caching is enabled not only in the air but in mobile devices as well. In addition, taking into consideration inaccurate channel estimation and measurement, we formulate the virtual resource allocation and caching optimization as a discrete stochastic optimization problem in which imperfect channel state information is incorporated. Because the formulated virtual resource allocation problem is a large-scale combinational optimization problem, we exploit discrete stochastic approximation approaches to cope with it. Finally, extensive simulations are conducted to demonstrate the effectiveness of the proposed scheme with different system parameters. Simulation results show that MVNOs can benefit from not only the sharing of physical infrastructure but from the caching functionality that exists in both the air and mobile devices as well.

On Monotonicity of the Optimal Transmission Policy in Cross-Layer Adaptive <inline-formula> <tex-math notation="LaTeX">$m$ </tex-math> </inline-formula>-QAM Modulation

This paper considers a cross-layer adaptive modulation system that is modeled as a Markov decision process. We study how to utilize the monotonicity of the optimal transmission policy to relieve the computational complexity of dynamic programming (DP). In this system, a scheduler controls the bit rate of the $m$ -quadrature amplitude modulation in order to minimize the long-term losses incurred by the queue overflow in the data link layer and the transmission power consumption in the physical layer. The work is done in two steps. First, we observe the $ L^{\natural }$ -convexity and submodularity of DP to prove that the optimal policy is always nondecreasing in queue occupancy/state and derive the sufficient condition for it to be nondecreasing in both queue and channel states. We also show that, due to the $ L^{\natural }$ -convexity of DP, the variation of the optimal policy in queue state is restricted by a bounded marginal effect. The increment of the optimal policy between adjacent queue states is no greater than one. Second, we use the monotonicity results to present two low complexity algorithms: monotonic policy iteration (MPI) based on $ L^{\natural }$ -convexity and discrete simultaneous perturbation stochastic approximation (DSPSA). We run experiments to show that the time complexity of MPI based on $ L^{\natural }$ -convexity is much lower than that of DP and the conventional MPI that is based on submodularity and DSPSA is able to adaptively track the optimal policy when the system parameters change.

Software-Defined Device-to-Device (D2D) Communications in Virtual Wireless Networks With Imperfect Network State Information (NSI)

Software-defined networking (SDN) and network function virtualization (NFV) are a promising system architecture and control mechanism for future networks. Although some works have been done on wireless SDN and NFV, recent advancements in device-to-device (D2D) communications are largely ignored in this novel framework. In this paper, we study the integration of D2D communication in the framework of SDN and NFV. An inherent challenge in supporting software-defined D2D is the imperfectness of network state information, including channel state information (CSI) and queuing state information, in virtual wireless (QSI) networks. To address this challenge, we formulate the resource sharing problem in this framework as a discrete stochastic optimization problem and develop discrete stochastic approximation algorithms to solve this problem. Such algorithms can reduce the computational complexity compared with exhaustive search while achieving satisfactory performance. Both the static wireless channel and time-varying channels are considered. Extensive simulations show that users can benefit from both wireless network virtualization and software-defined D2D communications, and our proposed scheme can achieve considerable performance gains in both system throughput and user utility under practical network settings.

Achievable sum‐rate maximization using relay/antenna selection for MIMO two‐way AF relaying with channel uncertainties

SummaryThis paper focuses on the layered relay‐and‐antenna selection (LRAS) for achievable sum‐rate (ASR) maximization while considering the impacts of residual self‐interference due to channel estimation errors in multiple‐input multiple‐output two‐way amplify‐and‐forward relaying systems. Two LRAS algorithms, namely, the Gram–Schmidt and the adaptive discrete stochastic approximation selection techniques, are investigated based on the ASR maximization under an equal power allocation. To alleviate the complexity burden of the LRAS strategies, the optimal relay and the subset of transmit‐and‐receive antenna pairs are determined by a two‐stage selection mechanism. By taking two LRAS strategies and correlated channel uncertainties into account, the development of a two‐way multiple‐input multiple‐output multi‐amplify‐and‐forward‐relay system is able to provide improved robustness against the channel state information mismatch and the residual self‐interference. Copyright © 2016 John Wiley & Sons, Ltd.

Resource Optimization-Based Interference Management for Hybrid Self-Organized Small-Cell Network

This paper jointly studies interference management (IM) and self-organization (SO) schemes in a hybrid-SO small-cell network, which is a combination of the centralized and distributed small-cell networks. From the view of downlink transmission, we formulate a resource optimization-based IM (ROIM) problem. In particular, to accommodate to the hybrid architecture, the problem is divided into two subproblems. The first is subchannel allocation, which is investigated with overall network interference alleviation under the assumption of imperfect channel-state information (CSI). The second is power assignment (PA), which aims at constraining the local interference while guaranteeing the network utility. To solve the ROIM problem, we propose a two-step SO-based IM (TSSOIM) scheme. At the first step, the operation and management (O&M) unit adopts an aggressive discrete stochastic approximation (ADSA) algorithm to find the proper channel set for users. At the second step, small cells employ the game-theory-based Kalai–Smorodinsky bargaining solution (KSBS) to assign power to the channels. Simulation results are presented to show the effectiveness of the proposed TSSOIM scheme.

Discrete Convexity and Stochastic Approximation for Cross-layer On– off Transmission Control

This paper considers the discrete convexity of a cross-layer on – off transmission control problem in wireless communications. In this system, a scheduler decides whether or not to transmit in order to optimize the long-term quality of service (QoS) incurred by the queueing effects in the data link layer and the transmission power consumption in the physical (PHY) layer simultaneously. Using a Markov decision process (MDP) formulation, we show that the optimal policy can be determined by solving a minimization problem over a set of queue thresholds if the dynamic programming (DP) is submodular. We prove that this minimization problem is discrete convex. In order to search the minimizer, we consider two discrete stochastic approximation (DSA) algorithms: 1) discrete simultaneous perturbation stochastic approximation (DSPSA) and 2) ${\bm{L}}^{\natural}$ -convex stochastic approximation ( ${\bm{L}}^{\natural}$ -convex SA). Through numerical studies, we show that the two DSA algorithms converge significantly faster than the existing continuous simultaneous perturbation stochastic approximation (CSPSA) algorithm in multiuser systems. Finally, we compare the convergence results and complexity of two DSA and CSPSA algorithms where we show that DSPSA achieves the best tradeoff between complexity and accuracy in multiuser systems.

Forecasting the economy development based on a stochastic model of economic growth given a turning point

The article notes the increasing infl uence of uncertainty and risk factors on the economic development in modern conditions. Some opportunities to consider this impact are connected with the use of stochastic models for economic growth. The authors study the methods of forecasting the economic development based on a discrete stochastic approximation for the constraints of the economic growth model using the Euler-Maruyama method. The proposed approach permits constructing the mean calculated economic growth trajectory starting from the current (initial) state. The method implementation in the simulation mode is presented with regard to the turning point induced by the 2008 global economic crisis. Special attention is drawn to the characteristic features of the proposed method and the problem of the economic growth stochastic model calibration. Th e calculations of GDP and consumption expenditure trajectories are performed according to the data for Greece, Denmark, and Spain in two time periods — before and aft er the 2008 crisis. Th e sets of model parameters are foundthat provide the compliance of the actual trajectories with 50% confi dence intervals of calculated values of the macroeconomic indicators under consideration. It is shown that the use of the baseline data defi ned in fi xed 1970 prices for the forecast calculations can increase the compliance. Refs 36. Figs 12. Tables 12.

Structured optimal transmission control in network-coded two-way relay channels

This paper considers a transmission control problem in network-coded two-way relay channels (NC-TWRC), where the relay buffers randomly arrived packets from two users, and the channels are assumed to be fading. The problem is modeled by a discounted infinite horizon Markov decision process (MDP). The objective is to find an adaptive transmission control policy that minimizes the packet delay, buffer overflow, transmission power consumption and downlink error rate simultaneously and in the long run. By using the concepts of submodularity, multimodularity and L ♮ -convexity, we study the structure of the optimal policy searched by dynamic programming (DP) algorithm. We show that the optimal transmission policy is nondecreasing in queue occupancies and/or channel states under certain conditions such as the chosen values of parameters in the MDP model, channel modeling method, and the preservation of stochastic dominance in the transitions of system states. Based one these results, we propose to use two low-complexity algorithms for searching the optimal monotonic policy: monotonic policy iteration (MPI) and discrete simultaneous perturbation stochastic approximation (DSPSA). We show that MPI reduces the time complexity of DP, and DSPSA is able to adaptively track the optimal policy when the statistics of the packet arrival processes change with time.