Backward Derivatives Research Articles

Towards communication-efficient vertical federated learning training via cache-enabled local updates

Vertical federated learning (VFL) is an emerging paradigm that allows different parties (e.g., organizations or enterprises) to collaboratively build machine learning models with privacy protection. In the training phase, VFL only exchanges the intermediate statistics, i.e., forward activations and backward derivatives, across parties to compute model gradients. Nevertheless, due to its geo-distributed nature, VFL training usually suffers from the low WAN bandwidth. In this paper, we introduce CELU-VFL, a novel and efficient VFL training framework that exploits the local update technique to reduce the cross-party communication rounds. CELU-VFL caches the stale statistics and reuses them to estimate model gradients without exchanging the ad hoc statistics. Significant techniques are proposed to improve the convergence performance. First, to handle the stochastic variance problem, we propose a uniform sampling strategy to fairly choose the stale statistics for local updates. Second, to harness the errors brought by the stateness, we devise an instance weighting mechanism that measures the reliability of the estimated gradients. Theoretical analysis proves that CELU-VFL achieves a similar sub-linear convergence rate as vanilla VFL training but requires much fewer communication rounds. Empirical results on both public and real-world workloads validate that CELU-VFL can be up to six times faster than the existing works.

Standardize or Let a Thousand Flowers Bloom? Interface Design Coordination between Software Platforms and Hosted Apps

Software platform ecosystems are emerging as a dominant model for IT-based services. To accelerate third-party development, platform owners often waive restrictions on app user interfaces to provide app developers with more autonomy in interface design. The literature has indicated, however, that standardized interface design across products is a strategic necessity for firms to introduce and manage their product lines, suggesting that platform owners should not waive restrictions on app user interfaces. This tension between the two approaches raises a crucial research question: Does standardized interface design used by apps and their host platform benefit the platform and apps that it hosts? Specifically, does the coordinated design approach enhance or hinder usage between the platform and apps such that apps attract usage from their host platform and vice versa? Drawing on the theory of basic Gestalts, we theorize the three interface design properties of interface similarity, embeddedness, and synchrony, and extend the theory to investigate how these properties help both apps and their host platform attract each other’s users, thus reflecting derivative usage between them. We performed a randomized field experiment to test the proposed hypotheses. The results reveal that interface similarity, embeddedness, and synchrony of the app-platform interface design strengthen users’ grouping perception of the platform and its apps, thus enhancing platform-to-app forward derivative usage. Contrary to our expectations, we found that this strengthened grouping perception does not improve the app-to-platform backward derivative usage. These findings have crucial implications for the value of interface design in software platform ecosystems.

Systematic Convergence of the Numerical Taylor Series to the Best Standard and Its Potential Implication for the Development of Composite Methods

The numerical Taylor series is used as an alternative to systematically converge to a high-level calculation showing that it has the potential to be used in the development of composite methods. Five methods are tested, with two of them differing in the truncation of the series expansion and basis sets, namely, Taylor-Dunning and Taylor-Pople, and three of the interpretations of the G4, ccCA-TZ, and CBS-QB3, which are referred to as G4-Taylor, ccCA-TZ-Taylor, and CBS-QB3-Taylor, respectively. The Taylor-Dunning and Taylor-Pople methods restricted to second-order expansion show mean absolute errors of 1.16 and 1.11 kcal mol-1, respectively, for a training set involving enthalpies of formation, ionization energies, and electronic affinities. The G4-Taylor, ccCA-TZ-Taylor, and CBS-QB3-Taylor methods achieve the lowest mean absolute errors of 0.96, 1.54, and 0.58 kcal mol-1, respectively. The CBS-QB3-Taylor method is submitted to a validation step. The method using either forward or backward derivatives achieves a mean absolute error of 0.82 kcal mol-1. These results show that the numerical Taylor series and possibly other series expansions can be applied systematically for the development of accurate alternatives for composite methods.

Optimization for Software Implementation of Fractional Calculus Numerical Methods in an Embedded System.

In this article, some practical software optimization methods for implementations of fractional order backward difference, sum, and differintegral operator based on Grünwald–Letnikov definition are presented. These numerical algorithms are of great interest in the context of the evaluation of fractional-order differential equations in embedded systems, due to their more convenient form compared to Caputo and Riemann–Liouville definitions or Laplace transforms, based on the discrete convolution operation. A well-known difficulty relates to the non-locality of the operator, implying continually increasing numbers of processed samples, which may reach the limits of available memory or lead to exceeding the desired computation time. In the study presented here, several promising software optimization techniques were analyzed and tested in the evaluation of the variable fractional-order backward difference and derivative on two different Arm® Cortex®-M architectures. Reductions in computation times of up to 75% and 87% were achieved compared to the initial implementation, depending on the type of Arm® core.

Exact and Approximate Solutions of Fractional Partial Differential Equations for Water Movement in Soils

This paper presents solutions of the fractional partial differential equation (fPDE) for analysing water movement in soils. The fPDE explains processes equivalent to the concept of symmetrical fractional derivatives (SFDs) which have two components: the forward fractional derivative (FFD) and backward fractional derivative (BFD) of water movement in soils with the BFD representing the micro-scale backwater effect in porous media. The distributed-order time-space fPDE represents water movement in both swelling and non-swelling soils with mobile and immobile zones with the backwater effect operating at two time scales in large and small pores. The concept of flux-concentration relation is now updated to account for the relative fractional flux of water movement in soils.

The fractional Boussinesq equation of groundwater flow and its applications

This paper presents a set of fractional Boussinesq equations (fBEs) for groundwater flow in confined and unconfined aquifers and demonstrates the application of one of the fBEs for groundwater discharges known as recession curves. The fBEs are formulated with two-term distributed fractional orders in time and symmetrical fractional derivatives (SFD) in space applicable to both confined and unconfined aquifers. The SFD in theory consists of the forward fractional derivative (FFD) and the backward fractional derivative (BFD). The FFD represents the forward movement of water along the direction of mainstream flow while the BFD accounts for the backward motion of water in the direction opposite to the mainstream flow. The backward flow at the pore level can be referred to as the micro-scale backwater effect. The analogue of the backwater effect on a micro-scale using the BFD coincides with the wandering processes based on the continuous-time random walk (CTRW) theory which results in the fractional governing equation. With the analytical solutions of the fBE for given initial and boundary conditions of the first type for a finite depth, a set of formulae for groundwater recession has been derived using approximate solutions of the fBE. The examples of the applications of the recession curves are graphically illustrated and the effects of the orders of fractional derivatives on the geometry of the flow curves examined.

An examination of Professor Shaikh's proposal to tame Harrodian instability

This note reconsiders a recent proposal by A. Shaikh to tame Harrodian instability (Metroeconomica 2009), where besides the utilization gap investment depends on the expected growth of demand. His stability result has, however, been criticized as not credible. The crucial point is that Shaikh's continuous-time treatment does not distinguish between forward and backward derivatives. In order to check whether or not this poses a problem, several slight modifications of the model in continuous and discrete time are formulated and investigated for their stability. Roughly speaking, it is found that unless one assumes (myopic) perfect foresight, the destabilizing Harrodian mechanism continues to be effective.

Refined instrumental variable estimation: Maximum likelihood optimization of a unified Box–Jenkins model

For many years, various methods for the identification and estimation of parameters in linear, discrete-time transfer functions have been available and implemented in widely available Toolboxes for Matlab™. This paper considers a unified Refined Instrumental Variable (RIV) approach to the estimation of discrete and continuous-time transfer functions characterized by a unified operator that can be interpreted in terms of backward shift, derivative or delta operators. The estimation is based on the formulation of a pseudo-linear regression relationship involving optimal prefilters that is derived from an appropriately unified Box–Jenkins transfer function model. The paper shows that, contrary to apparently widely held beliefs, the iterative RIV algorithm provides a reliable solution to the maximum likelihood optimization equations for this class of Box–Jenkins transfer function models and so its en bloc or recursive parameter estimates are optimal in maximum likelihood, prediction error minimization and instrumental variable terms.

Non-linear partial derivative and its De Wolf approximation for non-linear seismic inversion

We demonstrate that higher order Fréchet derivatives are not negligible and the linear Fréchet derivative may not be appropriate in many cases, especially when forward scattering is involved for large-scale or strong perturbations (heterogeneities). We then introduce and derive the non-linear partial derivative including all the higher order Fréchet derivatives for the acoustic wave equation. We prove that the higher order Fréchet derivatives can be realized by consecutive applications of the scattering operator and a zero-order propagator to the source. The full non-linear partial derivative is directly related to the full scattering series. The formulation of the full non-linear derivative can be used to non-linearly update the model. It also provides a new way for deriving better approximations beyond the linear Fréchet derivative (Born approximation). In the second part of the paper, we derive the De Wolf approximation (DWA; multiple forescattering and single backscattering approximation) for the non-linear partial derivative. We split the linear derivative operator (i.e. the scattering operator) into forward and backward derivatives, and then reorder and renormalize the multiple scattering series before making the approximation of dropping the multiple backscattering terms. This approximation can be useful for both theoretical derivation and numerical calculation. Through both theoretical analyses and numerical simulations, we show that for large-scale perturbations, the errors of the linear Fréchet derivative (Born approximation) are significant and unacceptable. In contrast, the DWA non-linear partial derivative (NLPD), can give fairly accurate waveforms. Application of the NLPD to the least-square inversion leads to a different inversion algorithm than the standard gradient method.

Singular forward–backward stochastic differential equations and emissions derivatives

We introduce two simple models of forward-backward stochastic differential equations with a singular terminal condition and we explain how and why they appear naturally as models for the valuation of CO2 emission allowances. Single phase cap-and-trade schemes lead readily to terminal conditions given by indicator functions of the forward component, and using fine partial differential equations estimates, we show that the existence theory of these equations, as well as the properties of the candidates for solution, depend strongly upon the characteristics of the forward dynamics. Finally, we give a first order Taylor expansion and show how to numerically calibrate some of these models for the purpose of CO2 option pricing.

A new look into the discrete-time fractional calculus: derivatives and exponentials

A derivative based discrete-time signal processing is presented. Both nabla (forward) and delta (backward) derivatives are studied and generalised including the fractional case. The corresponding exponentials are introduced as eigenfunctions of such derivatives.

A new look into the discrete-time fractional calculus: transform and linear systems

A derivative based discrete-time signal processing is presented. Both nabla (forward) and delta (backward) derivatives are studied and generalised including the fractional case. The corresponding exponentials are introduced as eigenfunctions of such derivatives. These lead to discrete-time Laplace transforms that are used to define and study the linear discrete-time derivatives.

Discussion on: “Why Is Resorting to Fate Wise? A Critical Look at Randomized Algorithms in Systems and Control”

For many years, various methods for the identification and estimation of parameters in linear, discrete-time transfer functions have been available and implemented in widely available Toolboxes for Matlab™. This paper considers a unified Refined Instrumental Variable (RIV) approach to the estimation of discrete and continuous-time transfer functions characterized by a unified operator that can be interpreted in terms of backward shift, derivative or delta operators. The estimation is based on the formulation of a pseudo-linear regression relationship involving optimal prefilters that is derived from an appropriately unified Box–Jenkins transfer function model. The paper shows that, contrary to apparently widely held beliefs, the iterative RIV algorithm provides a reliable solution to the maximum likelihood optimization equations for this class of Box–Jenkins transfer function models and so its en bloc or recursive parameter estimates are optimal in maximum likelihood, prediction error minimization and instrumental variable terms.

Use of an automatic data quality control algorithm for crude oil viscosity data

When building correlations for physical properties, the data used must be quality controlled to ensure suitable performance of regression procedures and provide acceptable predictions. When using data from different sources and large datasets, this screening could become tedious and laborious unless a systematic and automated consistency check is used. For this study, we had a database of almost 3000 records of PVT properties and black-oil viscosity data coming from 324 differential liberation tests performed in different commercial laboratories. We developed a procedure to “clean up” the data on a test basis, before processing it with a regression routine. We individually screened each test, identified outlying observations and removed those from the regression calculations. The criteria used to discard data relied on the numerical evaluation of the first derivative of selected functions of one variable. If the physical behavior is predicted correctly, these functions should be monotonic. For example oil viscosity (observed function) should always increase as the pressure in the differential liberation tests decreases. Forward and backward derivatives were used to account for the end points. The filtered data resulting from this quality control process consisted of 2324 observations. The data were used to adapt two commonly used compositional viscosity models, Pedersen’s and Lohrenz, Bray and Clark (LBC), such that these models can be used for black-oil systems when compositional data are missing. The oil viscosity ranged from 0.13 to 78.3 cP, with pressure from 14.7 to 5602 psia (0.1–38.62 MPa) and temperature from 537 to 766 R (298–425.55 K). The oil specific gravity ranged from 0.389 to 0.921. These models were validated against an independent dataset consisting of 150 observations. The two models had lower statistical errors than currently available correlations.

Fermions in light front transverse lattice quantum chromodynamics

We briefly describe motivations for studying transverse lattice QCD. Presence of constraint equation for fermion field on the light front allows different methods to put fermions on a transverse lattice. We summarize our numerical investigation of two approaches using (a) forward and backward derivatives and (b) symmetric derivatives.

Can we use complex-valued fractional Brownian motion to derive a fractal space-time theory in micro-physics

Recently, a theory of fractal space-time has been proposed to unify the huge number of axioms of quantum mechanics, and one of the keys of this approach is the introduction of a complex-valued velocity which is a combination of the forward derivative (derivative on the right) with the backward derivative (derivative on the left) of the state co-ordinate vector. It is shown that if instead, one uses the classical velocity (forward derivative) but with complex-valued increments for the admissible variations of the state, it is possible to derive Schrödinger equation in exactly the same manner. The prospect of this alternative is to provide a generalization by using complex-valued Brownian motion.

A knowledge-based thinning algorithm

One common defect of thinning algorithms is deformation at crossing points. To solve this problem, a new thinning method, called the knowledge-based thinning algorithm (KBTA), is proposed. It first represents a binary pattern by coded run lengths of the horizontal line segments. Then the relationship between line segments is described quantitatively by another new algorithm which makes use of both forward and backward derivatives. It afterwards identifies the regions where branches of the pattern meet, then extracts their shape features and thins all of them. Knowing the identities of these regions, perfect skeletons can be obtained. Other regions are thinned by an existing algorithm which is based on contour generation. Experiments with a wide variety of binary patterns show that this new technique generates better skeletons than several other well-known algorithms.