Gradient-based Algorithm Research Articles

Iterative parameter identification algorithms for transformed dynamic rational fraction input–output systems

The rational fraction system is a special nonlinear system, the existence of the denominator polynomial leads to the difficulty of identifying rational fraction models. Inspired by the gradient search and the Newton method, the gradient-based iterative algorithm and Newton iterative algorithm are presented to estimate the parameters of rational fraction system models. Furthermore, in order to avoid a large amount of calculation and complex equations encountered in the process of solving partial derivatives, the model transformation-based gradient iterative algorithm and the model transformation-based Newton iterative algorithm are proposed for parameter identification. Two examples are carried out to show the effectiveness of the proposed algorithms. This paper focuses on solving the identification problem of rational fraction systems.

Cell-Free UAV Networks: Asymptotic Analysis and Deployment Optimization

Recently, cell-free (CF) architectures, in which every user can potentially communicate with every base station, have received a lot of attention. This paper considers the uplink of fully and partially centralized CF networks where unmanned aerial vehicles serve as flying base stations (FBSs). A subset of FBSs participates in the reception of each user and a subset of users is received by each FBS. Deterministic equivalent expressions, exact asymptotically in the subset sizes and approximate for finite dimensions thereof, are derived for the spectral efficiency under Rician fading. Capitalizing on these expressions, the FBS deployment problem is investigated for different receiver architectures. The nonconvex deployment problem, tackled through a combination of gradient-based and Gibbs sampling algorithms, results in a superior performance with respect to a square grid deployment; this superiority extends to the minimum and aggregate spectral efficiency for both fully and partially centralized cell-free networks.

Design and Optimization of Metamaterial-Based Dual-Band 28/38 GHz 5G MIMO Antenna With Modified Ground for Isolation and Bandwidth Improvement

This letter presents a high-isolation dual-band multiple-input multiple-output (MIMO) antenna based on the ground plane modification and optimized metamaterials (MMs) for 5G millimeter-wave applications. The antenna is a monopole providing a dual-band response at 5G 28/38 bands with a small physical size (4.8 × 2.9 × 0.762 mm <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">3</sup> , excluding the feeding line). The MIMO consists of two symmetric radiating elements arranged adjacently with the mutual coupling of −19 dB and −17 dB at 28 and 38 GHz, respectively. Reducing the ground length enhances the isolation to 34.6 dB at 28 GHz, whereas embedding the MMs diminishes the coupling to −47 dB at 38 GHz. The trust-region gradient-based search algorithm is employed to optimize the MM structure dimensions and to enhance the isolation at 38 GHz. The MMs also contribute to the enhancement of bandwidth from 1.6 GHz to 1.7 GHz at 28 GHz (27.1–28.8 GHz) and from 2.75 GHz to 3.7 GHz at 38 GHz (35.2–38.9 GHz). The proposed system is capable of covering both 28/38 bands with low profile structure, high isolation, low envelope correlation coefficient (ECC) ( <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\lt 0.2 \cdot 10^{-4}$</tex-math></inline-formula> ), and high diversity gain (DG) (>9.99 dB). These properties demonstrate the system applicability for 5G millimeter-wave cellular communications.

Pure-displacement formulation and bulk modulus propagation for topology optimization with pressure loads

This work proposes a pure-displacement formulation for topology optimization with pressure loads and the use of a continuous boundary propagation technique. This technique allows controlling the non-solid (fluid or void) constitutive behavior by disseminating the desired bulk modulus to holes in the solid domain. The novelty of the proposed approach is to enable the distribution of solid, fluid, and void simultaneously with one design variable field. A density-based topology optimization approach is considered, and a gradient-based interior point algorithm is used. The linear elasticity equilibrium equations are discretized by the finite element method and solved with a sparse direct solver. The sensitivities are calculated by the discrete adjoint method with automatic differentiation. The optimization problems encompass compliance minimization (in 2D and 3D) and a compliant mechanism design to maximize the output displacement. Three examples evaluate the proposed formulation: the internally-pressurized lid, the piston head, and the inverter-compliant mechanism. The results show that different designs are obtained by considering different constitutive behavior for the non-solid phase.

\(\boldsymbol{\mathcal{L}_2}\)-Optimal Reduced-Order Modeling Using Parameter-Separable Forms

We provide a unifying framework for -optimal reduced-order modeling for linear time-invariant dynamical systems and stationary parametric problems. Using parameter-separable forms of the reduced-model quantities, we derive the gradients of the cost function with respect to the reduced matrices, which then allows a nonintrusive, data-driven, gradient-based descent algorithm to construct the optimal approximant using only output samples. By choosing an appropriate measure, the framework covers both continuous (Lebesgue) and discrete cost functions. We show the efficacy of the proposed algorithm via various numerical examples. Furthermore, we analyze under what conditions the data-driven approximant can be obtained via projection.

Optimization on Elastoplasticity of Functionally Graded Shells of Revolution under Axisymmetric Loading

Depending on the load level, structures can experience a material nonlinearity known as elastoplasticity, which has an important role in the behaviour of structures. In order to avoid the elastoplastic behaviour, it is necessary to find the optimal thickness distribution, which corresponds to the minimum mass that provides an elastic behaviour for a certain load level. The elastoplasticity analysis of functionally graded axisymmetric shells under axisymmetric mechanical loading, and the subsequent optimization, was performed by using a simple conical frustum finite element model with two nodal circles; three degrees of freedom per node, which was based on Kirchhoff’s theory allowing for shear deformation; and using a reduced numerical integration procedure that is essential for its success when applied to thin shells. The formulation accounts for the calculation of the displacements and through-thickness stress distribution, including the effective stress. In this work, the thickness was the design variable in the optimization procedure and the mass was the objective function that needed to be minimized subject to a constraint imposed on the effective stress. The optimization solutions were obtained by using a feasible arc interior point gradient-based algorithm. Some illustrative examples were performed, and the corresponding results are presented and discussed.

Progressive Learning Hill Climbing Algorithm with Energy-Map-Based Initialization for Image Reconstruction.

Image reconstruction is an interesting yet challenging optimization problem that has several potential applications. The task is to reconstruct an image using a fixed number of transparent polygons. Traditional gradient-based algorithms cannot be applied to the problem since the optimization objective has no explicit expression and cannot be represented by computational graphs. Metaheuristic search algorithms are powerful optimization techniques for solving complex optimization problems, especially in the context of incomplete information or limited computational capability. In this paper, we developed a novel metaheuristic search algorithm named progressive learning hill climbing (ProHC) for image reconstruction. Instead of placing all the polygons on a blank canvas at once, ProHC starts from one polygon and gradually adds new polygons to the canvas until reaching the number limit. Furthermore, an energy-map-based initialization operator was designed to facilitate the generation of new solutions. To assess the performance of the proposed algorithm, we constructed a benchmark problem set containing four different types of images. The experimental results demonstrated that ProHC was able to produce visually pleasing reconstructions of the benchmark images. Moreover, the time consumed by ProHC was much shorter than that of the existing approach.

Assistive Data Glove for Isolated Static Postures Recognition in American Sign Language Using Neural Network

Sign language recognition is one of the most challenging tasks of today’s era. Most of the researchers working in this domain have focused on different types of implementations for sign recognition. These implementations require the development of smart prototypes for capturing and classifying sign gestures. Keeping in mind the aspects of prototype design, sensor-based, vision-based, and hybrid approach-based prototypes have been designed. The authors in this paper have designed sensor-based assistive gloves to capture signs for the alphabet and digits. These signs are a small but important fraction of the ASL dictionary since they play an essential role in fingerspelling, which is a universal signed linguistic strategy for expressing personal names, technical terms, gaps in the lexicon, and emphasis. A scaled conjugate gradient-based back propagation algorithm is used to train a fully-connected neural network on a self-collected dataset of isolated static postures of digits, alphabetic, and alphanumeric characters. The authors also analyzed the impact of activation functions on the performance of neural networks. Successful implementation of the recognition network produced promising results for this small dataset of static gestures of digits, alphabetic, and alphanumeric characters.

An Integrated Co-Design Optimization Toolchain Applied to a Conjugate Cam-Follower Drivetrain System

Due to ever increasing performance requirements, model-based optimization and control strategies are increasingly being adopted by machine builders and automotive companies. However, this demands an increase in modelling effort and a growing knowledge of optimization techniques, as a sufficient level of detail is required in order to evaluate certain performance characteristics. Modelling tools such as MATLAB Simscape have been created to reduce this modelling effort, allowing for greater model complexity and fidelity. Unfortunately, this tool cannot be used with high-performance gradient-based optimization algorithms due to obfuscation of the underlying model equations. In this work, an optimization toolchain is presented that efficiently interfaces with MATLAB Simscape to reduce user effort and the necessary skill and computation time required for the optimization of high-fidelity drivetrain models. The toolchain is illustrated on an industrially relevant conjugate cam-follower system, which is modelled in the Simscape environment and validated with respect to a higher-fidelity modeling technique, namely, the finite element method (FEM).

UAV-Aided Computation Offloading in Mobile-Edge Computing Networks: A Stackelberg Game Approach

Unmanned aerial vehicles (UAVs) are considered as a promising method to provide additional computation capability and wide coverage for mobile users (MUs), especially when MUs are not within the communication range of the infrastructure. In this article, a UAV-aided mobile-edge computing (MEC) network, including one UAV-MEC server, one BS-MEC server, and several MUs, is investigated for computation offloading, in which the edge service provider (ESP) manages two kinds of servers. It is considered that MUs have a large number of computation tasks to conduct, while the ESP has idle computational resources. MUs can choose to offload their tasks to the ESP to reduce their pressure and cost, and the ESP can make a profit by selling computational resources. The interaction among the ESP and MUs is modeled as a Stackelberg game, and both the ESP and MUs want to maximize their utility. The proposed game is analyzed by using the backward induction method, and it is proved that a unique Nash equilibrium can be achieved in the game. Then, a gradient-based dynamic iterative search algorithm (GDISA) is proposed to get the approximate optimal solution. Finally, the effectiveness of GDISA is verified by extensive simulations, and the results show that GDISA performs better than other benchmark methods under different scenarios.

Physics-informed deep learning for simultaneous surrogate modeling and PDE-constrained optimization of an airfoil geometry

We use a physics-informed neural network (PINN) to simultaneously model and optimize the flow around an airfoil to maximize its lift to drag ratio. The parameters of the airfoil shape are provided as inputs to the PINN and the multidimensional search space of shape parameters is populated with collocation points to ensure that the Navier–Stokes equations are approximately satisfied throughout. We use the fact that the PINN is automatically differentiable to calculate gradients of the lift-to-drag ratio with respect to the airfoil shape parameters. This allows us to optimize with the L-BFGS gradient-based algorithm, which is more efficient than non-gradient-based algorithms, without deriving an adjoint code. We train the PINN with adaptive sampling of collocation points, such that the accuracy of the solution improves as the solution approaches the optimal point. We demonstrate this on two examples: one that optimizes a single parameter, and another that optimizes eleven parameters. The method is successful and, by comparison with conventional CFD, we find that the velocity and pressure fields have small pointwise errors and that the method converges to optimal parameters. We find that different PINNs converge to slightly different parameters, reflecting the fact that there are many closely-spaced local minima when using stochastic gradient descent. This method can be applied relatively easily to other optimization problems and avoids the difficult process of writing adjoint codes. It is, however, more computationally expensive than adjoint-based optimization. As knowledge about training PINNs improves and hardware dedicated to neural networks becomes faster, this method of simultaneous training and optimization with PINNs could become easier and faster than using adjoint codes.

Recursive inverse dynamics sensitivity analysis of open-tree-type multibody systems

We present a first-order recursive approach to sensitivity analysis based on the application of the direct differentiation method to the inverse Lagrangian dynamics of rigid multibody systems. Our method is simple and efficient and is characterized by the following features. Firstly, it describes the kinematics of multibody systems using branch connectivity graphs and joint-branch connectivity matrices. For most mechanical systems with an open-tree kinematic structure, this method turns out to be more efficient compared to other kinematic descriptions employing joint or link connectivity graphs. Secondly, a recursive sensitivity analysis is presented for a dynamic system with an open-tree kinematic structure and inverse dynamic equations described in terms of the Lagrangian formalism. Thirdly, known approaches to recursive inverse dynamic and sensitivity analyses are modified to include dynamic systems with external forces and torques acting simultaneously at all joints. Finally, the proposed method for sensitivity analysis is easy to implement and computationally efficient. It can be utilized to evaluate the derivatives of the dynamic equations of multibody systems in gradient-based optimization algorithms. It also allows less experienced users to perform sensitivity analyses using the power of high-level programming languages such as MATLAB. To illustrate the method, simulation results for a human body model are discussed. The shortcomings of the method and possible directions for future work are outlined.

Efficient Algorithms for High-Dimensional Quantum Optimal Control of a Transmon Qubit

Hybrid quantum classical optimization using near-term quantum technology is an emerging direction for exploring quantum advantage in high-dimensional systems. However, precise characterization of all experimental parameters is often impractical and challenging. A viable approach is to use algorithms that rely entirely on black-box inference rather than analytical gradients. Here, we combine randomized perturbation gradient estimation with adaptive momentum gradient updates and propose AdamSPSA and AdamRSGF algorithms. We prove the asymptotic convergence of the proposed algorithms in a convex setting and benchmark them against other gradient-based black-box optimization algorithms on nonconvex quantum optimal control tasks. Our results indicate that these algorithms accelerate the optimization rate, lower the optimization loss, and efficiently tune up high-fidelity Hann-window single-qubit gates from trivial initial conditions with up to 80 variables for a transmon qubit.

Change detection in moving-camera videos with limited samples using twin-CNN features and learnable morphological operations

This paper presents a new system to detect changes between reference and target videos suitable for small-scale datasets. Twin pre-trained ResNet-50 features are processed using a learning-based pipeline that has a limited number of adjustable parameters, allowing end-to-end training even on relatively small databases. This is achieved with two innovative modules in tandem: a low-complexity dissimilarity module and a post-processing step using learnable morphological operations. Both can be smoothly incorporated in optimization procedures that employ gradient-based algorithms. The pipeline ends with temporal consistency and change classification modules, and it is evaluated on the VDAO dataset, a challenging database of videos recorded with moving cameras in a cluttered industrial environment. Ablation studies show how each proposed module contributes to the final system performance, with a prominence role for the newly proposed ones. Results indicate that the proposed system achieves a detection performance that is about 18% superior to the one of current state-of-the-art methods. Software, results, and a pre-trained architecture of the proposed framework are available at https://github.com/rafaelpadilla/TCF-LMO.

Jacobi-type algorithms for homogeneous polynomial optimization on Stiefel manifolds with applications to tensor approximations

This paper mainly studies the gradient-based Jacobi-type algorithms to maximize two classes of homogeneous polynomials with orthogonality constraints, and establish their convergence properties. For the first class of homogeneous polynomials subject to a constraint on a Stiefel manifold, we reformulate it as an optimization problem on a unitary group, which makes it possible to apply the gradient-based Jacobi-type (Jacobi-G) algorithm. Then, if the subproblem can always be represented as a quadratic form, we establish the global convergence of Jacobi-G under any one of three conditions. The convergence result for the first condition is an easy extension of the result by Usevich, Li, and Comon [SIAM J. Optim. 30 (2020), pp. 2998–3028], while other two conditions are new ones. This algorithm and the convergence properties apply to the well-known joint approximate symmetric tensor diagonalization. For the second class of homogeneous polynomials subject to constraints on the product of Stiefel manifolds, we reformulate it as an optimization problem on the product of unitary groups, and then develop a new gradient-based multiblock Jacobi-type (Jacobi-MG) algorithm to solve it. We establish the global convergence of Jacobi-MG under any one of the above three conditions, if the subproblem can always be represented as a quadratic form. This algorithm and the convergence properties are suitable to the well-known joint approximate tensor diagonalization. As the proximal variants of Jacobi-G and Jacobi-MG, we also propose the Jacobi-GP and Jacobi-MGP algorithms, and establish their global convergence without any further condition. Some numerical results are provided indicating the efficiency of the proposed algorithms.

Distributed strategy selection: A submodular set function maximization approach

Joint utility-maximization problems for multi-agent systems often should be addressed by distributed strategy-selection formulation. Constrained by discrete feasible strategy sets, these problems are broadly formulated as NP-hard combinatorial optimization problems. In many cases, these problems can be cast as constrained submodular set function maximization problems, which also belong to the NP-hard domain of problems. A prominent example is the problem of multi-agent mobile sensor dispatching over a discrete domain. This paper considers a class of submodular optimization problems that consist of maximization of a monotone and submodular set function subject to a partition matroid constraint over a group of networked agents that communicate over a connected undirected graph. We work with the value oracle model. Consequently, the only access of the agents to the utility function is through a black box that returns the utility function value given a specific strategy set. We propose a distributed suboptimal polynomial-time algorithm that enables each agent to obtain its respective strategy via local interactions with its neighboring agents. Our solution is a fully distributed gradient-based algorithm using the submodular set functions’ multilinear extension followed by a distributed stochastic Pipage rounding procedure. This algorithm results in a strategy set that when the team utility function is evaluated at the worst case, the utility function value is in 1c(1−e−c−O(1/T)) of the optimal solution with c being the curvature of the submodular function. An example demonstrates our results.

Ensemble Classification With Noisy Real-Valued Base Functions

In data-intensive applications, it is advantageous to perform partial processing close to the data, and communicate intermediate results to a central processor, instead of the data itself. When the communication or computation medium is noisy, the resulting degradation in computation quality at the central processor must be mitigated. We study this problem for the setup of binary classification performed by an ensemble of base functions communicating real-valued confidence levels. We propose a noise-mitigation solution that optimizes the transmission gains and aggregation coefficients of the base functions. Toward that, we formulate a post-training gradient-based optimization algorithm that minimizes the error probability given the training dataset and the noise parameters. We further derive lower and upper bounds on the optimized error probability, and show empirical results that demonstrate the enhanced performance achieved by our approach on real data.

OsmoticGate: Adaptive Edge-Based Real-Time Video Analytics for the Internet of Things

Edge computing has gained momentum in recent years, and can provide more immediate analysis of streaming video data. However, the edge devices often lack the computing capabilities (processing power, memory) to guarantee reasonable performance (e.g., accuracy, latency, throughput) for complex video analytics tasks. To alleviate this critical problem, the prevalent trend is to offload some video analytics tasks from the edge devices to the cloud. However, existing offloading approaches fail to consider the dynamic nature of the video analytical tasks (e.g., varying encoding format for different video content) and are unable to adapt system dynamics (e.g., varying workload between the edge and the cloud). To overcome the limitation of existing approaches, we develop an edge-cloud offloading performance model based on the concept of hierarchical queues. The resource constraints (e.g., computing capacity and network bandwidth) of each edge nodes and dynamic edge-cloud network conditions are used to parameterize the performance model. Since finding optimal solutions for the performance model is NP-hard, we develop a two-stage gradient-based algorithm and compare it with some state-of-the-art (SOTA) solutions (e.g., FastVA, DeepDecision, Hill Climbing). Experiments have shown our performance model's advantages and the stability of the proposed offloading approach given different systems (edge-cloud) and video analytics application dynamics.

Multi-modal multi-resolution atlas of the human neonatal cerebral cortex based on microstructural similarity

The neonatal period is a critical window for the development of the human brain and may hold implications for the long-term development of cognition and disorders. Multi-modal connectome studies have revealed many important findings underlying the adult brain but related studies were rare in the early human brain. One potential challenge is the lack of an appropriate and unbiased parcellation that combines structural and functional information in this population. Using 348 multi-modal MRI datasets from the developing human connectome project, we found that the information fused from the structural, diffusion, and functional MRI was relatively stable across MRI features and showed high reproducibility at the group level. Therefore, we generated automated multi-resolution parcellations (300 - 500 parcels) based on the similarity across multi-modal features using a gradient-based parcellation algorithm. In addition, to acquire a parcellation with high interpretability, we provided a manually delineated parcellation (210 parcels), which was approximately symmetric, and the adjacent areas around each boundary were statistically different in terms of the integrated similarity metric and at least one kind of original features. Overall, the present study provided multi-resolution and neonate-specific parcellations of the cerebral cortex based on multi-modal MRI properties, which may facilitate future studies of the human connectome in the early development period.

Jacobian methods for dynamic polarization control in optical applications.

Dynamic polarization control (DPC) is beneficial for many optical applications. It is often realized via tunable waveplates to perform automatic polarization tracking and manipulation. Efficient algorithms are essential to realize an endless polarization control process at high speed. However, the standard gradient-based algorithm is not well analyzed. Here, we model the DPC with a Jacobian-based control theory framework that finds a lot in common with robot kinematics. We then give a detailed analysis of the condition of the Stokes vector gradient as a Jacobian matrix. We identify the multi-stage DPC as a redundant system enabling control algorithms with null-space operations. An efficient, reset-free algorithm can be found. We anticipate more customized DPC algorithms to follow the same framework in various optical systems.