Convergence Properties Research Articles

Alternating minimization for generalized rank-1 matrix sensing: sharp predictions from a random initialization

Abstract We consider the problem of estimating the factors of a rank-$1$ matrix with i.i.d. Gaussian, rank-$1$ measurements that are nonlinearly transformed and corrupted by noise. Considering two prototypical choices for the nonlinearity, we study the convergence properties of a natural alternating update rule for this non-convex optimization problem starting from a random initialization. We show sharp convergence guarantees for a sample-split version of the algorithm by deriving a deterministic one-step recursion that is accurate even in high-dimensional problems. Notably, while the infinite-sample population update is uninformative and suggests exact recovery in a single step, the algorithm—and our deterministic one-step prediction—converges geometrically fast from a random initialization. Our sharp, non-asymptotic analysis also exposes several other fine-grained properties of this problem, including how the nonlinearity and noise level affect convergence behaviour. On a technical level, our results are enabled by showing that the empirical error recursion can be predicted by our deterministic one-step updates within fluctuations of the order $n^{-1/2}$ when each iteration is run with $n$ observations. Our technique leverages leave-one-out tools originating in the literature on high-dimensional $M$-estimation and provides an avenue for sharply analyzing complex iterative algorithms from a random initialization in other high-dimensional optimization problems with random data.

Low-Rank Tensor Completion Based on Self-Adaptive Learnable Transforms.

The tensor nuclear norm (TNN), defined as the sum of nuclear norms of frontal slices of the tensor in a frequency domain, has been found useful in solving low-rank tensor recovery problems. Existing TNN-based methods use either fixed or data-independent transformations, which may not be the optimal choices for the given tensors. As the consequence, these methods cannot exploit the potential low-rank structure of tensor data adaptively. In this article, we propose a framework called self-adaptive learnable transform (SALT) to learn a transformation matrix from the given tensor. Specifically, SALT aims to learn a lossless transformation that induces a lower average-rank tensor, where the Schatten- p quasi-norm is used as the rank proxy. Then, because SALT is less sensitive to the orientation, we generalize SALT to other dimensions of tensor (SALTS), namely, learning three self-adaptive transformation matrices simultaneously from given tensor. SALTS is able to adaptively exploit the potential low-rank structures in all directions. We provide a unified optimization framework based on alternating direction multiplier method for SALTS model and theoretically prove the weak convergence property of the proposed algorithm. Experimental results in hyperspectral image (HSI), color video, magnetic resonance imaging (MRI), and COIL-20 datasets show that SALTS is much more accurate in tensor completion than existing methods. The demo code can be found at https://faculty.uestc.edu.cn/gaobin/zh_CN/lwcg/153392/list/index.htm.

A Comparative Analysis of Two Semi Analytic Approaches in Solving Systems of First-Order Differential Equations

The resolution of systems of first-order ordinary differential equations (ODEs) stands as a pivotal pursuit with extensive implications across scientific and engineering domains. In tackling this fundamental task, this study undertakes a rigorous comparative assessment of two semi-analytic methodologies, the Variational Iterative Method (VIM) and the New Iterative Method (NIM). Motivated by the need to address a critical research gap, our investigation delves into these approaches' relative merits and demerits. Firstly, it conducts a comprehensive examination of VIM, a well-established method, juxtaposed with NIM, a relatively unexplored approach, to uncover their comparative strengths and limitations. Secondly, the study contributes to the existing knowledge in numerical methods for ODEs by shedding light on essential performance characteristics, including convergence properties, computational efficiency, and accuracy, across a diverse array of ODE systems. Through meticulous numerical experimentation, we not only reveal practical insights into the efficacy of VIM and NIM but also bridge a significant knowledge gap in the field of numerical ODE solvers. Our findings highlight VIM as the more effective method, thus advancing our understanding of semi-analytic approaches for solving ODE systems and furnishing valuable guidance for practitioners and researchers in selecting the most suitable method for their specific applications

A variable-order fractional memristor neural network: Secure image encryption and synchronization via a smooth and robust control approach

In this research, we introduce and investigate a variable-order fractional memristor neural network, focusing on its engineering applications in synchronization and image encryption. This study stands out as a pioneering effort in proposing such an architecture for image encryption purposes. Distinct from conventional fractional-order systems, our model incorporates a time-varying fractional derivative, leading to more complex behaviors. Through numerical simulations, we vividly demonstrate the chaotic dynamics of the system. Our results further reveal the system's outstanding performance in image encryption applications. To augment the system's efficiency, we introduce a robust control strategy that guarantees smooth stabilization and synchronization of the variable-order fractional system. Considering the unique variable-order fractional nature of the system, we provide theoretical validations and empirical evidence supporting its stability and convergence properties. Additionally, we present synchronization outcomes between pairs of such neural networks employing our robust control approach. Our numerical analyses firmly substantiate the superiority of our control strategy, particularly highlighting its precision, robustness, and ability to maintain chattering-free performance under external disturbances.

New Methods for Optimal Power Allocation and Joint Resource Scheduling in 5G Network which Use Mobile Edge Computing

Mobile Edge Computing (MEC) is considered one of the enabling and promising technologies in 5G networks, especially with the massive data movement of various devices and the increased demand for computing. Here, computational offloading of tasks to edge clouds provides an effective, flexible, low-latency solution for mobile users in the network. However, the limited computing resources in edge clouds and the dynamic demands of mobile users make it difficult to schedule computing requests to appropriate edge clouds, and make the offloading process energetically expensive for devices. Therefore, it is very important to design an energy-efficient offloading strategy. To this end, we first formulate the transmission power allocation (PA) problem for mobile phone users to minimize power consumption. Using a quasi-convex technique, we address the (PA) problem using a (Hybrid GAPSO) algorithm resulting from combining the Genetic Algorithm (GA) with the Particle Swarm Optimization (PSO) algorithm. Next, we model the joint request offloading and resource scheduling (JRORS) problem as a mixed- nonlinear program to minimize the response delay of requests. The (JRORS) problem can be divided into two problems, namely the request offloading (RO) problem and the computing resource scheduling (RS) problem. Therefore, we analyze the JRORS problem as a dual decision problem and propose a multi-objective particle swarm optimization algorithm referred as (MO-PSO). The simulation results show that (HGAPSO) can save transmission power consumption and has good convergence property, and (MO-PSO) outperforms existing methods in terms of response rate and can maintain good performance in a dynamic network.

Positivity-preserving discontinuous spectral element methods for compressible multi-species flows

We introduce a novel positivity-preserving numerical stabilisation approach for high-order discontinuous spectral element approximations of compressible multi-species flows. The underlying stabilisation method is the adaptive entropy filtering approach (Dzanic and Witherden, J. Comput. Phys., 468, 2022), which is extended to the conservative formulation of the multi-species flow equations. We show that the straightforward enforcement of entropy constraints in the filter yields poor results around species interfaces and propose an adaptive switch for the entropy bounds based on the convergence properties of the pressure field which drastically improves its performance for multi-species flows. The proposed approach is shown in a variety of numerical experiments applied to the multi-species Euler and Navier–Stokes equations computed on unstructured grids, ranging from shock-fluid interaction problems to three-dimensional viscous flow instabilities. We demonstrate that the approach can retain the high-order accuracy of the underlying numerical scheme even at smooth extrema, ensure the positivity of the species density and pressure in the vicinity of shocks and contact discontinuities, and accurately predict small-scale flow features with minimal numerical dissipation.

Discretizing and validating Keyfitz' entropy for any demographic classification

Abstract Keyfitz's entropy originally is a shape measure of senescence for continuous‐time models. But a later formula of Keyfitz's entropy for discrete time also exists. de Vries et al. (2023) showed that this discrete‐time formula is not a genuine shape measure of senescence and proposed a new discrete‐time formula for entropy that qualifies as a shape measure. However, these authors obtained their new formula by discretizing an alternative version of Keyfitz' continuous‐time formula and not by following his original derivation, that is not by computing the elasticity of life expectancy to an age‐uniform mortality change. Hence, they regarded as a loose end of their work whether discretizing Keyfitz' original derivation would also lead to their new formula. These authors also deemed lack of direct applicability to stage‐classified models a major downside of the new formula they proposed. Here we do two separate things: we discretize Keyfitz' original derivation of entropy and we generalize the new discrete‐time formula for this quantity proposed by de Vries et al. (2023). We show that the discrete‐time formula for entropy that de Vries et al. (2023) propose to supersede, while potentially problematic for studying senescence, is, as Keyfitz' original formula, an elasticity. Generalizing the work of de Vries et al. (2023) leads to a formula for entropy that can be directly applied as a shape measure of senescence to models with any demographic classification. We also prove convergence properties that validate the application of this general formula to models classified by stage. We thus support and generalize the successful approach by de Vries et al. (2023) of obtaining new discrete‐time formulas for Keyfitz' entropy from a shape perspective: merits of shape measures of senescence, which capture the overall age‐pattern of mortality, should be based on their desirable properties and not necessarily on their means of derivation. Moreover, our work gives one more example of how in ecological and evolutionary modelling the passage from continuous to discrete time may be non‐obvious.

New efficient four-stage implicit trigonometrically fitted modified RKN for second-order ODEs

The construction of implicit RKN is investigated in this paper. We finally obtain four four-stages implicit integrators by considering the symmetric, symplectic, and trigonometric fitting conditions. For the new obtained methods, we analyze their global convergence and stability property. And we carry out numerical experiments on some commonly considered problems in the literature. In view of the numerical experiments, we observe that the new methods outperform several efficient RKN methods in terms of accuracy and efficiency.

On convergence properties of the brain-state-in-a-convex-domain

Convergence in the presence of multiple equilibrium points is one of the most fundamental dynamical properties of a neural network (NN). Goal of the paper is to investigate convergence for the classic Brain-State-in-a-Box (BSB) NN model and some of its relevant generalizations named Brain-State-in-a-Convex-Body (BSCB). In particular, BSCB is a class of discrete-time NNs obtained by projecting a linear system onto a convex body of Rn. The main result in the paper is that the BSCB is convergent when the matrix of the linear system is symmetric and positive semidefinite or, otherwise, it is symmetric and the step size does not exceed a given bound depending only on the minimum eigenvalue of the matrix. This result generalizes previous results in the literature for BSB and BSCB and it gives a solid foundation for the use of BSCB as a content addressable memory (CAM). The result is proved via Lyapunov method and LaSalle’s Invariance Principle for discrete-time systems and by using some fundamental inequalities enjoyed by the projection operator onto convex sets as Bourbaki–Cheney–Goldstein inequality.

Another hybrid conjugate gradient method as a convex combination of WYL and CD methods

Abstract Conjugate gradient (CG) methods are a popular class of iterative methods for solving linear systems of equations and nonlinear optimization problems. In this paper, a new hybrid conjugate gradient (CG) method is presented and analyzed for solving unconstrained optimization problems, where the parameter β k \beta_{k} is a convex combination of β k WYL \beta_{k}^{\mathrm{WYL}} and β k CD \beta_{k}^{\mathrm{CD}} . Under the strong Wolfe line search, the new method possesses the sufficient descent condition and the global convergence properties. The preliminary numerical results show the efficiency of our method in comparison with other CG methods. Furthermore, the proposed algorithm HWYLCD was extended to solve the problem of a mode function.

Formation tracking control in open multiagent systems with general linear dynamics

This paper studies the formation tracking problem of the so-called open multiagent systems. In these systems, agents can freely join or leave at any time, i.e. the system sizes are changeable. To achieve formation tracking control in these systems, we propose a general theoretical analysis framework that consists of a formation adjustment strategy and a formation tracking protocol. Therein, the former is used to adjust the predefined task-oriented formation shape to the varying size of the system. This strategy is built on the existing scalable self-healing idea and then modified by a size estimation method. The latter is used to drive the present followers to form the adjusted formation and track a leader. To accommodate undirected and directed graphs, two distributed formation tracking protocols are designed respectively. Due to possible agent migration, the designed tracking protocols are endowed with self-adaptive ability, i.e. without depending on global topology information. Then, their convergence properties are analysed using the Lyapunov theory. Finally, two simulations are presented to verify the effectiveness of the proposed framework.

A hybrid SCSO-QNN approach based load frequency control of three area power system with renewable sources using FOPID controller

ABSTRACT This research proposes a hybrid SCSO-QNN method for designing a FOPID controller in a three-area power system to regulate load frequency. The proposed hybrid approach combines quantum neural networks with sand cat swarm optimization, and it is commonly named as the SCSO-QNN method. The proposed method’s primary goal is to overcome load disturbances and attain the intended output of the interconnected system. The SCSO generates controller parameters tuned by the FOPID controller, providing a significant improvement in performance, while the QNN is employed to forecast the optimal control signal of the converter. Reducing steady-state errors and improving system stability are areas where the FOPID performs better than a traditional integer-order PID controller. Moreover, SCSO-QNN optimization technique exhibits excellent convergence properties, resulting in improved control performance. The proposed method ensures system frequency control by controlling frequency deviation and power fluctuations in the connect line during load disturbances. The proposed method is executed on the MATLAB platform and contrasted with current techniques. The proposed technique outperforms all current techniques, including ABC-FOPID, Genetic Algorithm, and Artificial Bee Colony PID. The proposed method Area control error is 1.8%, which is less than other existing methods.

A multi-objective integrated scheduling of remanufacturing system considering time window constrained outsourcing option

The current strategy of collaborative production is garnering attention in the remanufacturing industry. The collaborative execution of tasks through the sharing of production resources such as facilities and labor can effectively augment the utilization rate of end-of-life products. However, existing studies on scheduling problem in remanufacturing systems does not take into this issue. To fill this gap, this study considers an integrated scheduling problem in remanufacturing systems with parallel disassembly workstations, a flexible-shop-type reprocessing shop and a reassembly workstation, where the components obtained by disassembling can be outsourced within the time window given by the outsourcer. To address this issue, a mathematical model aiming at minimizing the completion time and total cost is established. Then, a knowledge guided artificial bee colony algorithm (KGABC) with a five-layer encoding scheme is proposed to solve it. In the KGABC, a two-step heuristic initialization approach is invented based on the characteristics of the disassembly and reprocessing stage. According to the problem-specific knowledge, a reprocessing advance scheduling approach is proposed to reduce idle time; an outsourcing assessment method is presented for determining the feasibility of outsourcing operation; a reassembly priority management strategy is put forward to determine the reassembly sequence of products. In addition, a knowledge guided search strategy is proposed in the onlooker bee phase to enhance the local search ability of the algorithm. Finally, the KGABC is subjected to comparative testing against the state-of-the-art algorithms on numerical benchmark instances with different scales. Simulation results show that the KGABC exhibits significant advantages in terms of solution accuracy, solution diversity, and convergence properties. Moreover, a real case study is adopted to validate KGABC's supremacy in solving the real-world remanufacturing scheduling problem with relatively lower time and cost.

Theoretical Evaluation of Machine Learning and Deep Learning Applications in Various Domain

The theoretical evaluation of machine learning (ML) and deep learning (DL) applications encompasses the rigorous analysis of their mathematical foundations, algorithmic principles, and performance metrics. This evaluation aims to understand the capabilities and limitations of various ML and DL models, including their generalization ability, convergence properties, and computational efficiency. By exploring theoretical aspects such as bias-variance tradeoff, overfitting, underfitting, and the impact of hyperparameters, researchers can optimize model architectures and training processes. Additionally, the theoretical examination provides insights into the interpretability and robustness of models, guiding the development of more reliable and efficient applications across diverse domains such as computer vision, natural language processing, and predictive analytics.

On some algorithms for estimation in Gaussian graphical models

Abstract In Gaussian graphical models, the likelihood equations must typically be solved iteratively. This paper investigates two algorithms: a version of iterative proportional scaling, which avoids inversion of large matrices, and an algorithm based on convex duality and operating on the covariance matrix by neighbourhood coordinate descent, which corresponds to the graphical lasso with zero penalty. For large, sparse graphs, the iterative proportional scaling algorithm appears feasible and has simple convergence properties. The algorithm based on neighbourhood coordinate descent is extremely fast and less dependent on sparsity, but needs a positive-definite starting value to converge. We provide an algorithm for finding such a starting value for graphs with low colouring number. As a consequence, we also obtain a simplified proof of existence of the maximum likelihood estimator in such cases.

Automatic control method of spherical wave exposure interference field based on the Moiré alignment principle

Aberration-corrected gratings are widely applied in spectral analysis owing to their dispersion and convergence properties. However, the phase distribution error of the exposure interference field reduces the accuracy of the groove density distribution, making it difficult to satisfy the needs of high-precision spectral instruments. Therefore, this paper establishes an error model for the phase distribution of the spherical wave exposure interference field, describing the relationship between the phase distribution error and the recording parameter error. This model is used to propose a method of automatically controlling a spherical wave exposure interference field based on Moiré alignment principle. This method automatically measures the phase of the interference field by extracting the phase from the Moiré fringes generated by the superposition of the interference field and the reference grating, and then inversely calculates the recording parameters. The measurement results are then fed back to the automatic calibration mechanism for compensation, thereby achieving automatic control of the exposure interference field. Applying this method to calibrate the exposure interference field reduces the average relative error of the groove density of the produced plane aberration-corrected grating by two orders of magnitude compared with that of the traditional control method. This method significantly enhances the control accuracy for the spherical wave exposure interference field, improving the distribution accuracy of the groove density of the aberration-corrected grating, thereby supporting spectral analysis.

Schrödinger as a Quantum Programmer: Estimating Entanglement via Steering

Quantifying entanglement is an important task by which the resourcefulness of a quantum state can be measured. Here, we develop a quantum algorithm that tests for and quantifies the separability of a general bipartite state by using the quantum steering effect, the latter initially discovered by Schrödinger. Our separability test consists of a distributed quantum computation involving two parties: a computationally limited client, who prepares a purification of the state of interest, and a computationally unbounded server, who tries to steer the reduced systems to a probabilistic ensemble of pure product states. To design a practical algorithm, we replace the role of the server with a combination of parameterized unitary circuits and classical optimization techniques to perform the necessary computation. The result is a variational quantum steering algorithm (VQSA), a modified separability test that is implementable on quantum computers that are available today. We then simulate our VQSA on noisy quantum simulators and find favorable convergence properties on the examples tested. We also develop semidefinite programs, executable on classical computers, that benchmark the results obtained from our VQSA. Thus, our findings provide a meaningful connection between steering, entanglement, quantum algorithms, and quantum computational complexity theory. They also demonstrate the value of a parameterized mid-circuit measurement in a VQSA.

Multi-view reduced dimensionality K-means clustering with [formula omitted]-norm and Schatten [formula omitted]-norm

Recently, multi-view high dimensional data obtained from diverse domains or various feature extractors has drawn great attention due to its reflection of different properties or distributions. In this paper, we propose a novel unsupervised multi-view clustering method, which is called Multi-View Reduced Dimensionality K-means clustering (MRDKM) and integrates the dimension reduction mechanism, σ-norm, Schatten p-norm, and multi-view K-means clustering. Moreover, an unsupervised optimization scheme was proposed to solve the minimization problem with good convergence properties. Comprehensive evaluations of five benchmark datasets and comparisons with several multi-view clustering algorithms demonstrate the superiority of the proposed work.

Optimization of robot manipulator configuration calibration by using Zhang neural network for repetitive motion

High precision and low complexity control algorithm plays an important role in the developing of the end-effector instrumentation of different robot manipulators. In order to reduce the kinetic energy and the high-speed drift phenomenon of the repetitive motion tracking task, the robot manipulator needs to calibrate its configuration. In this paper, we formulate the configuration calibration of the robot manipulator for the repetitive motion task as a future quadratic programming optimization problem constrained with equality constraints, which is also regarded as a fundamental problem in artificial intelligence and modern control engineering. Zhang neural network, which is a canonical method, can be adopted to deal with the continuous form of the future optimization problem, named as temporally dependent quadratic programming problem with equality constraints. In order to overcome the issue of temporally dependent inverse computing, a novel Zhang neural network model and its uncertain disturbance tolerant model, which are termed as filtered reciprocal-kind Zhang neural network model and uncertain disturbance tolerant filtered reciprocal-kind Zhang neural network model, respectively, are proposed by integrating the energy-type cost function and Zhang neural network design formula for solving the temporally dependent quadratic programming problem with equality constraints in this paper. Based on the Euler discrete formula and the models, the discrete filtered reciprocal-kind Zhang neural network and the discrete uncertain disturbance tolerant filtered reciprocal-kind Zhang neural network algorithms are proposed for solving the future quadratic programming problem with equality constraints and the robot manipulator configuration calibration problem of repetitive motion. The convergence properties of the reciprocal-kind Zhang neural network model and its corresponding uncertain disturbance tolerant model are obtained by Lyapunov stability theory of nonlinear system and its corresponding perturbed system, while the convergence property of the filtered reciprocal-kind Zhang neural network model is analyzed by the limit thinking. For the repetitive motion task, three experiments for solving the configuration calibration problem of PUMA560, Kinova Jaco2, and Franka Emika Panda robot manipulators are performed to illustrate the effectiveness, robustness and superiority of our proposed discrete filtered reciprocal-kind Zhang neural network algorithms.

THE BEST SPECTRAL CORRECTION OF DMDY CONJUGATE GRADIENT METHOD

In this paper, we present an enhanced spectral correction for the DMDY conjugate gradient method. Our approach involves integrating a third term and determining its parameter through three different approaches. The primary objective is to ensure the sufficient descent condition. By applying the Wolfe line search conditions, we establish the global convergence property for all three proposed algorithms. Numerical tests conclusively demonstrate the superior efficiency of our algorithms, surpassing that of existing methods.