Nonlinear Gradient Descent Research Articles

Comprehensive water quality indicators modeling by environmental protection view using multi optimized weighted ensemble machine learnings

The environmental protection water quality is a critical subject in dominance of the sustainable water resources management. Accordingly, the essential indicators of water quality were considered, which are not only appropriate indicators for water quality, but they can also serve as crucial indicators for the health of water environment and ecosystems. Therefore, this study uses well-known ensemble machine learning methodologies to investigate and predict the fluctuations of water quality parameters. Optimization procedures used to assemble machine learnings were non-linear programming (NLP), genetic, gradient descent, and least square algorithms, linear programming, particle swarm optimization, Nelder-Mead optimization, and simulating annealing optimization. Using optimization procedures, the basic MLs were assembled and eight new ensemble machine learning were developed. The studied area was the South Platte River basin, USA. The primary dataset was obtained through the online database of the United States Geological Survey, which contained sampling information on river water related to 2023–2024. Then, using clean missing and outlier data preprocessing techniques, the dataset was modified. Finally, using the 10-fold cross-validation technique, the primary data was validated. The results showed that NLP significantly improved the accuracy and performance of models, achieving the best performance with R2 of 0.9836 and 0.9031 across DO and pH modelings. The modeling results indicated that the pH parameter fluctuated within the safe range. While the DO seems that tolerated in unsafe domain for aquatic ecosystems. The findings of this research could help a wide range of decision-makers.

Temporal convex combination‐based secure distributed estimation against cyberattacks and noisy input

AbstractThis paper proposes a new secure distributed estimation for cyber‐physical systems against adversarial attacks with noisy input. To mitigate the effect of attacks, a novel distributed attack detection based on a reliable reference estimation obtained by temporal convex combination is proposed. Furthermore, for more effective and robust performance, an adaptation rule to adjust convex combination weights is presented, in which the generalized correntropy method with nonlinear loss function and stochastic gradient descent are utilized. Besides, to eliminate input noise, a bias‐compensation method in local adaptation of the secure distributed estimation is proposed. Simulations show superior dynamic and real‐time adaptability of the proposed algorithm under complex attacking scenarios.

Thermal-economic optimization design of shell and tube heat exchanger using an improved sparrow search algorithm

Optimal design of heat exchangers is challenging, since complex non-linear relationship exists among the various design parameters. This study aims to construct a full-dimensional thermal-economic model for shell-and-tube heat exchanger using two commonly (i.e., Kern and Bell-Delaware) methods based on Logarithmic-Mean-Temperature-Difference (LMTD) or ε-Number-of-Transfer-Units (ε-NTU) method and proposes a novel solving mechanism based on improved sparrow search algorithm to achieve thermal-economic optimization design of heat exchanger. The improved method organically combines nonlinear gradient descent weight factor, elite reverse strategy, selection-mutation operator to maintain population diversity, enhance convergence speed and avoid being stuck in local optima. Firstly, the constructed model’s accuracy is verified by numerical analysis. Then, the improved sparrow search algorithm is applied to four examples after algorithm parameter tuning, and the results show that the area decreases by 15.62%∼49.30% and total annual cost is reduced by 27.93% compared to the literature. It can also identify feasible solutions at a lower total cost than the metaheuristic method, with reductions of 7.56%∼59.42%(Kern) and 6.26%∼60.19%(Bell-Delaware). Moreover, a multi-objective optimization algorithm is obtained by adding fast non-dominated sorting mechanism and archival gene pool, and the superiority is verified by standard test functions. Finally, a multi-objective optimization scheme with the higher effectiveness-economic benefit is conducted to achieve cost-effectiveness tradeoff for two real-world cases. The results show that total cost of the most economic-efficient scheme decreases by 53.73% and effectiveness increases by 2.33%, while the runtime decreases by 35%. Furthermore, sensitivity analysis of effectiveness and total cost varying with crucial variables is performed and discussed.

Novel Contact Modeling for High Aspect Ratio Soft Robots

Contact modeling between a soft robot and its environment is challenging due to soft robots' compliance and the difficulty of embedding sensors. Current modeling methods are computationally expensive and require highly accurate material characterization to produce useful results. In this article, we present a contact model that utilizes linear complementarity and Hencky bar-chain methods, and requires only static images to efficiently predict the interaction between actuator and environment. These methods have yet to be introduced to the soft robotics community for modeling robots that deform due to eigenstrains or strains not caused by external forces. We validated our model using a custom experimental setup and computer vision algorithm on 3-mm OD, 90-mm long, tube-like actuators. Our results indicated a 1.06% difference in shape between model and experiment, with computation times in 10 s of ms—three to four orders of magnitude faster than nonlinear gradient descent. Additionally, the error in interaction forces between the model and experiment decreased as pressure increased, with an average error magnitude of 45% and 21% for pressures at the low and high ends of the tested range, respectively.

A CONTINUOUS SPATIAL AND TEMPORAL MATHEMATICAL MODEL FOR ASSESSING THE DISTRIBUTION OF DENGUE IN BRAZIL WITH CONTROL

In this paper, we developed an optimal control of a reaction–diffusion mathematical model, describing the spatial spread of dengue infection. Compartments for human and vector populations are considered in the model, including a compartment for the aquatic phase of mosquitoes. This enabled us to discuss the vertical transmission effects on the spread of the disease in a two-dimensional domain, using demographic data for different scenarios. The model was analyzed, establishing the existence and convergence of the weak solution for the model. The convergence of the numerical scheme to the weak solution was proved. For numerical approximation, we adopted the finite element scheme to solve direct and adjoint state systems. We also used the nonlinear gradient descent method to solve the optimal control problem, where the optimal management of government investment was proposed and leads to more effective dengue fever infection control. These results may help us understand the complex dynamics driven by dengue and assess the public health policies in the control of the disease.

Hardware‐Friendly Stochastic and Adaptive Learning in Memristor Convolutional Neural Networks

Memristors offer great advantages as a new hardware solution for neuromorphic computing due to their fast and energy‐efficient matrix vector multiplication. However, the nonlinear weight updating property of memristors makes it difficult to be trained in a neural network learning process. Several compensation schemes have been proposed to mitigate the updating error caused by nonlinearity; nevertheless, they usually involve complex peripheral circuits design. Herein, stochastic and adaptive learning methods for weight updating are developed, in which the inaccuracy caused by the memristor nonlinearity can be effectively suppressed. In addition, compared with the traditional nonlinear stochastic gradient descent (SGD) updating algorithm or the piecewise linear (PL) method, which are most often used in memristor neural network, the design is more hardware friendly and energy efficient without the consideration of pulse numbers, duration, and directions. Effectiveness of the proposed method is investigated on the training of LeNet‐5 convolutional neural network. High accuracy, about 93.88%, on the Modified National Institute of Standards and Technology handwriting digits datasets is achieved (with typical memristor nonlinearity as ±1), which is close to the network with complex PL method (94.7%) and is higher than the original nonlinear SGD method (90.14%).

3D super-resolution microscopy based on nonlinear gradient descent structured illumination.

Three-dimensional structured illumination microscopy (3D-SIM) is an essential tool for volumetric fluorescence imaging, which improves both axial and lateral resolution by down-modulating high-frequency information of the sample into the passband of optical transfer function (OTF). And when combining with the 4Pi structure, the performance of 3D-SIM can be further improved. The reconstruction results of generally used linear 3D algorithm, however, are lack of high-fidelity and proneess to generate artifacts. In this paper, we proposed a novel iterative algorithm based on gradient descent combined with a nonlinear optimizer, which can be applied to all 3D-SIM setups (including I5S setup). We verified through simulation that the proposed solution, termed as nonlinear gradient descent structured illumination microscopy (NGD-SIM), achieves more fidelity results which can reach the limitation of theoretical resolution improvement of SIM. Moreover, it can be firmly validated on simulation that this algorithm can effectively reduce the amount of raw data in the case of sinusoidal-pattern illumination, i.e., the algorithm doesn't need five-step phase shifting; data with any number of phases can theoretically be reconstructed. Our method also provides the possibility to extend the application of sinusoidal-pattern illumination to any kind of interference fringe, which is generated by diversified types of illumination mode.

Deterministic control of stochastic reaction-diffusion equations

<p style='text-indent:20px;'>We consider the control of semilinear stochastic partial differential equations (SPDEs) via deterministic controls. In the case of multiplicative noise, existence of optimal controls and necessary conditions for optimality are derived. In the case of additive noise, we obtain a representation for the gradient of the cost functional via adjoint calculus. The restriction to deterministic controls and additive noise avoids the necessity of introducing a backward SPDE. Based on this novel representation, we present a probabilistic nonlinear conjugate gradient descent method to approximate the optimal control, and apply our results to the stochastic Schlögl model. We also present some analysis in the case where the optimal control for the stochastic system differs from the optimal control for the deterministic system.</p>

Quantitative X-ray fluorescence imaging of gold nanoparticles using joint L1 and total variation regularized reconstruction.

This work proposed a joint L1 and total variation (TV) regularized reconstruction method for X-ray fluorescence tomography (XFT), and investigated the performance of this method in quantitative imaging of gold nanoparticles (GNPs). We developed a dual-modality XFT/CT imaging system which consisted of a benchtop X-ray source, a translation/rotation stage, a silicon drift detector for X-ray fluorescence (XRF) detection, and a flat panel detector for transmission X-ray detection. A pencil-beam collimator was 3D printed with steel and employed in sample excitation. The sensitivity of the XFT imaging system was determined by imaging water phantoms with multiple inserts containing GNP solutions of various concentrations (0.02-0.16 wt.%). A joint L1 and total variation (TV) regularized algorithm was developed for XFT reconstruction, where the L1 regularization was used to reduce image artifacts and the TV regularization was used to preserve the shape of targets. Nonlinear conjugate gradient (NCG) descent algorithm with backtracking line search was adopted to solve the reconstruction problem. We compared the L1 + TV regularization method with filtered back projection (FBP), maximum likelihood expectation maximization (ML-EM), L1 regularization, and TV regularization methods. Contrast-to-noise ratio (CNR), Dice similarity coefficient (DSC) and localization error (LE) metrics were used to compare the performance of different methods. The CT and XFT imaging doses were also measured using EBT2 radiochromic films. The 3D printed pencil-beam collimator shaped an excitation beam with a 2 mm full width at half maximum at the imaging isocenter. Based on the phantom imaging experiments, the joint L1 and TV regularization method performed better than FBP, ML-EM, L1 regularization and TV regularization methods, with higher localization accuracy (offset <0.6 mm), CNR and DSC values. Compared with CT, XFT with L1 + TV regularized reconstruction demonstrated higher sensitivity in GNP imaging, and could detect GNP at a concentration of 0.02 wt.% or lower. Moreover, there existed a significant linear correlation (R2>0.99) between the reconstructed and true GNP concentration. The estimated XFT imaging dose is about 41.22 cGy under current setting. The joint L1 + TV regularized reconstruction algorithm performed better in noise suppression and shape preservation. Using the L1 + TV regularized reconstruction, the XFT system is able to localize GNP targets with submillimeter accuracy and quantify GNP distribution at a concentration of 0.02 wt.% or lower.

System identification of proportional solenoid valve dynamics.

We present a system identification technique for the characterisation of the linearity and dynamic response of a PSOL valve and its corresponding electronic control unit (ECU) using bandlimited white noise, as well as pseudo random "non-sum non-difference" (NSND) waveforms consisting of mutually prime frequencies to mitigate the effects of nonlinear distortions. The parameters of several transfer function models were simultaneously estimated from the voltage-flow frequency response using a nonlinear gradient descent technique. Candidate transfer function models were assessed using the mean squared residual (MSR) criterion and the corrected Akaike information criterion (AICc). The MSR yielded a transfer function consisting of 10 poles and 9 zeros, while the AICc yielded a simpler transfer function consisting of 5 poles and 3 zeros. Monte Carlo analysis demonstrated fragile stability for the MSR-selected model with respect to varying parameter values within estimated uncertainties, yet a robust stability for the AICc-selected model.

Collaborative Adaptive Filtering Approach for the Identification of Complex-Valued Improper Signals

This paper proposes a novel hybrid filter for data-adaptive optimal identification and modeling of complex-valued real-world signals based on the convex combination approach. It is equipped with different complex domain characteristics of subfilter algorithm. The proposed hybrid filter takes advantage of the complex nonlinear gradient descent (CNGD) algorithm that exhibits fast convergence and the steady state of the augmented complex nonlinear gradient descent (ACNGD) algorithm. The output of CNGD and ACNGD was combined to work in parallel, feeding each individual subfilter output into a mixing algorithm, which in the end produced a single hybrid filter output. The mixing parameter $$ \lambda \left( k \right) $$?k within the hybrid filter architecture was made gradient adaptive in order to preserve the nature of inherent characteristics of the subfilters and to show its optimal performance in identifying and tracking second-order properness (circular) and improperness (noncircular) of the complex signals in real time. Further analysis was made on the properties of the algorithms, and the relationship between fast convergence and steady-state error was discussed. This analysis is supported by the complex-valued synthetic simulation and real-world application dataset as applied in renewable energy (wind).

Prediction of thin asphalt concrete overlay thickness and density using nonlinear optimization of GPR data

The processing of ground-penetration radar (GPR) signals collected from thin asphalt concrete (AC) overlay is a challenging task due to the limitation of GPR signal resolution. In this study, a gradient descent based nonlinear optimization approach was developed to analyze GPR signals collected from thin AC overlays to estimate their thickness and density. Both finite difference time domain (FDTD) simulation and field tests were conducted to validate the proposed algorithm. The simulation showed that the accuracy of dielectric constant estimation increased after the nonlinear gradient descent method was applied. This resulted in a thickness estimation error of less than 1 mm. When nonlinear gradient descent was applied to field test measured signals, the average AC overlay thickness prediction and AC density estimation errors were 3 mm and 1.81%, respectively. This study demonstrates that the nonlinear gradient descent is an effective approach for estimating thin AC overlay thickness and density from GPR data.

Optimal response to non-equilibrium disturbances under truncated Burgers–Hopf dynamics

We model and compute the average response of truncated Burgers–Hopf dynamics to finite perturbations away from the Gibbs equipartition energy spectrum using a dynamical optimization framework recently conceptualized in a series of papers. Non-equilibrium averages are there approximated in terms of geodesic paths in probability space that ‘best-fit’ the Liouvillean dynamics over a family of quasi-equilibrium trial densities. By recasting the geodesic principle as an optimal control problem, we solve numerically for the non-equilibrium responses using an augmented Lagrangian, non-linear conjugate gradient descent method. For moderate perturbations, we find an excellent agreement between the optimal predictions and the direct numerical simulations of the truncated Burgers–Hopf dynamics. In this near-equilibrium regime, we argue that the optimal response theory provides an approximate yet predictive counterpart to fluctuation-dissipation identities.

Direct diffusion tensor estimation using a model-based method with spatial and parametric constraints.

To develop a new model-based method with spatial and parametric constraints (MB-SPC) aimed at accelerating diffusion tensor imaging (DTI) by directly estimating the diffusion tensor from highly undersampled k-space data. The MB-SPC method effectively incorporates the prior information on the joint sparsity of different diffusion-weighted images using an L1-L2 norm and the smoothness of the diffusion tensor using a total variation seminorm. The undersampled k-space datasets were obtained from fully sampled DTI datasets of a simulated phantom and an ex-vivo experimental rat heart with acceleration factors ranging from 2 to 4. The diffusion tensor was directly reconstructed by solving a minimization problem with a nonlinear conjugate gradient descent algorithm. The reconstruction performance was quantitatively assessed using the normalized root mean square error (nRMSE) of the DTI indices. The MB-SPC method achieves acceptable DTI measures at an acceleration factor up to 4. Experimental results demonstrate that the proposed method can estimate the diffusion tensor more accurately than most existing methods operating at higher net acceleration factors. The proposed method can significantly reduce artifact, particularly at higher acceleration factors or lower SNRs. This method can easily be adapted to MR relaxometry parameter mapping and is thus useful in the characterization of biological tissue such as nerves, muscle, and heart tissue.

MODERN METHODS OF BUNDLE ADJUSTMENT ON THE GPU

Abstract. The task to compute 3D reconstructions from large amounts of data has become an active field of research within the last years. Based on an initial estimate provided by structure from motion, bundle adjustment seeks to find a solution that is optimal for all cameras and 3D points. The corresponding nonlinear optimization problem is usually solved by the Levenberg-Marquardt algorithm combined with conjugate gradient descent. While many adaptations and extensions to the classical bundle adjustment approach have been proposed, only few works consider the acceleration potentials of GPU systems. This paper elaborates the possibilities of time and space savings when fitting the implementation strategy to the terms and requirements of realizing a bundler on heterogeneous CPUGPU systems. Instead of focusing on the standard approach of Levenberg-Marquardt optimization alone, nonlinear conjugate gradient descent and alternating resection-intersection are studied as two alternatives. The experiments show that in particular alternating resection-intersection reaches low error rates very fast, but converges to larger error rates than Levenberg-Marquardt. PBA, as one of the current state-of-the-art bundlers, converges slower in 50 % of the test cases and needs 1.5-2 times more memory than the Levenberg- Marquardt implementation.

MODERN METHODS OF BUNDLE ADJUSTMENT ON THE GPU

The task to compute 3D reconstructions from large amounts of data has become an active field of research within the last years. Based on an initial estimate provided by structure from motion, bundle adjustment seeks to find a solution that is optimal for all cameras and 3D points. The corresponding nonlinear optimization problem is usually solved by the Levenberg-Marquardt algorithm combined with conjugate gradient descent. While many adaptations and extensions to the classical bundle adjustment approach have been proposed, only few works consider the acceleration potentials of GPU systems. This paper elaborates the possibilities of time and space savings when fitting the implementation strategy to the terms and requirements of realizing a bundler on heterogeneous CPUGPU systems. Instead of focusing on the standard approach of Levenberg-Marquardt optimization alone, nonlinear conjugate gradient descent and alternating resection-intersection are studied as two alternatives. The experiments show that in particular alternating resection-intersection reaches low error rates very fast, but converges to larger error rates than Levenberg-Marquardt. PBA, as one of the current state-of-the-art bundlers, converges slower in 50&amp;thinsp;% of the test cases and needs 1.5-2 times more memory than the Levenberg- Marquardt implementation.

A Sequential Two-Step Algorithm for Fast Generation of Vehicle Racing Trajectories

The problem of maneuvering a vehicle through a race course in minimum time requires computation of both longitudinal (brake and throttle) and lateral (steering wheel) control inputs. Unfortunately, solving the resulting nonlinear optimal control problem is typically computationally expensive and infeasible for real-time trajectory planning. This paper presents an iterative algorithm that divides the path generation task into two sequential subproblems that are significantly easier to solve. Given an initial path through the race track, the algorithm runs a forward–backward integration scheme to determine the minimum-time longitudinal speed profile, subject to tire friction constraints. With this fixed speed profile, the algorithm updates the vehicle's path by solving a convex optimization problem that minimizes the resulting path curvature while staying within track boundaries and obeying affine, time-varying vehicle dynamics constraints. This two-step process is repeated iteratively until the predicted lap time no longer improves. While providing no guarantees of convergence or a globally optimal solution, the approach performs very well when validated on the Thunderhill Raceway course in Willows, CA. The predicted lap time converges after four to five iterations, with each iteration over the full 4.5 km race course requiring only 30 s of computation time on a laptop computer. The resulting trajectory is experimentally driven at the race circuit with an autonomous Audi TTS test vehicle, and the resulting lap time and racing line are comparable to both a nonlinear gradient descent solution and a trajectory recorded from a professional racecar driver. The experimental results indicate that the proposed method is a viable option for online trajectory planning in the near future.

Parameter Estimation of a DC Motor-Gear-Alternator (MGA) System via Step Response Methodology

Mathematical models and their parameters are essential when designing controllers because they allow the designer to predict the closed loop behavior of the system. An accurate method for estimating the DC Motor-Gear-Alternator (MGA) system parameters is needed before constructing the reliable model. This paper proposed a new method of parameter estimation using Matlab/Simulink parameter estimation tool via Step Response Methodology. Optimization algorithms including the nonlinear least square, Gradient Descent, Simplex Search and Pattern Search are discussed. Simulink Design Optimization automatically estimated parameters of the MGA model from measured input-output data.

Sparsity-based acoustic inversion in cross-sectional multiscale optoacoustic imaging.

With recent advancement in hardware of optoacoustic imaging systems, highly detailed cross-sectional images may be acquired at a single laser shot, thus eliminating motion artifacts. Nonetheless, other sources of artifacts remain due to signal distortion or out-of-plane signals. The purpose of image reconstruction algorithms is to obtain the most accurate images from noisy, distorted projection data. In this paper, the authors use the model-based approach for acoustic inversion, combined with a sparsity-based inversion procedure. Specifically, a cost function is used that includes the L1 norm of the image in sparse representation and a total variation (TV) term. The optimization problem is solved by a numerically efficient implementation of a nonlinear gradient descent algorithm. TV-L1 model-based inversion is tested in the cross section geometry for numerically generated data as well as for in vivo experimental data from an adult mouse. In all cases, model-based TV-L1 inversion showed a better performance over the conventional Tikhonov regularization, TV inversion, and L1 inversion. In the numerical examples, the images reconstructed with TV-L1 inversion were quantitatively more similar to the originating images. In the experimental examples, TV-L1 inversion yielded sharper images and weaker streak artifact. The results herein show that TV-L1 inversion is capable of improving the quality of highly detailed, multiscale optoacoustic images obtained in vivo using cross-sectional imaging systems. As a result of its high fidelity, model-based TV-L1 inversion may be considered as the new standard for image reconstruction in cross-sectional imaging.

Feature-Based Lucas–Kanade and Active Appearance Models

Lucas-Kanade and active appearance models are among the most commonly used methods for image alignment and facial fitting, respectively. They both utilize nonlinear gradient descent, which is usually applied on intensity values. In this paper, we propose the employment of highly descriptive, densely sampled image features for both problems. We show that the strategy of warping the multichannel dense feature image at each iteration is more beneficial than extracting features after warping the intensity image at each iteration. Motivated by this observation, we demonstrate robust and accurate alignment and fitting performance using a variety of powerful feature descriptors. Especially with the employment of histograms of oriented gradient and scale-invariant feature transform features, our method significantly outperforms the current state-of-the-art results on in-the-wild databases.