Weight Update Research Articles

The Medication Quantification Scale 4.0: An Updated Index Based on Prescribers' Perceptions of the Risk Associated With Chronic Pain Medications

To quantify risks associated with drug utilization in the real world for the treatment of chronic pain (CP), an index called the Medication Quantification Scale (MQS) was developed in 1992 in the United States and last updated in 2003. This study aimed to update, adapt to the contemporary Canadian context, and validate a revised version of the MQS (the MQS-4.0). Step 1: An expert committee adapted the MQS to the Canadian clinical practice context. Step 2: An update of risk weights given to medication subclasses was achieved using a prescriber survey (weights were derived from median 0–10 scores given to each subclass). Step 3: Construct validity of the MQS-4.0 was assessed after applying risk weights to the medication use profile of persons living with CP covered by public drug insurance plan. Thirty-six medication subclasses were included in the MQS-4.0. A total of 207 prescribers (physicians, pharmacists, and nurse practitioners) participated in the perception survey; 10.63% identified as pain specialists. When risk weights were applied to prescription claims (n = 9,122), the MQS-4.0 score was associated (P < .05) with the MQS-III score and variables associated with polypharmacy (eg, Charlson Comorbidity Index, number of prescribers or health care visits). This study provides an updated index intended for adult populations based on prescribers' perceptions of the risk associated with CP medications that can be useful for clinical practice and research among persons living with CP in Canada. It will, however, be relevant to verify whether similar risk weights are obtained in future pain specialist surveys. PerspectiveThe MQS-4.0 is an update of the MQS used for quantifying the risk associated with the use of analgesics/coanalgesics. Adequate psychometrics properties were found.

Lifelong learning‐based multilayer neural network control of nonlinear continuous‐time strict‐feedback systems

AbstractIn this paper, we investigate lifelong learning (LL)‐based tracking control for partially uncertain strict feedback nonlinear systems with state constraints, employing a singular value decomposition (SVD) of the multilayer neural networks (MNNs) activation function based weight tuning scheme. The novel SVD‐based approach extends the MNN weight tuning to layers. A unique online LL method, based on tracking error, is integrated into the MNN weight update laws to counteract catastrophic forgetting. To adeptly address constraints for safety assurances, taking into account the effects caused by disturbances, we utilize a time‐varying barrier Lyapunov function (TBLF) that ensures a uniformly ultimately bounded closed‐loop system. The effectiveness of the proposed safe LL MNN approach is demonstrated through a leader‐follower formation scenario involving unknown kinematics and dynamics. Supporting simulation results of mobile robot formation control are provided, confirming the theoretical findings.

Weight thresholded regularized robust coding for vision sensing-based hand gesture detection in collaborative robotics

Vision sensing-based hand gesture recognition is considered an important contemporary research problem of collaborative robotics in a human-robot coexisting environment. The problem becomes more complex in challenging environments, where a robot needs to correctly recognize a human gesture to perform navigation or some other designated job, even in the presence of poor illumination conditions, occlusion, etc. This work proposes a novel approach of successfully utilizing regularized robust coding (RRC)-based models to solve such hand gesture detection problems in real-life, challenging situations. The RRC model is used for robust regression of a signal or image and is known as an improvement over the classical sparse representation-based classification (SRC) model. In this work, we propose three novel variants of weight thresholding mechanism in conjunction with RRC (named WTRRC algorithms) which essentially employ more error-tolerant concepts of logistic functions for weight update, when the dictionary is formed by real-world, photometrically irregular hand gesture images. Extensive case studies in real-world environments (i) with poor illumination, and (ii) with both poor illumination and occlusions firmly establish superior performances of the proposed WTRRC variants compared to other state-of-the-art algorithms in such collaborative robotics problems.

Section optimization design of UHPC beam bridges based on improved particle swarm optimization

Ultra-high performance concrete has excellent mechanical properties such as ultra-high strength and high durability, and has a broad application prospect in the field of bridge engineering. To make full use of the superior mechanical properties of UHPC and reduce the cost of the bridge, the particle swarm optimization algorithm is used to optimize the structural design of the UHPC beam bridge; Aiming at the problem that the traditional particle swarm optimization algorithm is easy to fall into local optimum, the nonlinear adaptive weight update method is used to improve the particle swarm optimization algorithm. Based on the above-mentioned improved particle swarm optimization algorithm, the optimal design method of ordinary reinforced UHPC beams and prestressed UHPC beams with commonly used highway spans is studied. The reliability of the optimized structural limit state is analyzed by Monte Carlo (MC) importance sampling method. The results show that compared with the traditional particle swarm optimization algorithm, the improved particle swarm optimization algorithm has higher convergence speed and convergence accuracy. The optimal height-span ratio of ordinary reinforced UHPC beams decreases with the increase of the span, and the construction cost gradually increases; The optimal height-span ratio of the prestressed beam decreases first and then increases with the increase of the span, and the construction cost gradually increases. The calculation results of load capacity reliability indexes of optimization results are all higher than the target reliability indexes of similar components stipulated in China’s “Uniform Standard for Structural Reliability Design of Highway Engineering”.

Distributed leader-follower based adaptive consensus control for networked microgrids

Networked microgrids (NMGs) provide a promising solution for accommodating various distributed energy resources (DERs) and enhance its performance. However, the coordinated operation of the system with integration of a large number of DERs is major challenge. Therefore, this article aims to provide a distributed control strategy (DCS) based on a leader-follower framework for the coordination of multiple DERs in NMGs. A fuzzy optimized Recurrent Hopfield Neural Network (F-HNN) designed self-adaptive fractional order proportional integral derivative (FOPID) controller is proposed for distributed frequency control of NMGs. Initially, a Lyapunov-based objective function is derived for weight updation of the proposed network. The fuzzy approach is used to optimize the output of the HNN based on its gradients. The proposed F-HNN based DCS is implemented in MATLAB/Simulink and validated for frequency regulation of NMGs through hardware-in-the-loop simulation (HIL) using OPAL-RT. The results obtained are compared with the conventional FOPID HNN tuned and other classical controls. The self-adaptiveness of the controller is demonstrated for change in renewable power generation. Furthermore, the resiliency of the controller is tested with communication failures and, plug and play operation of MGs. The results obtained showed that the frequency of the NMG system is well regulated within the band of ±0.2 Hz. Also, the transient and steady state performance reveals that the proposed DCS is more significant than other techniques.

Research on the focus of national traditional sports culture inheritance based on weighted function algorithm

Abstract In order to make the traditional national sports can be inherited in a lasting way, this paper focuses on the study of national traditional sports culture inheritance from the perspective of moral education. Firstly, we add the teaching content of traditional ethnic sports in physical education courses; secondly, we use the weight update method and base classifier in the weighting function to improve it. And finally, we use the system to conduct experiments. The results show that the teaching experiment of pearl ball as the teaching content of public physical education courses can better improve students’ performance in 4 qualities, and the improved teaching mode can promote students’ learning attitude towards physical education. From the perspective of cultural inheritance, it is of practical significance to explore the new mode of national traditional sports education in colleges and universities under the new conditions of the times in order to inherit traditional national sports and enrich the cultural connotation of college students.

Neural Network Adaptive Observer design for Nonlinear Systems with Partially and Completely Unknown Dynamics Subject to Variable Sampled and Delay Output Measurement

This paper proposes a novel Neural Network Adaptive Observer (NNAO) for Nonlinear Systems with Partially and Completely Unknown Dynamics (NSPCUD), subject to variable sampled and delayed output. The method involves designing a neural network observer for partially unknown nonlinear systems with sampled and delayed outputs, using a radial basis function (RBF) neural network to approximate the system’s unknown part. A new weight update algorithm is proposed, along with a closed-loop output predictor for coping with variable samples, and a closed-loop integral compensation to handle variable delay. This approach is then extended to cover completely unknown systems as well. Numerical simulations and comparisons between the proposed method and previous methods on autonomous ground vehicle models were conducted to verify the effectiveness of the proposed NNAO.

Voltage-Mode Ferroelectric Synapse for Neuromorphic Computing.

Ferroelectric materials with a modulable polarization extent hold promise for exploring voltage-driven neuromorphic hardware, in which direct current flow can be minimized. Utilizing a single active layer of an insulating ferroelectric polymer, we developed a voltage-mode ferroelectric synapse that can continuously and reversibly update its states. The device states are straightforwardly manifested in the form of variable output voltage, enabling large-scale direct cascading of multiple ferroelectric synapses to build a deep physical neural network. Such a neural network based on potential superposition rather than current flow is analogous to the biological counterpart driven by action potentials in the brain. A high accuracy of over 97% for the simulation of handwritten digit recognition is achieved using the voltage-mode neural network. The controlled ferroelectric polarization, revealed by piezoresponse force microscopy, turns out to be responsible for the synaptic weight updates in the ferroelectric synapses. The present work demonstrates an alternative strategy for the design and construction of emerging artificial neural networks.

Accurate Weight Update in an Electrochemical Random‐Access Memory Based Cross‐Point Array Using Channel‐High Half‐Bias Scheme for Deep Learning Accelerator

AbstractRecently cross‐point arrays of synaptic memory devices have been intensively studied to accelerate deep neural network computations. Among various synaptic devices, electrochemical random‐access memory (ECRAM) is emerging as a promising non‐volatile memory candidate owing to its superior synaptic characteristics. However, an optimized update scheme for a three‐terminal ECRAM‐based cross‐point array is yet to be developed. In this study, a metal‐oxide‐based ECRAM (MO‐ECRAM) shows superior synaptic characteristics and the weight update of devices in the MO‐ECRAM cross‐point array is analyzed using the half‐bias (HB) scheme. Additionally, A channel‐high half‐bias (CHB) scheme is proposed to overcome the degraded selectivity of the weight update caused by the three‐terminal configuration of the ECRAM device. In the CHB scheme, the conductance change in the selected device can be increased considerably by applying a calculated additional voltage to the channel. Using the CHB scheme, parallel and selective updates are successfully performed in a 2 × 2 MO‐ECRAM cross‐point array. Finally, an experimental demonstration of the training algorithm shows the impact of selective updates when using the CHB scheme. This new update scheme is expected to improve training accuracy in ECRAM cross‐point array‐based deep learning accelerators.

All-Solid-State Synaptic Transistors with Lithium-Ion-Based Electrolytes for Linear Weight Mapping and Update in Neuromorphic Computing Systems.

Neuromorphic computing, an innovative technology inspired by the human brain, has attracted increasing attention as a promising technology for the development of artificial intelligence systems. This study proposes synaptic transistors with a Li1-xAlxTi2-x(PO4)3 (LATP) layer to analyze the conductance modulation linearity, which is essential for weight mapping and updating during on-chip learning processes. The high ionic conductivity of the LATP electrolyte provides a large hysteresis window and enables linear weight update in synaptic devices. The results demonstrate that optimizing the LATP layer thickness improves the conductance modulation and linearity of synaptic transistors during potentiation and degradation. A 20 nm-thick LATP layer results in the most nonlinear depression (αd = -6.59), whereas a 100 nm-thick LATP layer results in the smallest nonlinearity (αd = -2.22). Additionally, a device with the optimal 100 nm-thick LATP layer exhibits the highest average recognition accuracy of 94.8% and the smallest fluctuation, indicating that the linearity characteristics of a device play a crucial role in weight update during learning and can significantly affect the recognition accuracy.

Linear and symmetric synaptic weight update characteristics in van der Waals heterostructure transistors based on 2D In4/3P2Se6 barrier layer

Two-dimensional van der Waals heterostructures with fast-switching speed and ultra-low power consumption are promising building blocks for artificial synaptic devices (ASD) that aim to solve problems faced by traditional computing. However, the nonlinearity and asymmetry of synaptic weight updates in artificial synapses have hindered the achievement of high image classification accuracy in most reported neuromorphic networks. Here, we develop a nonvolatile MoS2/In4/3P2Se6/graphene ASD with excellent performance. Due to the intrinsic low conduction band offset (0.86 eV) of MoS2/In4/3P2Se6, the vdW heterostructure device exhibits an ultra-large storage window of 71.2 V at ±40 V back-gate sweep, a low nonlinear factor of 0.014/0.042, and a low asymmetry factor at 0.028. Additionally, the device exhibits synaptic behaviors including excitatory postsynaptic current, pair-pulse facilitation, and important “learning-experience” synaptic function under electrical and optical stimulation. This work indicates that In4/3P2Se6 provides remarkable opportunities for band alignment-related applications and can facilitate ongoing research of neuromorphic functional electronics.

Integrated photonic reservoir computing with an all-optical readout.

Integrated photonic reservoir computing has been demonstrated to be able to tackle different problems because of its neural network nature. A key advantage of photonic reservoir computing over other neuromorphic paradigms is its straightforward readout system, which facilitates both rapid training and robust, fabrication variation-insensitive photonic integrated hardware implementation for real-time processing. We present our recent development of a fully-optical, coherent photonic reservoir chip integrated with an optical readout system, capitalizing on these benefits. Alongside the integrated system, we also demonstrate a weight update strategy that is suitable for the integrated optical readout hardware. Using this online training scheme, we successfully solved 3-bit header recognition and delayed XOR tasks at 20 Gbps in real-time, all within the optical domain without excess delays.

ETA: An Efficient Training Accelerator for DNNs Based on Hardware-Algorithm Co-Optimization.

Recently, the efficient training of deep neural networks (DNNs) on resource-constrained platforms has attracted increasing attention for protecting user privacy. However, it is still a severe challenge since the DNN training involves intensive computations and a large amount of data access. To deal with these issues, in this work, we implement an efficient training accelerator (ETA) on field-programmable gate array (FPGA) by adopting a hardware-algorithm co-optimization approach. A novel training scheme is proposed to effectively train DNNs using 8-bit precision with arbitrary batch sizes, in which a compact but powerful data format and a hardware-oriented normalization layer are introduced. Thus the computational complexity and memory accesses are significantly reduced. In the ETA, a reconfigurable processing element (PE) is designed to support various computational patterns during training while avoiding redundant calculations from nonunit-stride convolutional layers. With a flexible network-on-chip (NoC) and a hierarchical PE array, computational parallelism and data reuse can be fully exploited, and memory accesses are further reduced. In addition, a unified computing core is developed to execute auxiliary layers such as normalization and weight update (WU), which works in a time-multiplexed manner and consumes only a small amount of hardware resources. The experiments show that our training scheme achieves the state-of-the-art accuracy across multiple models, including CIFAR-VGG16, CIFAR-ResNet20, CIFAR-InceptionV3, ResNet18, and ResNet50. Evaluated on three networks (CIFAR-VGG16, CIFAR-ResNet20, and ResNet18), our ETA on Xilinx VC709 FPGA achieves 610.98, 658.64, and 811.24 GOPS in terms of throughput, respectively. Compared with the prior art, our design demonstrates a speedup of 3.65x and an energy efficiency improvement of 8.54x on CIFAR-ResNet20.

BP-SRM: A directly training algorithm for spiking neural network constructed by spike response model

Spiking neural networks (SNNs) have attracted widespread attention due to their unique bio-interpretability and low-power properties, but the non-differentiability of discrete spike sequences fired by spiking neurons brings difficulties to the learning of SNNs. Recently, surrogate gradient (SG) and back-propagation through time (BPTT) have provided an excellent idea for training SNNs constructed by leaky integrate-and-fire (LIF) neuron models. The LIF neuron model has been widely used in previous SNNs due to its simplicity and low computational cost, however, this also limits its simulation of biological neuron dynamics, reducing the biological interpretability of SNNs. In this paper, we generalize SG and BPTT to SNNs constructed by spike response model (SRM) and propose the BP-SRM algorithm. Specifically, we address why BPTT succeeded in LIF neurons-based SNNs but failed in SRM neurons-based SNNs in previous researches. Then we establish an iterative form of the SRM neuron model by selecting different state variables. Based on iterable SRM, we get the spatiotemporal dependencies between the state variables of SRM neurons-based SNNs, which allows us to derive the gradient and update weights in SNN by BP-SRM. Then we design temporal channel normalization for BP-SRM and verify the performance of the SNNs on static image dataset, dynamic image dataset and engineering dataset, including Fashion-MNIST, Weather Dataset, N-MNIST, American Sign Language Dataset and bearing fault diagnosis dataset. The experiment results indicate that BP-SRM achieves the state-of-the-art performance of SNNs.

A variable step size total least squares affine-projection-like algorithm: Formula derivation and performance analysis

In the errors-in-variables (EIV) model, the steady state characteristics of the affine-projection-like (APL) algorithm are poor. Hence, the total least squares APL (TLS-APL) algorithm different from the bias-compensated is proposed, which can reduce the adverse influence of input and output noise on weight update. Then, the local extremum of the TLS-APL algorithm is verified, that is, the expected parameter is the optimal value of the iterative estimation. Moreover, the mean, mean square, and steady-state mean square deviation (MSD) performance of the proposed TLS-APL algorithm are deduced by using the Kronecker product and vec operator, respectively. To balance the convergence characteristics and steady-state characteristics, the step size optimization of the TLS-APL algorithm, namely the variable step size TLS-APL (VSS-TLS-APL) algorithm, is derived from the analysis of MSD. Considering the existence of gradient errors, the fitting effects of theoretical steady-state MSD curves and actual steady-state MSD curves under different parameters are analyzed separately, which proves the rationality of theoretical derivation. In addition, the VSS-TLS-APL algorithm with optimized step size is compared with the TLS-APL algorithm with fixed step size in the application of system identification, and the effect of the variable step size (VSS) is verified. The experiments of system identification and acoustic echo cancelation prove that the VSS-TLS-APL algorithm is superior to the existing known algorithms.

Effects of Mg Doping to a LiCoO2 Channel on the Synaptic Plasticity of Li Ion-Gated Transistors.

Artificial synapses with ideal functionalities are essential in hardware neural networks to allow for energy-efficient analog computing. However, the realization of linear and symmetric weight updates in real synaptic devices has proven challenging and ultimately limits the online training capabilities of neural network systems. Herein, we investigate the effect of Mg doping on a LiCoO2 (LCO) channel in a Li ion-gated synaptic transistor, so as to improve long-term and short-term plasticity. Two transistor structures, based on a lithium phosphorus oxynitride electrolyte, were examined by using undoped LCO and Mg-doped LCO as the channel material between the source and drain electrodes. It was found that Mg doping increased the initial channel conductance by 3 orders of magnitude, which is probably due to the substitution of Co3+ by Mg2+ and the compensation of hole creation. It was further found that the doped channel transistor showed good retention characteristics and better linearity of long-term potentiation and depression when voltage pulses were applied to the gate electrode. The improved retention and linearity are attributed to an extended range of the insulator-to-conductor transition by Mg doping and Li-ion extraction/insertion cooperated in the LCO channel. Using the obtained synaptic weight update, artificial neural network simulations demonstrated that the doped channel transistor shows an image recognition accuracy of ∼80% for handwritten digits, which is higher than ∼65% exhibited by the undoped channel transistor. Mg doping also improved short-term plasticity such as paired-pulse facilitation/depression and Hebbian spike timing-dependent plasticity. These results indicate that elemental doping to the channel of Li ion-gated synaptic transistors could be a useful procedure for realizing robust neuromorphic systems based on analog computing.

Research on multi-layer network routing selection strategy based on cooperative evolutionary game in IoT environment

As a new technology and application mode, the Internet of Things has an important impact on social life and economic development. In recent years, low-cost optimization of network transmission to solve the congestion problem of multi-layer communication networks has become one of the research hotspots. In this paper, a multi-dimensional communication data transmission model based on a multi-layer network is proposed. It then uses cooperative evolutionary game theory to calculate revenue, update weights, and adapt neighbors. Finally, the attention mechanism is dynamically introduced to share the weights of the multi-layer network, and the multi-dimensional communication propagation and routing strategies in the Internet of Things are studied and analyzed. The experimental results show that the model proposed in this paper has higher game revenue and application value than traditional single-layer network game theory. In particular, the indicators of cooperation rate, stable state, and maximum cooperation rate are better than the latter. The research results of this paper have important reference value for solving the problems of cooperation dilemma, social stickiness, and synergy in multi-layer networks.

Adaptive neural network fixed-time sliding mode control for trajectory tracking of underwater vehicle

The effectiveness of Autonomous Underwater Vehicles (AUVs) in diverse underwater tasks is heavily reliant on their ability to perform accurate trajectory tracking. However, due to uncertainties in AUVs modeling and the complex underwater environment disturbances, designing effective trajectory tracking controllers and disturbance observers for AUVs is still a major challenge. To address these uncertainties and enable faster convergence of tracking errors, a trajectory-tracking controller based on fixed-time sliding mode control (FTSMC) and a Radial Basis function neural network (RBFNN) observer are used in this paper. In most cases, the AUV platform carries limited computational resources. In most cases, AUV platforms carry limited computational resources, which restricts the practical use of online neural network methods, and it is particularly important to reduce the complexity of computational neural networks and enhance the real-time performance of the observer. Therefore, we adopted a fast online weight update strategy based on a single parameter. Considering actuator faults and input saturation, passive fault-tolerant control (PFTC) is used in this scheme to further reduce the computational burden. Furthermore, the Lyapunov method is used to demonstrate the fixed-time stability of the individual signals of the system. Finally, simulation results and theoretical analysis demonstrate the superiority and effectiveness of the proposed method.

Power Optimized VLSI Architecture of Distributed Arithmetic Based Block LMS Adaptive Filter

In this paper, we are presenting a power-efficient Distributed Arithmetic (DA) based Block Least Mean Square (BLMS) Adaptive Digital Filter (ADF). The proposed DA BLMS architecture proposes a shared area-efficient Multiplier Accumulate Block that calculates both the partial filter products and the weight increment terms in the same module. It also uses Multiplexers (MUX) and Demultiplexers (DEMUX) which passes only L out of N inputs, where N and L are the filter length and chosen block size respectively, into the MAC thus helping in achieving the DA functionality along with reduced power consumption. Also, efficient truncation of the obtained error and weight update terms is performed by being able to select the non-zero-bit part of the signal to be fed back. The entire architecture is driven by a single slow clock which reduces the power consumption of the device further. On comparing with the best existing DA BLMS Structures, the proposed architecture uses 15% lesser power, 14% lesser EPS according to ASIC Synthesis, and for a filter length of N=16 and a block size of L=4 respectively.

Two-player zero-sum game based neural critic tracking control for UAUV with unknown disturbance via backstepping method

In this work, a new tracking control scheme is developed for underactuated autonomous underwater vehicle(UAUV) with unknown disturbance. First, an error tracking system is constructed using backstepping method, which transforms the tracking control problem into the two-player zero-sum game problem. One player is the optimal controller which is designed to make the UAUV track the desired trajectory. The other player is the unknown disturbance that represent the overall damping effect of the UAUV. Then, the single critic network based online policy iteration algorithm is designed to get the optimal control law and the worst-case disturbance, which can achieve the near-optimal control performance and relax the requirements for initial admissible control conditions. In order to improve the converge velocity of tracking error, the weight update law are designed. Furthermore, a discount coefficient is introduced into the performance index due to the nonlinearity and the complexity of UAUV. In addition, the stability of the UAUV is analyzed based on the Lyapunov stability theory the system is guaranteed to be uniformly ultimately bounded (UUB). Finally, the effectiveness of the proposed method is demonstrated through the real-time simulations on the UAUV model.