Network Weights Research Articles

Automatic Metric Search for Few-Shot Learning.

Few-shot learning (FSL) aims to learn a model that can identify unseen classes using only a few training samples from each class. Most of the existing FSL methods adopt a manually predefined metric function to measure the relationship between a sample and a class, which usually require tremendous efforts and domain knowledge. In contrast, we propose a novel model called automatic metric search (Auto-MS), in which an Auto-MS space is designed for automatically searching task-specific metric functions. This allows us to further develop a new searching strategy to facilitate automated FSL. More specifically, by incorporating the episode-training mechanism into the bilevel search strategy, the proposed search strategy can effectively optimize the network weights and structural parameters of the few-shot model. Extensive experiments on the miniImageNet and tieredImageNet datasets demonstrate that the proposed Auto-MS achieves superior performance in FSL problems.

Representations and generalization in artificial and brain neural networks

Humans and animals excel at generalizing from limited data, a capability yet to be fully replicated in artificial intelligence. This perspective investigates generalization in biological and artificial deep neural networks (DNNs), in both in-distribution and out-of-distribution contexts. We introduce two hypotheses: First, the geometric properties of the neural manifolds associated with discrete cognitive entities, such as objects, words, and concepts, are powerful order parameters. They link the neural substrate to the generalization capabilities and provide a unified methodology bridging gaps between neuroscience, machine learning, and cognitive science. We overview recent progress in studying the geometry of neural manifolds, particularly in visual object recognition, and discuss theories connecting manifold dimension and radius to generalization capacity. Second, we suggest that the theory of learning in wide DNNs, especially in the thermodynamic limit, provides mechanistic insights into the learning processes generating desired neural representational geometries and generalization. This includes the role of weight norm regularization, network architecture, and hyper-parameters. We will explore recent advances in this theory and ongoing challenges. We also discuss the dynamics of learning and its relevance to the issue of representational drift in the brain.

Application of Improved Sliding Mode and Artificial Neural Networks in Robot Control

Mobile robots are autonomous devices capable of self-motion, and are utilized in applications ranging from surveillance and logistics to healthcare services and planetary exploration. Precise trajectory tracking is a crucial component in robotic applications. This study introduces the use of improved sliding surfaces and artificial neural networks in controlling mobile robots. An enhanced sliding surface, combined with exponential and hyperbolic tangent approach laws, is employed to mitigate chattering phenomena in sliding mode control. Nonlinear components of the sliding control law are estimated using artificial neural networks. The weights of the neural networks are updated online using a gradient descent algorithm. The stability of the system is demonstrated using Lyapunov theory. Simulation results in MATLAB/Simulink R2024a validate the effectiveness of the proposed method, with rise times of 0.071 s, an overshoot of 0.004%, and steady-state errors approaching zero meters. Settling times were 0.0978 s for the x-axis and 0.0902 s for the y-axis, and chattering exhibited low amplitude and frequency.

GeneWorker: An end-to-end robotic reinforcement learning approach with collaborative generator and worker networks

Reinforcement learning aided by the skill conception exhibits potent capabilities in guiding autonomous agents toward acquiring meaningful behaviors. However, in the current landscape of reinforcement learning, a skill is often merely a rudimentary abstraction of a sequence of primitive actions, serving as a component of the input to policy networks with fixed network parameters. This rigid methodology presents obstacles when attempting to integrate with burgeoning techniques such as meta-learning and large language models. To address this issue, we introduce a unique neural skill representation that abstracts the activation of neurons in each neural layer. Based on this, a novel end-to-end robotic reinforcement learning algorithm is proposed, in which two sub-networks, i.e., generator and worker networks, implement collaborative inferences via neural skills. Specifically, the generator produces a series of multi-spatial neural skills, providing efficient guidance for subsequent decision-making; by integrating these skills, the worker can determine its own network weights and biases to cope with various environmental conditions. Therefore, actions can be sampled with flexibly changeable network parameters through the collaboration between generator and worker networks. The experiments demonstrate that GeneWorker can achieve a mean success rate of over 90.67% on continuous robotic tasks and outperforms previous state-of-the-art methods by a minimum of 54% on the pick-and-place task.

Application of the BP neural network algorithm to the ultimate bearing capacity test of building structures

The ultimate bearing capacity test effect directly influences the safety performance of the design of building structures. To enhance the safety of building structures, applying the BP neural network algorithm to their ultimate bearing capacity test is studied to improve the test effect. The shear wave velocity of the building structure during stress is collected using the static probing technique. The input samples of the BP neural network are the building structure’s shear wave velocity and construction parameters. They are processed by dimensionality reduction through principal component analysis. The firework algorithm is used to optimize the weight of the BP neural network. An early termination training method is designed, and the optimal weight is combined to train the BP neural network. After training, the samples are input after dimensionality reduction, and the building structure’s ultimate bearing capacity test results are output. Experimental results show that this method can effectively collect the shear wave velocity of building structures and complete the dimensionality reduction of samples. Under different coaxial stresses, this method can effectively measure the ultimate bearing capacity, about 3800 kN. After parameter optimization, the test value of this method is very close to the target value; that is, the ultimate bearing capacity test precision is high.

MFSC: A Multimodal Aspect-Level Sentiment Classification Framework with Multi-Image Gate and Fusion Networks

Currently, there is a great deal of interest in multimodal aspect-level sentiment classification using both textual and visual information, which changes the traditional use of only single-modal to identify sentiment polarity. Considering that existing methods could be strengthened in terms of classification accuracy, we conducted a study on aspect-level multimodal sentiment classification with the aim of exploring the interaction between textual and visual features. Specifically, we construct a multimodal aspect-level sentiment classification framework with multi-image gate and fusion networks called MFSC. MFSC consists of four parts, i.e., text feature extraction, visual feature extraction, text feature enhancement, and multi-feature fusion. Firstly, a bidirectional long short-term memory network is adopted to extract the initial text feature. Based on this, a text feature enhancement strategy is designed, which uses text memory network and adaptive weights to extract the final text features. Meanwhile, a multi-image gate method is proposed for fusing features from multiple images and filtering out irrelevant noise. Finally, a text-visual feature fusion method based on an attention mechanism is proposed to better improve the classification performance by capturing the association between text and images. Experimental results show that MFSC has advantages in classification accuracy and macro-F1.

Development of a video encryption algorithm for critical areas using 2D extended Schaffer function map and neural networks

This paper proposes an encryption algorithm for crucial areas of a video based on chaos and a neural network, which SVEA (Selective Video Encryption Algorithm). The critical areas of each frame in a video are extracted by deep learning to the encryption system. A one-step encryption algorithm is used to encrypt these critical areas based on chaos, where scrambling and diffusion are simultaneously performed. A new chaotic system 2D extended Schaffer function map (2D-ESFM) is utilized in the encryption system, inspired by the Schaffer function. The system has demonstrated excellent performance through Lyapunov exponents (LEs), permutation entropy (PE), the 0-1 test, and other methods. Additionally, to resist chosen plaintext attacks, the secret key is generated by a neural network, with the critical areas of the video as inputs to the neural network. The chaotic system generates the biases and weights for the neural network. We evaluate SVEA on our dataset (Gymnastics at the Olympic Games) and public datasets. SVEA exhibits strong security characteristics compared to state-of-the-art algorithms and reduces time complexity by approximately 51.3%.

Improving prediction accuracy of laser-induced shock wave velocity prediction using neural networks

The velocity of laser-induced shock waves affects the efficiency and efficacy of laser-based processes. The ability to accurately estimate shock wave velocity is critical for optimizing experimental combinations, creating laser-based systems, and assuring desired results. Traditional approaches to predict shock wave velocity involve empirical equations and analytical models based on simplified assumptions. However, these methods often lack accuracy and fail to capture the complex dynamics of laser-matter interactions. To overcome these limitations, we used a combination of an artificial neural network and a genetic algorithm to predict shock wave velocity. In this method, the neural network structure is dynamically designed. The optimization method does this by modifying the neural network's weights and figuring out the network's structure on our behalf. Based on the findings, our suggested technique worked very well; it surpassed other comparison methods by achieving the lowest average errors in terms of RMSE and MAE, which are 4.38 and 3.74, respectively. Moreover, the analysis has shown that our proposed method has a high level of reliability in predicting impulsive wave velocity using a neural network.

Robust reinforcement learning control for quadrotor with input delay and uncertainties

In this paper, a new control method is proposed for the control of the quadrotor with time-varying input delay and uncertainties. The controller is primarily composed of two components: the reinforcement learning (RL) component and the robust component. The robust component is designed to ensure basic control performance. The basic control performance can guarantee the tracking characteristics with nonlinear uncertainties. The tracking error can be arbitrarily small by robust component. The RL component further improves the convergence speed and control precision on the basis of the basic control performance. Both components have input delay problems, and each component of the proposed method solves this problem separately. For the robust component, the filter is used to eliminate the uncertainties and restrain the influence of the input delay. It can ensure safety during the initial training period. For the RL component, the extended state method and the extending control period method are used to handle the problem of input delay with lower computational complexity. In addition, the convergence of network weights is proven, and the stability of the whole system is analyzed according to the Lyapunov method. Finally, simulation results are presented to illustrate the effectiveness and superior performance of our method.

Performance Evaluation of Laguerre Transform and Neural Network-based Cryptographic Techniques for Network Security

As the world evolves day by day with new technologies, there is a need to design a secure network in such a way that intruders and unauthorized persons should not have access to the network as well as information regarding the personnel in any firm. In this study, a new cryptographic technique for securing data transmission based on the LaplaceLaguerre polynomial (LLP) is developed and compared to an existing auto-associative neural network technique (AANNT).The performance of the LLPT and AANNT was tested with some selected files in a MATLAB environment and the results obtained provided comparative information (in respect of AANNT versus LLPT) as follows: encryption time (1.67 ms versus 3.9931s), decryption time (1.833 ms versus 2.1172s), throughput (26.2975 Kb/s versus 0.01098 Kb/s), memory consumption (3.349 KB versus 15.958 KB). From the compared results, it shows that AANNT offers a faster processing time, higher throughput, and takes up less memory space than the LLPT. However, cryptanalysis of the AANNT is possible if the network's weight and design are known; hence, the technique is unreliable for ensuring the data integrity and confidentiality of encrypted data. The proposed LLP cryptographic algorithm is designed to provide a higher security level by making the LLP algorithm computationally tedious to invert using the standard Laplace transform inversion method. When compared to the AANN-based cryptographic technique, cracking the algorithm to uncover the encryption key takes time. This shows the strength and robustness of the proposed LLP cryptographic algorithm against attacks, as well as its suitability for solving the problem of data privacy and security when compared to the AANN-based cryptographic algorithm.

An improved log-cosine variation slime mold - simplified gated recurrent neural network for the high-precision state of charge estimation of lithium-ion batteries

Accurate estimation of SOC is crucial for the safe and efficient utilization of electric vehicles, portable devices and renewable energy systems. Neural networks have become a powerful tool in the field of SOC estimation, but they have a complex structure, difficult parameter tuning, and high performance requirements. For this reason, this paper proposes a simplified gated recurrent neural network (SGRN) that greatly simplifies the unitary network structure and parameters. A Nadam gradient optimizer is introduced to update the network weight parameters, and the network hyperparameters are optimized using the log-cosine variant slime mold algorithm (LVSM). The algorithm is validated under HPPC, BBDST, and DST operating conditions under 25 °C, 10 °C, and − 5 °C. The results show that the LVSM-SGRN algorithm has higher accuracy, lower performance consumption, and better temperature adaptability than the conventional LSTM algorithm as well as the EKF algorithm. This contributes to the integration and large-scale industrialization of the algorithm, and comprehensively improves the safety of lithium-ion batteries and accuracy of SOC estimation.

Detecting Logos for Indoor Environmental Perception Using Unsupervised and Few-Shot Learning

Indoor scenes are crucial components of urban spaces, with logos serving as vital information within these environments. The accurate perception of logos is essential for effectively operating mobile robots in indoor environments, which significantly contributes to many upper-level applications. With the rapid development of neural networks, numerous deep-learning-based object-detection methods have been applied to logo detection. However, most of these methods depend on large labeled datasets. Given the fast-changing nature of logos in indoor scenes, achieving reliable detection performance with either the existing large labeled datasets or a limited number of labeled logos remains challenging. In this article, we propose a method named MobileNetV2-YOLOv4-UP, which integrates unsupervised learning with few-shot learning for logo detection. We develop an autoencoder to obtain latent feature representations of logos by pre-training on a public unlabeled logo dataset. Subsequently, we construct a lightweight logo-detection network and embed the encoder weights as prior information. Training is performed on a small dataset of labeled indoor-scene logos to update the weights of the logo-detection network. Experimental results on the public logo625 dataset and our self-collected LOGO2000 dataset demonstrate that our method outperforms classic object-detection methods, achieving a mean average detection precision of 83.8%. Notably, our unsupervised pre-training strategy (UP) has proven effective, delivering a 15.4% improvement.

State estimation and structure identification of complex networks with time‐varying delays and model uncertainty based on probabilistic graph neural network

SummaryIn this paper, the state estimation problem and structure identification problem are investigated for an array of complex networks with time‐varying delays and model uncertainty. Model uncertainty is generated by uncertain connections between nodes in complex networks. Combining probabilistic graph model and graph neural network, a probabilistic graph neural network is proposed to deal with the uncertainty, and the multiple linear terms of the errors between nodes are estimated. The purpose of the addressed state estimation problem is to design a state estimator such that, in the presence of the model uncertainty, the estimation error to converge to zero can be guaranteed and the explicit expression of the estimator parameters is given. Based on a Lyapunov function, the parameters of the estimator and the updating laws of the weights of probabilistic graph neural network can be obtained by the solutions to linear matrix inequalities. The purpose of the addressed structure identification problem is to infer a definite graph structure model based on Bayes' theorem in the presence of a stable probabilistic graph neural network and state estimator. Finally, an illustrative example is provided to demonstrate the feasibility and effectiveness of the developed state estimation and structure identification scheme.

Combinatorics and topological weights of chromatin loop networks.

Polymer physics models suggest that chromatin spontaneously folds into loop networks with transcription units (TUs), such as enhancers and promoters, as anchors. Here we use combinatoric arguments to enumerate the emergent chromatin loop networks, both in the case where TUs are labeled and where they are unlabeled. We then combine these mathematical results with those of computer simulations aimed at finding the inter-TU energy required to form a target loop network. We show that different topologies are vastly different in terms of both their combinatorial weight and energy of formation. We explain the latter result qualitatively by computing the topological weight of a given network-i.e., its partition function in statistical mechanics language-in the approximation where excluded volume interactions are neglected. Our results show that networks featuring local loops are statistically more likely with respect to networks including more nonlocal contacts. We suggest our classification of loop networks, together with our estimate of the combinatorial and topological weight of each network, will be relevant to catalog three-dimensional structures of chromatin fibers around eukaryotic genes, and to estimate their relative frequency in both simulations and experiments.

An iterative neural network approach applied to human-induced force reconstruction using a non-linear electrodynamic shaker

Human-induced force analysis plays an important role across a wide range of disciplines, including biomechanics, sport engineering, health monitoring or structural engineering. Specifically, this paper focuses on the replication of ground reaction forces (GRF) generated by humans during movement. They can provide critical information about human-mechanics and be used to optimize athletic performance, prevent and rehabilitate injuries and assess structural vibrations in engineering applications. It is presented an experimental approach that uses an electrodynamic shaker (APS 400) to replicate GRFs generated by humans during movement, with a high degree of accuracy. Successful force reconstruction implies a high fidelity in signal reproduction with the electrodynamic shaker, which leads to an inverse problem, where a reference signal must be replicated with a nonlinear and non-invertible system. The solution presented in this paper relies on the development of an iterative neural network and an inversion-free approach, which aims to generate the most effective drive signal that minimizes the error between the experimental force signal exerted by the shaker and the reference. After the optimization process, the weights of the neural network are updated to make the shaker behave as desired, achieving excellent results in both time and frequency domains.

Auto-berthing Control for MSVs with a Time-based Generator under Actuator Faults: A Concise Neural Single-Parameter Approach

Abstract In this paper, we study the control problem of auto-berthing marine surface vessels (MSVs) within a predefined, finite time in the restricted waters of a port, in the face of internal and external uncertain dynamics and actuator faults. We first use radial basis function neural networks to reconstruct the internal uncertainties of the system; then, using the minimum learning parameter method, we transform the weights of the neural networks, the external disturbances of the system, and the bias fault factors into an indirect single-parameter neural learning mode. We also apply a robust depth information adaptation technique to estimate the upper bound on the composite disturbances online. Dynamic surface control technology alleviates the burden of virtual control derivative calculations. Finite-time convergence of the system is guaranteed by a predetermined finite-time function based on a time-based generator (TBG). Based on these methods, we design a finite-time fault-tolerant auto-berthing control scheme based on TBG. The stability of the system is analysed based on Lyapunov stability theory. Finally, we verify the effectiveness of the proposed control scheme through simulation.

Robust optimal control for uncertain wheeled mobile robot based on reinforcement learning: ADP approach

This paper presents a robust optimal control approach for the wheel mobile robot system, which considers the effects of external disturbances, uncertainties, and wheel slipping. The proposed method utilizes an adaptive dynamic programming (ADP) technique in conjunction with a disturbance observer. Initially, the system's state space model is formulated through the utilization of kinematic and dynamic models. Subsequently, the ADP method is employed to establish an online adaptive optimal controller, which solely relies on a single neural network for the purpose of function approximation. The utilization of the disturbance observer in conjunction with the compensation controller serves to alleviate the effects of disturbances. The Lyapunov theorem establishes the stability of the complete closed-loop system and the convergence of the weights of the neural network. The proposed approach has been shown to be effective through simulation under the effect of the disturbances and the change of the desired trajectory.

Distributed Optimal Attitude Synchronization Control of Multiple QUAVs via Adaptive Dynamic Programming.

This article proposes a distributed optimal attitude synchronization control strategy for multiple quadrotor unmanned aerial vehicles (QUAVs) through the adaptive dynamic programming (ADP) algorithm. The attitude systems of QUAVs are modeled as affine nominal systems subject to parameter uncertainties and external disturbances. Considering attitude constraints in complex flying environments, a one-to-one mapping technique is utilized to transform the constrained systems into equivalent unconstrained systems. An improved nonquadratic cost function is constructed for each QUAV, which reflects the requirements of robustness and the constraints of control input simultaneously. To overcome the issue that the persistence of excitation (PE) condition is difficult to meet, a novel tuning rule of critic neural network (NN) weights is developed via the concurrent learning (CL) technique. In terms of the Lyapunov stability theorem, the stability of the closed-loop system and the convergence of critic NN weights are proved. Finally, simulation results on multiple QUAVs show the effectiveness of the proposed control strategy.

Dual-stream mutually adaptive quality assessment for authentic distortion image

To address semantic misperception caused by distortion and accurately simulate human perceptual processes, this study proposed a method that utilised dual-stream mutually adaptive feature mapping. To extract complementary features, a dual-stream unsupervised method was used during the training phase. One stream was responsible for the extraction of low-level and global features, whereas the other was dedicated to extracting high-level semantic and positional features. Following the freezing of feature extraction network weights, we proposed a structure that used standard deviation labels to predict the quality distribution. The experimental results obtained from 10 published image quality databases demonstrated the superiority of the proposed algorithm. The algorithm outperformed the majority of mainstream methods when evaluated on authentic distortion databases and exhibited competitive performance on synthetic distortion databases.

An effective weight initialization method for deep learning: Application to satellite image classification

The growing interest in satellite imagery has triggered the need for efficient mechanisms to extract valuable information from these vast data sources, providing deeper insights. Even though deep learning has shown significant progress in satellite image classification. Nevertheless, in the literature, only a few results can be found on weight initialization techniques. These techniques traditionally involve initializing the networks’ weights before training on extensive datasets, distinct from fine-tuning the weights of pre-trained networks. In this study, a novel weight initialization method is proposed in the context of satellite image classification. The proposed weight initialization method is mathematically detailed during the forward and backward passes of the convolutional neural network (CNN) model. Extensive experiments are carried out using six real-world datasets. Comparative analyses with existing weight initialization techniques made on various well-known CNN models reveal that the proposed weight initialization technique outperforms the previous competitive techniques in classification accuracy. The complete code of the proposed technique, along with the obtained results, is available at https://github.com/WadiiBoulila/Weight-Initialization.