Update Rule Research Articles

Uncertainty handling in convolutional neural networks.

The performance of convolutional neural networks is degraded by noisy data, especially in the test phase. To address this challenge, a new convolutional neural network structure with data indeterminacy handling in the neutrosophic (NS) domain, named as Neutrosophic Convolutional Neural Networks, is proposed for image classification. For this task, images are firstly mapped from the pixel domain to three sets true (T), indeterminacy (I) and false (F) in NS domain by the proposed method. Then, NCNN with two parallel paths, one with the input of T and another with I, is constructed followed by an appropriate combination of paths to generate the final output. Here, two paths are trained simultaneously, and neural network weights are updated using back propagation algorithm. The effectiveness of NCNN to handle noisy data is analyzed mathematically in terms of the weights update rule. Proposed two paths NS idea is applied to two basic models: CNN and VGG-Net to construct NCNN and NVGG-Net, respectively. The proposed method has been evaluated on MNIST, CIFAR-10 and CIFAR-100 datasets contaminated with 20 levels of Gaussian noise. Results show that two-path NCNN outperforms CNN by 5.11% and 2.21% in 5 pairs (training, test) with different levels of noise on MNIST and CIFAR-10 datasets, respectively. Finally, NVGG-Net increases the accuracy by 3.09% and 2.57% compared to VGG-Net on CIFAR-10 and CIFAR-100 datasets, respectively.

Non-Bernoulli operations of stochastic binary synapses using resistive switching devices for unsupervised learning in a spiking neural network

We study the performance of unsupervised learning using a stochastic synaptic update rule in a spiking neural network with binary synapses. For the binary synapse, we propose a circuit that employs a pair of resistive switching devices with switching properties described by a Weibull distribution. Through simulations, we find that the learning performance is much better with a shape parameter larger than 1, in which case the synaptic operations are regarded as non-Bernoulli stochastic trials, than with a simple Bernoulli model, and that the performance is even better than that achieved using a deterministic rule with continuous synaptic weights.

The Effects of Interaction Functions between Two Cellular Automata

Biological systems are notorious for complex behavior within short timescales (e.g., metabolic activity) and longer timescales (e.g., evolutionary selection), along with their complex spatial organization. Because of their complexity and their ability to innovate with respect to their environment, living systems are considered to be open-ended. Historically, it has been difficult to model open-ended evolution and innovation. As a result, our understanding of the exact mechanisms that distinguish open-ended living systems from nonliving ones is limited. One of the biggest barriers is understanding how multiple, complex parts within a single system interact and contribute to the complex, emergent behavior of the system as a whole. How do interactions between parts of a system lead to more complex behavior of the system as a whole? This paper presents two interacting cellular automata (CAs) as an abstract model to address the effects of complex interactions between two individual entities embedded within a larger system. Unlike elementary CAs, each cellular automaton (CA) changes its update rules as a function of the system’s state as a whole. The resulting behavior of the two-CA system suggests that complex interaction functions between the two CAs have little to no effect on the complexity of each individual CA’s behavior and structure. However, having an interaction function that is random results in open-ended evolution regardless of the specific type of state-dependency.

Efficient Bayesian phase estimation via entropy-based sampling

Bayesian estimation approaches, which are capable of combining the information of experimental data from different likelihood functions to achieve high precisions, have been widely used in phase estimation via introducing a controllable auxiliary phase. Here, we present a Bayesian phase estimation (BPE) algorithm with an ingenious update rule of the auxiliary phase designed via entropy-based sampling. Unlike other adaptive BPE algorithms, the auxiliary phase in our algorithm is determined only once in a pre-estimation step. With simple statistical analysis on a small batch of data, an iteration rule for the auxiliary phase is pre-established and used in all afterward updates, instead of complex calculations in every update trails. During this pre-estimation process the most informative data can be selected, which guides one to perform the BPE with much less measurement times. As the measurement times for the same amount of Bayesian updates is significantly reduced, our algorithm via entropy-based sampling can work as efficient as other adaptive BPE algorithms and shares the advantages (such as wide dynamic range and perfect noise robustness) of non-adaptive BPE algorithms. Our algorithm is of promising applications in various practical quantum sensors such as atomic clocks and quantum magnetometers.

Severe damage recovery in evolving soft robots through differentiable programming

Biological systems are very robust to morphological damage, but artificial systems (robots) are currently not. In this paper we present a system based on neural cellular automata, in which locomoting robots are evolved and then given the ability to regenerate their morphology from damage through gradient-based training. Our approach thus combines the benefits of evolution to discover a wide range of different robot morphologies, with the efficiency of supervised training for robustness through differentiable update rules. The resulting neural cellular automata are able to grow virtual robots capable of regaining more than 80\% of their functionality, even after severe types of morphological damage.

Node Localization Algorithm Based on Modified Archimedes Optimization Algorithm in Wireless Sensor Networks

Node localization information plays an important role in wireless sensor networks (WSNs). To solve the problem of low localization accuracy of distance vector hop (DV-Hop) localization algorithm in wireless sensor networks, an improved localization algorithm called MAOADV-Hop based on the modified Archimedes optimization algorithm (MAOA) and DV-Hop is proposed, which can achieve the balance between the localization speed and the localization precision. Firstly, tent chaotic mapping and particle swarm optimization (PSO) algorithm are introduced into Archimedes optimization algorithm to improve the initial population diversity and change the update rules of density and volume, which improve the global convergence ability and convergence speed of the algorithm. Secondly, the MAOA is used to replace the least square part of the DV-Hop localization algorithm to improve the localization accuracy of the algorithm. Finally, MAOADV-Hop is verified through four different network environments and compared with DE_DV-Hop, BOA_DV-Hop, and DV-Hop. The simulation results show that the localization speed of the proposed approach is faster than that of DE_DV-Hop and BOA_DV-Hop, and the localization error is less than that of DV-Hop, DE_DV-Hop, and BOA_DV-Hop.

A new robust fuzzy clustering framework considering different data weights in different clusters

In conventional fuzzy C-means clustering algorithms, each data and each feature are treated equally, the clustering performance is sensitive to the noise points; in existing weighting clustering algorithms, few studies have focus on data weighting and feature weighting simultaneously, besides, the same data in different clusters is treated equally. To address this issue, in this paper, taking the different data weights in different clusters and the different feature weights in different clusters into consideration, we present a new robust fuzzy C-means clustering framework. For the first time, we propose a whole new idea that the same data in different clusters should have different importance, the different data in a cluster should have different importance, as well; By the new data weighting method, the proposed clustering algorithm can weaken the impact of noise points on the formation of each clustering center, which could enhance the robustness of clustering; to stimulate more data and more features to take part in the process of clustering and to avoid overfitting, we add l2-norm regularization of the data weights and l2-norm regularization of the feature weights to the objective function. Then, based on the presented objective function, we get the scientific update rules of the different data weights in different clusters, the different feature weights in different clusters, the membership degrees, and the cluster centers, during each iteration. To assess the performance of the new fuzzy C-means framework, experimental verifications on synthetic dataset and real-world datasets are conducted, experimental results have shown that the new algorithm can achieve better clustering performances in comparison to other related clustering methods.

Роль дезинфекционных мероприятий в профилактике инфекций, связанных с оказанием медицинской помощи (по материалам новых нормативных и методических документов)

The article considers the main state and regulatory documents of sanitary and epidemiological legislation related to ISMP. The issues of updating a number of sanitary and epidemiological requirements for medical organizations, which were subsequently included in the updated sanitary and epidemiological rules currently in force, were considered. Keywords: infections associated with the provision of medical care (ISMP), biological threat, epidemiological process of ISMP, prevention of ISMP, regulatory documents on ISMP

Parallel bandit architecture based on laser chaos for reinforcement learning

Accelerating artificial intelligence by photonics is an active field of study aiming to exploit the unique properties of photons. Reinforcement learning is an important branch of machine learning, and photonic decision-making principles have been demonstrated with respect to the multi-armed bandit problems. However, reinforcement learning could involve a massive number of states, unlike previously demonstrated bandit problems where the number of states is only one. Q-learning is a well-known approach in reinforcement learning that can deal with many states. The architecture of Q-learning, however, does not fit well photonic implementations due to its separation of update rule and the action selection. In this study, we organize a new architecture for multi-state reinforcement learning as a parallel array of bandit problems in order to benefit from photonic decision-makers, which we call parallel bandit architecture for reinforcement learning or PBRL in short. Taking a cart-pole balancing problem as an instance, we demonstrate that PBRL adapts to the environment in fewer time steps than Q-learning. Furthermore, PBRL yields faster adaptation when operated with a chaotic laser time series than the case with uniformly distributed pseudorandom numbers where the autocorrelation inherent in the laser chaos provides a positive effect. We also find that the variety of states that the system undergoes during the learning phase exhibits completely different properties between PBRL and Q-learning. The insights obtained through the present study are also beneficial for existing computing platforms, not just photonic realizations, in accelerating performances by the PBRL algorithms and correlated random sequences.

Streaming Approach to In Situ Selection of Key Time Steps for Time‐Varying Volume Data

AbstractKey time steps selection, i.e., selecting a subset of most representative time steps, is essential for effective and efficient scientific visualization of large time‐varying volume data. In particular, as computer simulations continue to grow in size and complexity, they often generate output that exceeds both the available storage capacity and bandwidth for transferring results to storage, making it indispensable to save only a subset of time steps. At the same time, this subset must be chosen so that it is highly representative, to facilitate post‐processing and reconstruction with high fidelity. The key time steps selection problem is especially challenging in the in situ setting, where we can only process data in one pass in an online streaming fashion, using a small amount of main memory and fast computation. In this paper, we formulate the problem as that of optimal piece‐wise linear interpolation. We first apply a method from numerical linear algebra to compute linear interpolation solutions and their errors in an online streaming fashion. Using that method as a building block, we can obtain a global optimal solution for the piece‐wise linear interpolation problem via a standard dynamic programming (DP) algorithm. However, this approach needs to process the time steps in multiple passes and is too slow for the in situ setting. To address this issue, we introduce a novel approximation algorithm, which processes time steps in one pass in an online streaming fashion, with very efficient computing time and main memory space both in theory and in practice. The algorithm is suitable for the in situ setting. Moreover, we prove that our algorithm, which is based on a greedy update rule, has strong theoretical guarantees on the approximation quality and the number of time steps stored. To the best of our knowledge, this is the first algorithm suitable for in situ key time steps selection with such theoretical guarantees, and is the main contribution of this paper. Experiments demonstrate the efficacy of our new techniques.

Semi-blind hyperspectral and multispectral image fusion based on a non-factorization model

Fusion of low-resolution hyperspectral images (LR-HSI) with high-resolution multispectral images (HR-MSI) is currently the most cost-effective method for acquiring high-resolution hyperspectral images (HR-HSI). The existing fusion models like spectral unmixing, matrix-factorization, and tensor-factorization require decomposing the target HR-HSI into multiple factor matrices or tensors before fusion. Usually requires a large number of variable optimization processes. So, the above approaches do not apply to the arguably real-time scenario. Moreover, the miscellaneous intermediate estimates render poor fusion stability. This paper proposes a novel non-factorization model to address the above problems. The model firstly gets access to a rough estimate of HR-HSI through the Moore–Penrose inverse operation. Then, we acquire the further HR-HSI forecast according to the correlation between the target HR-HSI and LR-HSI singular values through the coupled singular value decomposition (SVD) of the rough HR-HSI and LR-HIS. Finally, we obtain the precise HR-HSI estimates via the multiplicative update rule. Our proposed model directly estimates the target HR-HSI, reduces the estimation process of intermediate variables, effectively saves calculation time, and enhances the stability of the fusion. We conduct experiments on remote sensing, ground-based, and real datasets to validate our proposed model’s real-time and stability.

Aspergillus niger</em> Fungus Detection using Transfer Learning Technique and Modified Backpropagation Algorithm with Inertia and Legendre Polynomial

Looking at the loss due to health problems from fungal diseases in one hand and the benefits from its industrial/agricultural use, rapid automated fungal species identification is the need of the hour. Hence, proposed a fast identification of fungal species by a 15 minutes staining procedure followed by an artificial-intelligence-based image classification technique. In this modern era, deep architectures have shown a significant performance on computer vision problems. Instead of developing a new model from scratch, the pre-trained convolutional neural network models are available to obtain the appropriate features from input samples using the transfer-learning technique. This work utilizes the transfer-learning approach for feature extraction and classification performed using the proposed modified third-term Backpropagation (BP) algorithm. This proposed algorithm contains Inertia as a third factor in the weight updation rule expanded in the form of the Legendre polynomial to overcome the limitations of the traditional Backpropagation algorithm. The effectiveness of the proposed classifier compared to the results of the existing cutting-edge algorithms namely, Backpropagation algorithm, Backpropagation algorithm using Momentum, and softmax classifier. Compare to the existing models, the proposed model scored a high testing accuracy of 97.27%.

Generalised non-negative matrix factorisation for air pollution source apportionment

Source Apportionment (SA) techniques are widely used for identifying key sources of air pollution, thereby providing critical inputs for policy measures. Positive Matrix Factorisation (PMF) (Paatero and Tapper, 1994) is a widely used SA technique. PMF uses the speciated concentration data (X) collected over several days and factorises it into source contribution (G) and source profile (F) matrices, albeit under positivity constraint. Towards this end, it involves solving an optimisation problem where the elements of X are weighted by the inverse of the standard deviations of the corresponding errors introduced during the sampling and chemical analysis process. Thus, PMF implicitly assumes that the errors in different elements of the X matrix are uncorrelated. This assumption may not hold since the sampling, and chemical analysis steps deployed in any data-collection campaign will inevitably lead to correlated errors. While there are other existing Non-Negative Matrix Factorisation (NMF) methods in literature that can be potentially used for SA, these also make various restrictive assumptions about the error covariance structure. In this work, we propose a new method called Generalised Non-Negative Matrix Factorisation (GNMF) to fill this gap. In particular, the proposed method is able to incorporate any error covariance matrix without making any restrictive assumptions on its structure. Towards this end, we integrate the full error covariance matrix in the objective function to be minimised to obtain F and G matrices. We derive the corresponding update rules for obtaining these matrices iteratively. To ensure non-negativity, we extend the multiplicative and projected gradient-based ideas available in NMF literature to the proposed GNMF approach. The proposed method subsumes various NMF methods available in literature as special cases. The utility of the proposed approach is demonstrated by comparing its performance with other methods on an SA problem using a dataset derived from field measurements.

Fast multiplicative algorithms for symmetric nonnegative tensor factorization

Symmetric nonnegative tensor factorization (SNTF) is an important tool for clustering analysis. To date, most of algorithms for SNTF are based on multiplicative update rules, which have many attractive properties, e.g., they are often simple to implement and can enforce nonnegativity without extra projection steps. However, the existing multiplicative algorithms often converge slowly due to the conservative multiplicative learning steps or the use of low-level BLAS (basic linear algebra subprograms) in their implementation. In this paper, three new multiplicative algorithms are proposed for SNTF to overcome the drawback of slow convergence. First, a parallel multiplicative algorithm, which can be implemented with high-level BLAS, is derived by auxiliary optimization. To further accelerate the convergence, two new parallel multiplicative algorithms, which enable larger learning steps for improving efficiency, are developed based on weighted geometric mean and weighted arithmetic mean, respectively. Finally, we apply the proposed algorithms to multiway probabilistic clustering, where a new hyper-stochastic normalization scheme based on Euclidean distance is developed for better data preprocessing. The experiment results on both synthetic and real-world data show that the proposed SNTF algorithms converge faster than the state-of-the-art algorithms.

Analytical investigation on totally asymmetric simple exclusion process with Langmuir kinetics and a parallel update with two sub-steps

Motivated by the fact that parallel update rules need to be employed to study the traffic dynamic of vehicles and pedestrians with bulk on-off ramps. This paper studies the totally asymmetric simple exclusion process (TASEP) with Langmuir kinetics and a parallel update with two sub-steps. Moreover, this new model can be analyzed analytically. Based on the mean-field method, the phase diagrams are obtained and the influences of model parameters are investigated. Compared to the system under the random update, there do not exist the phases related to the maximum current phase and Meissner phase in the system under the parallel update.

Pattern Training, Inference, and Regeneration Demonstration Using On‐Chip Trainable Neuromorphic Chips for Spiking Restricted Boltzmann Machine

A fully silicon‐integrated restricted Boltzmann machine (RBM) with an event‐driven contrastive divergence (eCD) training algorithm is implemented using novel stochastic leaky integrate‐and‐fire (LIF) neuron circuits and six‐transistor/2‐PCM‐resistor (6T2R) synaptic unit cells on 90 nm CMOS technology. To elaborate, designed a bidirectional, asynchronous, and parallel pulse‐signaling scheme over an analog‐weighted phase‐change memory (PCM) synapse array to enable spike‐timing‐dependent plasticity (STDP) as a local weight update rule based on eCD is designed. Building upon the initial version of this work, significantly more experimental details are added, such as the on‐chip characterization results of LIF and backward‐LIF (BLIF) and stochasticity of our random walk circuitry. The experimental characterization of these on‐chip stochastic neuron circuits shows a reasonable symmetricity between LIF and BLIF as well as the necessary stochasticity for spiking RBM operation. Fully hardware‐based image classification recorded 93% on‐chip training accuracy from 100 handwritten MNIST digit images. In addition, we experimentally demonstrated the generative characteristics of the RBM by reconstructing partial patterns on hardware. As each synapse and neuron execute its computations in an asynchronous and fully parallel fashion, the chip can perform data‐intensive machine learning (ML) tasks in a power‐efficient manner and take advantage of the sparseness of spiking.

A stability-enhanced CycleGAN for effective domain transformation of unpaired ultrasound images

As a cost-effective, non-invasive and radiation-free medical imaging modality, ultrasonic imaging is widely used in clinical diagnosis. However, database bias is commonplace among different medical centers. Deep learning based ultrasound image analysis algorithms are usually data driven, rendering high requirements on the uniformity of image datasets. In this paper, we propose a stability-enhanced cycle-consistent generative adversarial network (CycleGAN) method with well detail preservation for the domain transformation to normalize ultrasound images from various medical centers. To stabilize the training process of CycleGAN model, we adopt spectral normalization for 1-Lipschitz continuity to reduce model oscillation. Besides, two time-scale update rule and label smoothing strategy are also utilized to maintain the balance between generators and discriminators for further stability enhancement. Moreover, our method applies skip connections to preserve ultrasound image details and prevent resolution loss during the domain transformation process. Experiments were conducted on clinical thyroid and carotid image datasets acquired from several medical centers. Massive results demonstrate that our proposed model is easier to reach a steady state when training, outstanding 50% from the basic CycleGAN model. Compared with representative algorithms, our proposed method reaches the state-of-the-art performance, with a 11.3% decrease in the mean absolute error and a 9.8% increase in the structural similarity. Hence, our proposed algorithm has a strong capacity of the domain transformation in ultrasound images to reduce the database bias for uniformly distributed datasets. We believe that our method can contribute to the development of the ultrasound image analysis and computer aided clinical diagnosis..

Perfect cycles in the synchronous Heider dynamics in complete network.

We discuss a cellular automaton simulating the process of reaching Heider balance in a fully connected network. The dynamics of the automaton is defined by a deterministic, synchronous, and global update rule. The dynamics has a very rich spectrum of attractors including fixed points and limit cycles, the length and number of which change with the size of the system. In this paper we concentrate on a class of limit cycles that preserve energy spectrum of the consecutive states. We call such limit cycles perfect. Consecutive states in a perfect cycle are separated from each other by the same Hamming distance. Also the Hamming distance between any two states separated by k steps in a perfect cycle is the same for all such pairs of states. The states of a perfect cycle form a very symmetric trajectory in the configuration space. We argue that the symmetry of the trajectories is rooted in the permutation symmetry of vertices of the network and a local symmetry of a certain energy function measuring the level of balance and frustration of triads.

Analysis of an environmental epidemic model based on voluntary vaccination policy

With the worsening of the environment and the increasing international trade, indirect transmission from exposure to contaminants in the surrounding environment has become an overlooked mode of transmission. This paper proposes a new game-theoretic model considering voluntary vaccination against imperfection and the unique integration of human-to-human and virus-to-human transmission routes. Based on the individual-based risk assessment update rule (IB-RA), the strategy-based risk assessment update rule (SB-RA), and the direct commitment update rule (DC), the different effects of individuals’ behaviors on disease prevalence are analyzed. To find the effect of indirect transmission on epidemic transmission, we compare our model with the traditional SVIR model. Finally, it can be seen that indirect transmission mechanisms will aggravate the spread of epidemics.

Multisample Online Learning for Probabilistic Spiking Neural Networks.

Spiking neural networks (SNNs) capture some of the efficiency of biological brains for inference and learning via the dynamic, online, and event-driven processing of binary time series. Most existing learning algorithms for SNNs are based on deterministic neuronal models, such as leaky integrate-and-fire, and rely on heuristic approximations of backpropagation through time that enforces constraints such as locality. In contrast, probabilistic SNN models can be trained directly via principled online, local, and update rules that have proven to be particularly effective for resource-constrained systems. This article investigates another advantage of probabilistic SNNs, namely, their capacity to generate independent outputs when queried over the same input. It is shown that the multiple generated output samples can be used during inference to robustify decisions and to quantify uncertainty-a feature that deterministic SNN models cannot provide. Furthermore, they can be leveraged for training in order to obtain more accurate statistical estimates of the log-loss training criterion and its gradient. Specifically, this article introduces an online learning rule based on generalized expectation-maximization (GEM) that follows a three-factor form with global learning signals and is referred to as GEM-SNN. Experimental results on structured output memorization and classification on a standard neuromorphic dataset demonstrate significant improvements in terms of log-likelihood, accuracy, and calibration when increasing the number of samples used for inference and training.