Incremental Algorithm Research Articles

What's the Difference? Incremental Processing with Change Queries in Snowflake

Incremental algorithms are the heart and soul of stream processing. Low latency results depend on the ability to react to the subset of changes in a dataset over time rather than reprocessing the entirety of a dataset as it evolves. But while the SQL language is well suited for representing streams of changes (via tables) and their application to tables over time (via DML), it entirely lacks a method to query the changes to a table or view in the first place. In this paper, we present CHANGES queries and STREAM objects, Snowflake's primitives for querying and consuming incremental changes to table objects over time. CHANGES queries and STREAMs have been in use within Snowflake for three years, and see broad adoption across our customers. We describe the semantics of these primitives, discuss the implementation challenges, present an analysis of their usage at Snowflake, and contrast with other offerings.

Conflict-Driven Synthesis for Layout Engines

Modern web browsers rely on layout engines to convert HTML documents to layout trees that specify color, size, and position. However, existing layout engines are notoriously difficult to maintain because of the complexity of web standards. This is especially true for incremental layout engines, which are designed to improve performance by updating only the parts of the layout tree that need to be changed. In this paper, we propose Medea, a new framework for automatically generating incremental layout engines. Medea separates the specification of the layout engine from its incremental implementation, and guarantees correctness through layout engine synthesis. The synthesis is driven by a new iterative algorithm based on detecting conflicts that prevent optimality of the incremental algorithm. We evaluated Medea on a fragment of HTML layout that includes challenging features such as margin collapse, floating layout, and absolute positioning. Medea successfully synthesized an incremental layout engine for this fragment. The synthesized layout engine is both correct and efficient. In particular, we demonstrated that it avoids real-world bugs that have been reported in the layout engines of Chrome, Firefox, and Safari. The incremental layout engine synthesized by Medea is up to 1.82× faster than a naive incremental baseline. We also demonstrated that our conflict-driven algorithm produces engines that are 2.74× faster than a baseline without conflict analysis.

Incremental Machine Learning for Soft Pneumatic Actuators with Symmetrical Chambers

Soft robotics is a specialized field of robotics that focuses on the design, manufacture, and control of robots made of soft materials, as opposed to those made of rigid links. One of the primary challenges for the future use of continuous or hyper-redundant robotics systems in industrial and medical technology is the development of suitable modeling and control approaches. Due to the complex non-linear behavior of soft materials and the unpredictable motion of actuators, the task of modeling complex soft actuators is very time-consuming. As a result, earlier studies have undertaken research into model-free methods for controlling soft actuators. In recent years, machine learning (ML) methods have become widely popular in research. The adaptability of an ML model to a non-linear soft drive system alongside the varying actuation behavior of soft drives over time as a result of material characteristics and performance requirements is the key rationale for including an ML model. The system requires the online updating of the ML model in order to work with the non-linear system. Sequential data collected from the test bench and converted into a hypothesis are used to perform incremental learning. These methods are called lifelong learning and progressive learning. Real-time data flow training is combined with incremental learning (IL), and a neural network model is tuned sequentially for each data input. In this article, a method for the intelligent control of soft pneumatic actuators based on an incremental learning algorithm is proposed. A soft pneumatic actuator was subjected to three distinct test conditions in a controlled test environment for a specified duration of data gathering. Additionally, data were collected through finite element method simulations. The collected data were used to incrementally train a neural network, and the resulting model was analyzed for errors with both training and test data. The training and testing errors were compared for different incremental learning (IL) algorithms, including K-nearest neighbors, a decision tree, linear regression, and a neural network. The feasibility of the modulo-free intelligent control of soft pneumatic actuators based on an incremental learning algorithm was verified, solving the problem of the control of software actuators.

Development and research of a neural network alternate incremental learning algorithm

In this paper, the relevance of developing methods and algorithms for neural network incremental learning is shown. Families of incremental learning techniques are presented. A possibility of using the extreme learning machine for incremental learning is assessed. Experiments show that the extreme learning machine is suitable for incremental learning, but as the number of training examples increases, the neural network becomes unsuitable for further learning. To solve this problem, we propose a neural network incremental learning algorithm that alternately uses the extreme learning machine to correct the only output layer network weights (operation mode) and the backpropagation method (deep learning) to correct all network weights (sleep mode). During the operation mode, the neural network is assumed to produce results or learn from new tasks, optimizing its weights in the sleep mode. The proposed algorithm features the ability for real-time adaption to changing external conditions in the operation mode. The effectiveness of the proposed algorithm is shown by an example of solving the approximation problem. Approximation results after each step of the algorithm are presented. A comparison of the mean square error values when using the extreme learning machine for incremental learning and the developed algorithm of neural network alternate incremental learning is made.

A Novel Dynamic Fusion Approach Using Information Entropy for Interval-Valued Ordered Datasets

Information fusion is capable of fusing and transforming information originated from multiple sources into an integrated representation. As an important representative of information, interval-valued ordered data aims at characterizing inaccurate and ambiguous information. However, existing methods are inappropriate when applied to the multi-source fusion for them. In addition, it is inevitable that in daily-life the sources and attributes of multi-source information systems may update at the same time. Hence there is a need to fuse data as efficiently as possible. Inspired by these deficiencies, we pay attention to the effective and efficient fusion of multi-source interval-valued ordered data in varieties of cases. First, the concepts of fuzzy dominance relation and dominance classes based on it are put forward between any two samples of interval-valued ordered information systems. Second, we define the fuzzy dominating and dominated conditional entropy. Then the fusion model is established and it is multi-source interval-valued ordered data oriented. Furthermore, it is true that there are numerous real-life applications related to concurrent change of both sources and attributes. Consequently, we design four incremental mechanisms and algorithms for fusing multi-source interval-valued data on the ground of the static condition. Eventually, a series of experiments is carried out on twelve datasets to verify that the proposed fusion approach outperforms other comparative methods on efficacy. Meanwhile, our incremental fusion algorithms are efficient compared with the static one for updating multi-source interval-valued data when sources and attributes are in the simultaneous variation.

Research on improved MPPT Fuzzy Logic Control-Incremental Conductance Algorithm

This research discusses the limitations of the Maximum Power Point Tracking (MPPT) incremental behaviour algorithm. Although MPPT's incremental behaviour algorithm is simple and easy to implement, despite its usefulness, this method is beset by several limitations which include a slow convergence rate towards the optimal operating point, significant oscillations surrounding the maximum power point at steady state, and momentary system movement away from the maximum power point after sudden changes or variations in irradiation. For these reasons, an improved MPPT Fuzzy Logic Control-Incremental conductance (FLC-IC) algorithm is proposed in this study. And the adjustment in the input variables of the MPPT Incremental Conductance algorithm controlled by the fuzzy intelligent control algorithm increases the convergence speed, decreases the oscillations, and remains stable despite radiation variations. The algorithm is simulated and applied in a charge controller that operates using the solar energy, and the outputs observed highlights the effectiveness of the proposed algorithm that is proposed over the IC algorithm in terms of speed and efficiency.

Self-Paced Broad Learning System.

Broad learning system (BLS), an efficient neural network with a flat structure, has received a lot of attention due to its advantages in training speed and network extensibility. However, the conventional BLS adopts the least square loss, which treats each sample equally and thus is sensitivity to noise and outliers. To address this concern, in this article we propose a self-paced BLS (SPBLS) model by incorporating the novel self-paced learning (SPL) strategy into the network for noisy data regression. With the assistance of the SPL criterion, the model output is used as feedback to learn appropriate priority weight to readjust the importance of each sample. Such a reweighting strategy can help SPBLS to distinguish samples from "easy" to "difficult" in model training, equipping the model robust to noise and outliers while maintaining the characteristics of the original system. Moreover, two incremental learning algorithms associated to SPBLS have also been developed, with which the system can be updated quickly and flexibly without retraining the entire model when new training samples are added or the network needs to be expanded. Experiments conducted on various datasets demonstrate that the proposed SPBLS can achieve satisfying performance for noisy data regression.

Visual Analytics on Network Forgetting for Task‐Incremental Learning

AbstractTask‐incremental learning (Task‐IL) aims to enable an intelligent agent to continuously accumulate knowledge from new learning tasks without catastrophically forgetting what it has learned in the past. It has drawn increasing attention in recent years, with many algorithms being proposed to mitigate neural network forgetting. However, none of the existing strategies is able to completely eliminate the issues. Moreover, explaining and fully understanding what knowledge and how it is being forgotten during the incremental learning process still remains under‐explored. In this paper, we propose KnowledgeDrift, a visual analytics framework, to interpret the network forgetting with three objectives: (1) to identify when the network fails to memorize the past knowledge, (2) to visualize what information has been forgotten, and (3) to diagnose how knowledge attained in the new model interferes with the one learned in the past. Our analytical framework first identifies the occurrence of forgetting by tracking the task performance under the incremental learning process and then provides in‐depth inspections of drifted information via various levels of data granularity. KnowledgeDrift allows analysts and model developers to enhance their understanding of network forgetting and compare the performance of different incremental learning algorithms. Three case studies are conducted in the paper to further provide insights and guidance for users to effectively diagnose catastrophic forgetting over time.

The Plausibility of Sampling as an Algorithmic Theory of Sentence Processing.

Words that are more surprising given context take longer to process. However, no incremental parsing algorithm has been shown to directly predict this phenomenon. In this work, we focus on a class of algorithms whose runtime does naturally scale in surprisal-those that involve repeatedly sampling from the prior. Our first contribution is to show that simple examples of such algorithms predict runtime to increase superlinearly with surprisal, and also predict variance in runtime to increase. These two predictions stand in contrast with literature on surprisal theory (Hale, 2001; Levy, 2008a) which assumes that the expected processing cost increases linearly with surprisal, and makes no prediction about variance. In the second part of this paper, we conduct an empirical study of the relationship between surprisal and reading time, using a collection of modern language models to estimate surprisal. We find that with better language models, reading time increases superlinearly in surprisal, and also that variance increases. These results are consistent with the predictions of sampling-based algorithms.

DeepWE: A Deep Bayesian Active Learning Waypoint Estimator for Indoor Walkers

Waypoint Estimation (WE) has a wide range of applications for indoor walkers, such as fire rescue, and navigation to find exit doors, lifts, or stairs as examples of waypoints, etc. Data-driven waypoint estimation has been on the rise with advancements in deep learning algorithms. The current waypoint estimation methods, however, face two challenges. On one hand, most waypoint detection approaches rely on visual sensors, hence, their estimation performance is limited by light when collecting visual data. On the other hand, data-driven methods necessitate a large number of labeled data to train a WE model, which significantly increases the time spent manually marking labels. Targeting the above two challenges, our work first proposes a novel Deep Bayesian Active Learning Waypoint Estimator for indoor walkers (DeepWE) based on Human Activity Recognition (HAR). This estimates six indoor waypoints through walkers daily activities due to the strong correlation between human activities and waypoints. Firstly, an initial DeepWE model is developed using a Bayesian ensembled Convolutional Neural Network (B-CNN) using the accelerometer and gyroscope data. Then, active learning is employed to query the most formative samples from pool points with four acquisition functions, and only these queried samples are labeled manually. Finally, the initial DeepWE model is updated from this labeled data using an incremental learning algorithm. Empirical results on two publicly available USC-HAD and OPPORTUNITY datasets show DeepWE performs a considerable accuracy boost for waypoint estimation, with a substantial amount of acquired pool points reduction (more than 40%)

Privacy-Preserving Shortest Distance Query for Dynamic Large-Scale Road Network

The road network plays an important role in guiding people's daily travel. In the road network, the shortest distance query is one of the most basic query operations. With the scale of the road network continuously expanding, people choose to outsource the road network to the cloud server in the form of an encrypted graph. To the best of authors' knowledge, the existing privacy-preserving shortest distance query schemes over large-scale encrypted graphs are all designed for the static graph. Nonetheless, the graph of road network is often updated over time. These static privacy-preserving shortest distance query schemes cannot work well in this practical application. In order to address this problem, we explore how to implement the privacy-preserving shortest distance query in an incremental large-scale road network. We use the 2-hop cover label index as the basic secure index structure, which can realize the shortest distance fast query. In order to update the 2-hop cover label index safely and efficiently, we combine a special adjacency index as an auxiliary index structure with homomorphic encryption to implement a dynamic incremental update algorithm for the encrypted graph. We prove that the proposed scheme achieves adaptive semantic security and verify its practicality through experiments.

Forgetful Forests: Data Structures for Machine Learning on Streaming Data under Concept Drift

Database and data structure research can improve machine learning performance in many ways. One way is to design better algorithms on data structures. This paper combines the use of incremental computation as well as sequential and probabilistic filtering to enable “forgetful” tree-based learning algorithms to cope with streaming data that suffers from concept drift. (Concept drift occurs when the functional mapping from input to classification changes over time). The forgetful algorithms described in this paper achieve high performance while maintaining high quality predictions on streaming data. Specifically, the algorithms are up to 24 times faster than state-of-the-art incremental algorithms with, at most, a 2% loss of accuracy, or are at least twice faster without any loss of accuracy. This makes such structures suitable for high volume streaming applications.

Efficient and Effective Cardinality Estimation for Skyline Family

Cardinality estimation, predicting the query result size, is a fundamental problem in databases. Existing skyline cardinality estimation methods are computationally infeasible for massive skyline queries over the large-scale database. In this paper, we introduce a unified skyline family w.r.t. various skyline variants. We propose an efficient and effective skyline family cardinality estimation model, named EECE, in an end-to-end manner. EECE consists of two modules, unsupervised data distribution learning (DDL) and supervised monotonic cardinality estimation (MCE). DDL leverages the mixture data guided transformer to learn the distribution of database and query parameters for model pre-training. MCE further incorporates supervised learning and parameter clamping to enhance the estimation under monotonicity guarantees. We develop an efficient incremental learning algorithm for EECE to adapt the database and query logs update. Extensive experiments on several real-world and synthetic datasets demonstrate that, EECE speeds up the cardinality estimation by six orders of magnitude, with more than 39% accuracy gain, compared to the state-of-the-art approaches.

IDLIQ: An Incremental Deterministic Finite Automaton Learning Algorithm Through Inverse Queries for Regular Grammar Inference.

We present an efficient incremental learning algorithm for Deterministic Finite Automaton (DFA) with the help of inverse query (IQ) and membership query (MQ). This algorithm is an extension of the Identification of Regular Languages (ID) algorithm from a complete to an incremental learning setup. The learning algorithm learns by making use of a set of labeled examples and by posing queries to a knowledgeable teacher, which is equipped to answer IQs along with MQs and equivalence query. Based on the examples (elements of the live complete set) and responses against IQs from the minimally adequate teacher (MAT), the learning algorithm constructs the hypothesis automaton, consistent with all observed examples. The Incremental DFA Learning algorithm through Inverse Queries (IDLIQ) takes time complexity in the presence of a MAT and ensures convergence to a minimal representation of the target DFA with finite number of labeled examples. Existing incremental learning algorithms; the Incremental ID, the Incremental Distinguishing Strings have polynomial (cubic) time complexity in the presence of a MAT. Therefore, sometimes, these algorithms even fail to learn large complex software systems. In this research work, we have reduced the complexity (from cubic to square form) of the DFA learning in an incremental setup. Finally, we prove the correctness and termination of the IDLIQ algorithm.

Fault detection of gearbox by multivariate extended variational mode decomposition-based time–frequency images and incremental RVM algorithm

A novel detection method based on multivariate extended variational mode decomposition-based time–frequency images and incremental RVM algorithm (MEVMDTFI–IRVM) is presented for fault detection of gearbox. The time–frequency images are constructed by multivariate extended variational mode decomposition. Compared with single-variable modal decomposition method, multivariate extended variational mode decomposition not only has an accurate mathematical framework, but also has good robustness to non-stationary multi-channel signals with low signal-to-noise ratio. The incremental RVM algorithm is presented for fault detection of gearbox based on the time–frequency images constructed by multivariate extended variational mode decomposition. The testing results demonstrate that the detection results of MEVMDTFI–IRVM for gearbox are stable, in addition, the detection results of MEVMDTFI–IRVM for gearbox are better than those of variational mode decomposition-based time–frequency images and incremental RVM algorithm (VMDTFI–IRVM), variational mode decomposition–RVM algorithm (VMD–RVM), and traditional RVM algorithm.

Dynamic conditional score model-based weighted incremental fuzzy clustering of consumer power load data

This study proposes a weighted incremental fuzzy C-mean power load clustering algorithm based on the dynamic conditional score model to solve the problems that the predominant power load data mining method only captures the mean characteristics of time series and ignores the heteroscedasticity and sequence correlations. Consequently, this method has unsatisfactory time series clustering and low clustering accuracy. A dynamic conditional score model is constructed to analyze and extract statistical characteristic parameters of a time series to calculate the autocorrelation value of the parameter series. A weighted fuzzy C-mean clustering analysis is performed, and the obtained data weight information is used as input for incremental clustering to improve the clustering accuracy. The DCS model parameter dataset and data weight information are combined, and the clustering analysis of the consumer power load data stream is performed. The power load time series of different companies is given, and the clustering validity indices are defined for the performance analysis to verify the proposed clustering algorithm. The experimental results show that the proposed algorithm achieves satisfactory clustering and improves the performance.

Intelligent Geomagnetic Indoor Positioning System

In the past, several firefighters have died in disaster relief operations. Although the firefighters were fully equipped, the scene of the disaster was smoky and disorienting, making the firefighters unable to identify their location. The commander wanted to direct the firefighters outside but could not confirm the correct location of the firefighters, causing delays in rescue. GPS cannot support indoor positioning or preset indoor positioning facilities at the moment of fire extinguishing. However, geomagnetism is everywhere, and it can be used to identify one’s location. Unfortunately, due to the uncertainty of the magnetic field strength, indoor geomagnetism is affected by the building environment, and the existing magnetic positioning methods have difficulty obtaining a location. To solve this problem, we propose a new incremental indoor localization scheme based on the difference in geomagnetic intensity. The proposed method achieves indoor localization in 2D environments successfully. The novelty of our geomagnetic indoor positioning system is that it can perform indoor positioning without adding any indoor positioning facilities, and the accuracy can reach 0.8~1.5 m. This article aims to verify that the geomagnetic turbulence filtering algorithm can filter out abnormal geomagnetic intensity, that the incremental algorithm can estimate the position of human motion, and that geomagnetism can be used for indoor positioning without any preset infrastructure. The contribution of this paper is that we have developed a practical system that can be used without any infrastructure and can be used for indoor positioning with meter-level accuracy. The geomagnetic indoor positioning system can be integrated with a wireless network and applied to disaster relief.

An active learning-based incremental deep-broad learning algorithm for unbalanced time series prediction

Time series are a kind of streaming data, which are chaotic and sequential. As real-world time series data are often not available at once and drift with time growth, Incremental Learning (IL) is well suited for Time Series Prediction (TSP). Most previous incremental TSP algorithms are limited by the assumption of data balance. However, real-world time series data are often unbalanced, with long-tailed distribution and other characteristics resulting in the failure of IL algorithms. In this paper, a balanced-driven Active Learning (AL) strategy is proposed to deal with data imbalance problems in IL processes. What’s more, by integrating the advantages of Deep Learning (DL) and the Broad Learning System (BLS), a novel Deep-Broad Learning (DeepBL) network with its incremental learning algorithm is proposed. The proposed Active Learning-based Incremental Deep-Broad Learning (AI_DeepBL) algorithm is applied to real-world univariate and multivariate time series datasets and achieves superior performance compared with classical and state-of-the-art TSP algorithms.

Analysis of Numerical Micromodulus Coupled with Influence Function for Brittle Materials via Bond-Based Peridynamics

In this paper, the numerical micromodulus is derived for the plane stress problem to develop a new insight into the application of bond-based peridynamics. Considering the nonlocal property of peridynamics, the numerical micromodulus coupled with influence function provides a reasonable description of the long-range force effect. Through several numerical applications, the effectiveness of the numerical modulus coupled with various influence functions to simulate deformation and failure is analyzed. In addition, a load increment algorithm based on fictitious density is developed specifically for quasi-static problems. It is indicated that the introduction of the influence function can enhance the accuracy in deformation and failure simulation, which is valuable for the advancement and application of numerical micromoduli. Through a comprehensive trade-off between simulation accuracy and stability, the numerical micromodulus coupled with the exponential influence function proves to be the more effective option for brittle material.

An incremental learning evolutionary algorithm for many-objective optimization with irregular Pareto fronts

Multi objective optimization problems (MOPs) with irregular Pareto Fronts (PFs) appear frequently in a large of practical cases. Proximity and diversity are the key indicators for decomposition-based multi-objective evolutionary algorithms (MOEA/D) to solve MOPs, which are decided by the neighborhood structure and a set of predefined direction vectors. In certain MOPs with irregular PFs, the effectiveness of MOEA/D is restricted by the shape of the PF. This limitation arises because MOEA/D generates direction vectors in a uniform manner, which can result in invalid direction vectors due to the irregularity of the PF. When dealing with many-objective optimization problems (MaOP) that have irregular PFs, such as degenerate or disconnected PFs, this decline in effectiveness becomes particularly problematic. To address this limitation, a copula incremental learning (CIL) scheme has been developed to progressively extract implicit knowledge on the appropriate distribution of direction vectors for generating non-uniform direction vectors. Additionally, a niche hierarchical selection (NHS) methodology is employed to construct the neighborhood structure and prevent the generation of duplicate solutions. These enhancements are designed to improve the overall effectiveness of MOEA/D. Efficiency is further ensured through the use of convergence-guided direction (CGD) to approximate irregular PFs. Statistical analysis indicates that the proposed method outperforms other competitive algorithms across most test benchmarks, particularly in its ability to effectively address MaOPs with irregular PFs.