Incremental Update Research Articles

A Comparative Study of AStar, LPA* and DStarLite Path Planning Algorithms Based on Matlab

In the realm of robotics and autonomous systems, path planning is a pivotal component that determines the efficacy and safety of navigational tasks. With the proliferation of autonomous vehicles, drones, and mobile robots, the need for efficient and adaptive path planning algorithms has become increasingly acute. This paper studies AStar, LPA and DStarLite path planning algorithms based on Matlab platform, and compares their performance through simulation experiments. AStar algorithm is simple and widely applicable, but it has some shortcomings in path smoothness and computational efficiency. LPA improves path smoothness by introducing dynamic cost updating, but it may sacrifice some computational efficiency. The DStarLite algorithm performs well in dynamic environments with an efficient incremental update strategy that maintains high path smoothness and low computational costs. The experimental results show that DStarLite is the fastest in most cases, LPA* and DStarLite are superior to AStar in path smoothness. Future research may explore combining the advantages of each algorithm to develop more efficient, flexible and robust path planning algorithms to cope with complex and changeable actual scenarios.

LinFa-Q: Accurate Q-learning with linear function approximation

Although Q-learning has achieved remarkable success in some practical cases, it often suffers from the overestimation problem in stochastic environments, which is commonly viewed as a shortcoming of Q-learning. Overestimated values are introduced by estimations on the next state Q value, which is well-known as the maximization bias. In this paper, we propose a more accurate method for estimating the Q value by the Q value decomposition and re-evaluation with similar samples based on linear function approximation. Specifically, we reform the parameterized incremental update formula of Q-learning and also demonstrate that the new formula is equivalent to the original one. Moreover, we propose a new parameterized incremental update formula of Q-learning to address the overestimation problem and present the more accurate computing method, which can be used in problems with continuous state spaces and stochastic environments. Experimentally, when compared with Doubly Bounded Q-learning and other Q-learning based methods, the new algorithm has more than 31% improvement of performance in Mountain Car and Cart Pole. Furthermore, the algorithm is robust to the learning rate and its memory capacity. Finally, the practical applicability of our algorithm is discussed through an analysis of time consumption.

The Influence of Pipeline-Backfill-Trench Interaction on Pipeline Response to Ice Gouging: A Numerical Investigation

Ice gouging is a destructive incident to subsea pipelines in Arctic regions. Trenching and backfilling is a cost-effective solution to physically protect the pipeline against ice gouging. Ice gouging imposes a complex combination of stresses and strains through the soil medium, the pipeline, and the interface. Remolded backfill materials with considerably less stiffness than native soil result in more complexity in soil failure mechanisms and pipe trajectories. However, this critical aspect is less explored in the literature on ice gouges. This paper investigated the influences of pipeline-backfill-trench interaction on the soil failure mechanisms and the pipeline responses by coupled Eulerian-Lagrangian (CEL) method. Two model configurations (shallowly buried and deeply buried pipeline) were set up to investigate the influence of trenching/backfilling, as well as pipe burial depth. Incorporation of the strain-rate dependency and strain-softening effects in the soil constitutive model involved the development of a user-defined subroutine and incremental update of the undrained shear strength within the Abaqus software. The research findings indicate that the conventional approach of assuming uniform seabed soil for trenched and backfilled pipelines may not accurately capture the pipeline behavior and soil failure mechanisms.

Incremental Discourse-Update Constrains Number Agreement Attraction Effect.

While a large body of work in sentence comprehension has explored how different types of linguistic information are used to guide syntactic parsing, less is known about the effect of discourse structure. This study investigates this question, focusing on the main and subordinate discourse contrast manifested in the distinction between restrictive relative clauses (RRCs) and appositive relative clauses (ARCs) in American English. In three self-paced reading experiments, we examined whether both RRCs and ARCs interfere with the matrix clause content and give rise to the agreement attraction effect. While the standard attraction effect was consistently observed in the baseline RRC structures, the effect varied in the ARC structures. These results collectively suggest that discourse structure indeed constrains syntactic dependency resolution. Most importantly, we argue that what is at stake is not the static discourse structure properties at the global sentence level. Instead, attention should be given to the incremental update of the discourse structure in terms of which discourse questions are active at any given moment of a discourse. The current findings have implications for understanding the way discourse structure, specifically the active state of discourse questions, constrains memoryretrieval.

APM: Adaptive parameter multiplexing for class incremental learning

In recent developments within the domain of image classification, deep neural networks (DNNs) have attracted considerable scholarly interest and have been extensively trained using data in closed environments. Such training methodologies contrast sharply with the inherently open, progressive, and adaptive processes of the natural visual system, leading to emergent challenges. Among these, catastrophic forgetting is notable, where the network acquisition of new class information precipitates the erosion of previously established knowledge. Additionally, the network encounters the stability-plasticity dilemma, necessitating a delicate equilibrium between assimilating novel classes and retaining existing ones. To address these issues, we propose a novel incremental learning model, termed Adaptive Parameter Multiplexing (APM), which incorporates a cross-class parameter adaptive incremental strategy. Central to our methodology is the conceptualization of parameter multiplexing or incremental as a learnable optimization problem, enabling the model to autonomously evaluate and decide on the necessity for parameter adjustment throughout its training lifecycle. This framework is designed to enhance the ability of the network to extract features for new class categories effectively through incremental parameters while simultaneously employing parameter multiplexing to augment storage optimization. Our model is underpinned by a dual strategy of coarse-grained and fine-grained parameter multiplexing, guided by a learnable score that dynamically assesses the appropriateness of parameter multiplexing versus incremental updates, facilitating an optimized balance for incremental model performance and storage. In addition, we have integrated a novel regularization loss mechanism for the learnable score to optimize storage efficiency. The effectiveness of APM is empirically validated through rigorous testing on benchmark datasets, including ImageNet100, CIFAR100, CIFAR10, and CUB200. The experimental outcomes indicate that, with a trace amount of parameter increase, our model achieves significant enhancements in classification performance across both new and previously established classes, thereby surpassing existing benchmarks set by state-of-the-art algorithms in the field.

Layered seabed effects on buried pipeline response to ice gouging

In ice gouging analyses, the seabed is often simplified as a uniform material domain, over-looking the potential complexities inherent in layered seabed formations, which are wide-spread in numerous Arctic regions. This study investigates the pipeline response into the distinct seabed configurations, including layered soft over stiff clay, layered stiff over soft clay, and uniformly soft and uniformly stiff compositions, to ice gouging. This was achieved through comprehensive large deformation finite element (LDFE) analysis, employing a Coupled Eulerian Lagrangian (CEL) algorithm. Incorporation of the strain-rate dependency and strain-softening effects involved the development of a user-defined subroutine and incremental update of the undrained shear strength within the Abaqus software. The pipeline is explicitly incorporated into the model, allowing for a detailed investigation of its response to ice gouging phenomena within these layered seabed scenarios. This research demonstrates that simplifying a layered seabed into a uniform can be misleading. Such an approach can lead to significant errors, potentially overestimating or underestimating pipeline response to ice gouging. By incorporating these complexities, engineers can develop more optimized designs with enhanced safety margins in Arctic environments prone to ice gouging phenomena.

Implementation of Lung Nodule Detection Model Based on Incremental Meta-Learning

In response to the issue that traditional lung nodule detection models cannot dynamically optimize and update with the increase of new data, a new lung nodule detection model-task incremental meta-learning model (TIMLM) is proposed. This model comprises of two loops: the inner loop imposes incremental learning regularization update constraints, while the outer loop employs a meta-update strategy to sample old and new knowledge and learn a set of generalized parameters that adapt to old and new data. Under the condition that the main structure of the model is not changed as much as possible, it preserves the old knowledge that was learned previously. Experimental verification on the publicly available lung dataset showed that, compared with traditional deep network models and mainstream incremental models, TIMLM has greatly improved in terms of accuracy, sensitivity, and other indicators, demonstrating good continuous learning and anti-forgetting capabilities.

BCDA: A blockchain-based dynamic auditing scheme for intelligent IoT

Intelligent Internet of Things (IoT) provides services for machine learning applications by collecting real-time data, where data correctness crucially impacts the training accuracy of models. Data owners host incremental real-time data in the cloud and use blockchain as a public platform for data query, usage, and certification. Malicious attackers may manipulate data through methods like data poisoning, compromising model parameters. Blockchain-based data integrity verification (auditing) schemes can effectively detect and monitor such activities. However, managing the storage demands for massive on-chain data certificates poses significant challenges. Existing compressed certificate storage strategies, such as the Merkle-Hash tree (MHT), incur significant on-chain storage costs because the auxiliary paths and the entire tree structure must be uploaded during verification, resulting in an O(n) storage overhead. Polynomial commitments offer a promising alternative by reducing storage costs to hundreds of bytes. However, current polynomial commitment methods lack effective support for incremental data updates, which are essential for real-time applications. In this paper, we propose a novel polynomial commitment-based dynamic integrity verification scheme that implements cumulative commitment updates for the first time. Our scheme includes a new data block insertion and deletion strategy that maintains the original order of data blocks within polynomial commitments. Theoretical security analysis confirms the robustness of the proposed scheme, while simulation results demonstrate that it meets the efficiency requirements for blockchain-based distributed data integrity verification mechanisms.

Dynamic compact data structure for temporal reachability with unsorted contact insertions

Abstract Temporal graphs represent interactions between entities over time. Deciding whether entities can reach each other through temporal paths is useful for various applications such as in communication networks and epidemiology. Previous works have studied the scenario in which addition of new interactions can happen at any point in time. A known strategy maintains, incrementally, a Timed Transitive Closure using a dynamic data structure composed of $O(n^{2})$ binary search trees containing non-nested time intervals. However, space usage for storing these trees grows rapidly as more interactions are inserted. In this paper, we introduce a compact data structure that represents each tree as two dynamic bit-vectors. Furthermore, we present two variants of this data structure: one representing bits in dynamic bit-vectors explicitly and the other representing only the active bits by encoding their consecutive distances. In our experiments, we observed that our data structure improves space usage while having similar time performance for incremental updates when comparing with the previous strategy. The first variant of our data structure gives the best space improvement when constructing Time Transitive Closures for temporally dense temporal graphs, and the second variant uses less space for temporally sparse temporal graphs.

An Efficient System for Large-Scale Collaborative Tasks in Government

Government collaboration tasks are integral for grassroots governance and essential for government administration. Large-scale government collaboration tasks often involve multiple departments working together to solve complex tasks that require handling large amounts of data. However, existing offline processes make it difficult to manage complex collaboration scenarios and obtain timely execution results. We introduce the concept of a spatio-temporal task tree and propose a log-based incremental update method for updating statistical values in the tree structure to fulfill the need for real-time aggregation analysis of cross-temporal task execution situations. The spatio-temporal task tree fosters collaboration across various departments and organizational hierarchies to execute complex and large-scale tasks in government effectively. By flexible adaptations to different business models, this framework addresses the shortcomings of conventional systems in promoting inter-organizational collaboration. The log-based incremental update method for calculating statistical values within a tree structure allows for the updating of multi-dimensional data statistics in seconds. This approach effectively decreases computational expenses, enhances query efficiency, and improves real-time computation response. Results from experiments demonstrate the feasibility and high availability of the proposed approach, showcasing their potential to address the challenges associated with large-scale collaborative tasks in government affairs.

A dynamic attribute reduction algorithm based on relative neighborhood discernibility degree

This paper addresses the current existence of attribute reduction algorithms for incomplete hybrid decision-making systems, including low attribute reduction efficiency, low classification accuracy and lack of consideration of unlabeled data types. To address these issues, this paper first redefines the weakly labeled relative neighborhood discernibility degree and develops a non-dynamic attribute reduction algorithm. In addition, this paper proposes an incremental update mechanism for weakly tagged relative neighborhood discernibility degree and introduces a new dynamic attribute reduction algorithm for increasing the set of objects based on it. Meanwhile, this paper also compares and analyses the improved algorithm proposed in this study with two existing attribute reduction algorithms using 8 data sets in the UCI database. The results show that the dynamic attribute reduction algorithm proposed in this paper achieves higher attribute reduction efficiency and classification accuracy, which further validates the effectiveness of the algorithm proposed in this paper.

Three-Dimensional Dead-Reckoning Based on Lie Theory for Overcoming Approximation Errors

This paper proposes a dead-reckoning (DR) method for vehicles using Lie theory. This approach treats the pose (position and attitude) and velocity of the vehicle as elements of the Lie group SE2(3) and follows the computations based on Lie theory. Previously employed DR methods, which have been widely used, suffer from cumulative errors over time due to inaccuracies in the calculated changes from velocity during the motion of the vehicle or small errors in modeling assumptions. Consequently, this results in significant discrepancies between the estimated and actual positions over time. However, by treating the pose and velocity of the vehicle as elements of the Lie group, the proposed method allows for accurate solutions without the errors introduced by linearization. The incremental updates for pose and velocity in the DR computation are represented in the Lie algebra. Experimental results confirm that the proposed method improves the accuracy of DR. In particular, as the motion prediction time interval of the vehicle increases, the proposed method demonstrates a more pronounced improvement in positional accuracy.

Incremental Sliding Window Connectivity over Streaming Graphs

We study index-based processing for connectivity queries within sliding windows on streaming graphs. These queries, which determine whether two vertices belong to the same connected component, are fundamental operations in real-time graph data processing and demand high throughput and low latency. While indexing methods that leverage data structures for fully dynamic connectivity can facilitate efficient query processing, they encounter significant challenges with deleting expired edges from the window during window updates. We introduce a novel indexing approach that eliminates the need for physically performing edge deletions. This is achieved through a unique bidirectional incremental computation framework, referred to as the BIC model. The BIC model implements two distinct incremental computations to compute connected components within the window, operating along and against the timeline, respectively. These computations are then merged to efficiently compute queries in the window. We propose techniques for optimized index storage, incremental index updates, and efficient query processing to improve BIC effectiveness. Empirically, BIC achieves a 14× increase in throughput and a reduction in P95 latency by up to 3900× when compared to state-of-the-art indexes.

Incremental analysis update (IAU) in the Model for Prediction Across Scales coupled with the Joint Effort for Data assimilation Integration (MPAS–JEDI 2.0.0)

Abstract. In a cycling system where data assimilation (DA) and model simulation are executed consecutively, the model forecasts initialized from the analysis (or data assimilation) can be systematically affected by dynamic imbalances generated during the analysis process. The high-frequency noise arising from the imbalances in the initial conditions can impose constraints on computational stability and efficiency during subsequent model simulations and can potentially become the low-frequency waves of physical significance. To mitigate these initial imbalances, the incremental analysis update (IAU) has long been utilized in the cycling context. This study introduces our recent implementation of the IAU in the Model for Prediction Across Scales – Atmospheric (MPAS-A) coupled with the Joint Effort for Data assimilation Integration (JEDI) through the cycling system called MPAS-Workflow. During the integration of the compressible nonhydrostatic equations in MPAS-A, analysis increments are distributed over a predefined time window (e.g., 6 h) as fractional forcing at each time step. In a real case study with the assimilation of all conventional and satellite radiance observations every 6 h for 1 month, starting from mid-April 2018, model forecasts with the IAU show that the initial noise illustrated by surface pressure tendency becomes well constrained throughout the forecast lead times, enhancing the system reliability. The month-long cycling with the assimilation of real observations demonstrates the successful implementation of the IAU capability in the MPAS–JEDI cycling system. Along with the comparison between the forecasts with and without the IAU, several aspects regarding the implementation in MPAS–JEDI are discussed. Corresponding updates have been incorporated into the MPAS-A model (originally based on version 7.1), which is now publicly available in MPAS–JEDI and MPAS-Workflow version 2.0.0.

Diagnostics for Imbalance on the Convective Scale

Abstract The analyses produced by a data assimilation system may be unbalanced, that is dynamically inconsistent with the forecasting model, leading to noisy forecasts and reduced skill. While there are effective procedures to reduce synoptic-scale imbalance, the situation on the convective scale is less clear because the flowon this scale is strongly divergent and non-hydrostatic. In this studywe compare three measures of imbalance relevant to convective-scale data assimilation: (i) surface pressure tendencies, (ii) vertical velocity variance in the vicinity of convective clouds, and (iii) departures from the vertical velocity prescribed by the weak temperature gradient (WTG) approximation. These are applied in a numerical weather prediction system, with three different data assimilation algorithms: 1. Latent Heat Nudging (LHN), 2. Local Ensemble Transform Kalman Filter (LETKF), and 3. LETKF in combination with incremental analysis updates (IAU). Results indicate that surface pressure tendency diagnoses a different type of imbalance than the vertical velocity variance and theWTG departure. The LETKF induces a spike in surface pressure tendencies, with a large-scale spatial pattern that is not clearly related to the precipitation pattern. This anomaly is notably reduced by the IAU. LHN does not generate a pronounced signal in the surface pressure, but produces the most imbalance in the other two measures. The imbalances measured by the partitioned vertical velocity variance andWTG departures are similar, and closely coupled to the convective precipitation. Between these two measures, the WTG departure has the advantage of being simpler and more economical to compute.

Shallow defects identification for urban roads using interpretable dynamic broad network

Deep learning is extensively utilised in transport geotechnical engineering. However, deep architectures have large computational costs and update times, while failing to understand decisions. To this regard, we propose an interpretable dynamic broad network combined with ground-penetrating radar for internal defect identification in roadbeds. The method is more suitable for feature characterisation of two-dimensional data and satisfies incremental updates. The test results indicated that the proposed method has an average recognition accuracy of 0.9124 for the four types of internal defects in roadbeds. Compared to the other four classical machine learning methods, it balances training efficiency and recognition accuracy. Robustness analysis results demonstrated that the method is noise-resistant. However, comprehending the recognition results of intelligent algorithms is a key topic. Local interpretation approach is introduced to quantify the feature importance that affects the decision of the model. Based on the feature importance calculation, it is possible to distinguish between positive and negative regions in one sample that influence the decision of the detection model. These interpretative analyses can assist us in better understanding the reasons for decisions generated by the detection model that provide technical support for subsequent enhancements.

SimLOD: Simultaneous LOD Generation and Rendering for Point Clouds

LOD construction is typically implemented as a preprocessing step that requires users to wait before they are able to view the results in real time. We propose an incremental LOD generation approach for point clouds that allows us to simultaneously load points from disk, update an octree-based level-of-detail representation, and render the intermediate results in real time while additional points are still being loaded from disk. LOD construction and rendering are both implemented in CUDA and share the GPU's processing power, but each incremental update is lightweight enough to leave enough time to maintain real-time frame rates. Our approach is able to stream points from an SSD and update the octree on the GPU at rates of up to 580 million points per second (~9.3GB/s) on an RTX 4090 and a PCIe 5.0 SSD. Depending on the data set, our approach spends an average of about 1 to 2 ms to incrementally insert 1 million points into the octree, allowing us to insert several million points per frame into the LOD structure and render the intermediate results within the same frame.

CEIU: Consistent and Efficient Incremental Update mechanism for mobile systems on flash storage

The ever-growing sizes and frequent updating of mobile applications cause high network and storage cost for updating. Hence, emerging mobile systems often employ incremental update algorithms, typically HDiffPatch, to upgrade mobile applications. However, we find that existing incremental update algorithms not only generate a significant amount of redundant data accesses, but also lacks of consistency guarantees for the whole package of application. In this paper, we present a novel Consistent and Efficient Incremental Update (CEIU) mechanism for upgrading mobile applications. Firstly, CEIU reduces the memory consumption and file access of incremental updates by reusing the indexes of blocks in the old image rather than copying the blocks. Secondly, CEIU employs a two-level journaling mechanism to ensure the consistency of the whole package and subfiles of the new image. We implement the proposed mechanism in Linux kernel based on TI-LFAT file system and evaluate it with real-world applications. The experimental results show that the proposed mechanism can reduce 30%–80% memory footprints in comparison with HDiffPatch, the state-of-the-art incremental update algorithm. It also significantly reduces the recovery time when power failure or system crash occurs.

Assessing the potential impact of assimilating total surface current velocities in the Met Office’s global ocean forecasting system

Accurate prediction of ocean surface currents is important for marine safety, ship routing, tracking of pollutants and in coupled forecasting. Presently, velocity observations are not routinely assimilated in global ocean forecasting systems, largely due to the sparsity of the observation network. Several satellite missions are now being proposed with the capability to measure Total Surface Current Velocities (TSCV). If successful, these would substantially increase the coverage of ocean current observations and could improve accuracy of ocean current forecasts through data assimilation. In this paper, Observing System Simulation Experiments (OSSEs) are used to assess the impact of assimilating TSCV in the Met Office’s global ocean forecasting system. Synthetic observations are generated from a high-resolution model run for all standard observation types (sea surface temperature, profiles of temperature and salinity, sea level anomaly and sea ice concentration) as well as TSCV observations from a Sea surface KInematics Multiscale monitoring (SKIM) like satellite. The assimilation of SKIM like TSCV observations is tested over an 11 month period. Preliminary experiments assimilating idealised single TSCV observations demonstrate that ageostrophic velocity corrections are not well retained in the model. We propose a method for improving ageostrophic currents through TSCV assimilation by initialising Near Inertial Oscillations with a rotated incremental analysis update (IAU) scheme. The OSSEs show that TSCV assimilation has the potential to significantly improve the prediction of velocities, particularly in the Western Boundary Currents, Antarctic Circumpolar Current and in the near surface equatorial currents. For global surface velocity the analysis root-mean-square-errors (RMSEs) are reduced by 23% and there is a 4-day gain in forecast RMSE. There are some degradations to the subsurface in the tropics, generally in regions with complex vertical salinity structures. However, outside of the tropics, improvements are seen to velocities throughout the water column. Globally there are also improvements to temperature and sea surface height when TSCV are assimilated. The TSCV assimilation largely corrects the geostrophic ocean currents, but results using the rotated IAU method show that the energy at inertial frequencies can be improved with this method. Overall, the experiments demonstrate significant potential benefit of assimilating TSCV observations in a global ocean forecasting system.

DLLog: An Online Log Parsing Approach for Large-Scale System

Syslog is a critical data source for analyzing system problems. Converting unstructured log entries into structured log data is necessary for effective log analysis. However, existing log parsing methods demonstrate promising accuracy on limited datasets, but their generalizability and precision are uncertain when applied to diverse log data. Enhancements in these areas are necessary. This paper proposes an online log parsing method called DLLog, which is based on deep learning and has the longest common subsequence. DLLog utilizes the GRU neural network to mine template words and applies the longest common subsequence to parse log entries in real-time. In the offline stage, DLLog combines multiple log features to accurately extract the template words, creating a log template set to assist online log parsing. In the online stage, DLLog parses log entries by calculating the matching degree between the real-time log entry and the log template in the log template set. This method also supports the incremental update of the log template set to handle new log entries generated by systems. We summarized the previous works and validated DLLog using real log data collected from 16 systems. The results demonstrate that DLLog achieves high parsing accuracy, universality, and adaptability.