Batch Algorithms Research Articles

A fast neural network approach to predict lung tumor motion during respiration for radiation therapy applications.

During radiotherapy treatment for thoracic and abdomen cancers, for example, lung cancers, respiratory motion moves the target tumor and thus badly affects the accuracy of radiation dose delivery into the target. A real-time image-guided technique can be used to monitor such lung tumor motion for accurate dose delivery, but the system latency up to several hundred milliseconds for repositioning the radiation beam also affects the accuracy. In order to compensate the latency, neural network prediction technique with real-time retraining can be used. We have investigated real-time prediction of 3D time series of lung tumor motion on a classical linear model, perceptron model, and on a class of higher-order neural network model that has more attractive attributes regarding its optimization convergence and computational efficiency. The implemented static feed-forward neural architectures are compared when using gradient descent adaptation and primarily the Levenberg-Marquardt batch algorithm as the ones of the most common and most comprehensible learning algorithms. The proposed technique resulted in fast real-time retraining, so the total computational time on a PC platform was equal to or even less than the real treatment time. For one-second prediction horizon, the proposed techniques achieved accuracy less than one millimeter of 3D mean absolute error in one hundred seconds of total treatment time.

Machine-generated algorithms, proofs and software for the batch verification of digital signature schemes

As devices everywhere increasingly communicate with each other, many security applications will require low-bandwidth signatures that can be processed quickly. Pairing-based signatures can be very short, but are often costly to verify. Fortunately, they also tend to have efficient batch verification algorithms. Finding these batching algorithms by hand, however, can be tedious and error prone.We address this by presenting AutoBatch, an automated tool for generating batch verification code in either Python or C++ from a high level representation of a signature scheme. AutoBatch outputs both software and, for transparency, a LaTeX file describing the batching algorithm and arguing that it preserves the unforgeability of the original scheme.We tested AutoBatch on over a dozen pairing-based schemes to demonstrate that a computer could find competitive batching solutions in a reasonable amount of time. In particular, it found an algorithm that is faster than a batching algorithm from Eurocrypt 2010. Another novel contribution is that it handles cross-scheme batching, where it searches for a common algebraic structure between two distinct schemes and attempts to batch them together.In this work, we expand upon our paper on AutoBatch appearing in ACM CCS 2012 [in: Proceedings of the 2012 ACM Conference on Computer and Communications Security, CCS'12, ACM, New York, NY, USA, 2012, pp. 474–487] in a number of ways. We add a new loop-unrolling technique and show that it helps cut the batch verification cost of one scheme by roughly half. We describe our pruning and search algorithms in greater detail, including pseudocode and diagrams. All experiments were also re-run using the RELIC pairing library. We compare those results to our earlier results using the MIRACL library, and discuss why RELIC outperforms MIRACL in all but two cases. Automated proofs of several new batching algorithms are also included.AutoBatch is a useful tool for cryptographic designers and implementors, and to our knowledge, it is the first attempt to outsource to machines the design, proof writing and implementation of signature batch verification schemes.

Adaptive Algorithm for Anti-Jamming in Broadband Communication

This paper presents a method to suppress narrowband interference in the direct sequence spread spectrum communication based frequency domain adaptive algorithm. The method determines the interference threshold by N-sigma algorithm and calculates the cut-off condition by separating the signal. The method suppresses the narrowband interference by frequency domain batch algorithm. This paper gives a detailed design method and simulation results, and makes a performance comparison with other methods. The results show that, the method in this paper has better performance than other methods. It is suitable for narrowband interference suppression in broadband systems.

Interaction-Rich Transfer Learning for Collaborative Filtering with Heterogeneous User Feedback

A real recommender system can usually make use of more than one type of user feedback--for example, numerical ratings and binary ratings-to learn a user's true preferences. Recent work has proposed a transfer learning algorithm called transfer by collective factorization (TCF) to exploit such heterogeneous user feedback. TCF performs via sharing data-independent knowledge and modeling data-dependent effects simultaneously. However, TCF is a batch algorithm and updates the model parameters only once after scanning all data, which might not be efficient enough for real systems. This article proposes a novel and efficient transfer learning algorithm called interaction-rich transfer by collective factorization (iTCF), which extends the efficient collective matrix factorization (CMF) algorithm by providing more interactions between the user-specific latent features. The assumption under iTCF is that the predictability with regards to the same user's rating behaviors in two related data is likely to be similar. Considering the shared predictability, the authors derive novel update rules for iTCF in a stochastic algorithmic framework. The advantages of iTCF include its efficiency compared with TCF, and its higher prediction accuracy compared with CMF. Experimental results on three real-world datasets show the effectiveness of iTCF over the state-of-the-art methods.

A Recursive Restricted Total Least-Squares Algorithm

We show that the generalized total least squares (GTLS) problem with a singular noise covariance matrix is equivalent to the restricted total least squares (RTLS) problem and propose a recursive method for its numerical solution. The method is based on the generalized inverse iteration. The estimation error covariance matrix and the estimated augmented correction are also characterized and computed recursively. The algorithm is cheap to compute and is suitable for online implementation. Simulation results in least squares (LS), data least squares (DLS), total least squares (TLS), and restricted total least squares (RTLS) noise scenarios show fast convergence of the parameter estimates to their optimal values obtained by corresponding batch algorithms.

Incremental Generalized Discriminative Common Vectors for Image Classification.

Subspace-based methods have become popular due to their ability to appropriately represent complex data in such a way that both dimensionality is reduced and discriminativeness is enhanced. Several recent works have concentrated on the discriminative common vector (DCV) method and other closely related algorithms also based on the concept of null space. In this paper, we present a generalized incremental formulation of the DCV methods, which allows the update of a given model by considering the addition of new examples even from unseen classes. Having efficient incremental formulations of well-behaved batch algorithms allows us to conveniently adapt previously trained classifiers without the need of recomputing them from scratch. The proposed generalized incremental method has been empirically validated in different case studies from different application domains (faces, objects, and handwritten digits) considering several different scenarios in which new data are continuously added at different rates starting from an initial model.

Online Multi-Task Learning via Sparse Dictionary Optimization

This paper develops an efficient online algorithm for learning multiple consecutive tasks based on the K-SVD algorithm for sparse dictionary optimization. We first derive a batch multi-task learning method that builds upon K-SVD, and then extend the batch algorithm to train models online in a lifelong learning setting. The resulting method has lower computational complexity than other current lifelong learning algorithms while maintaining nearly identical model performance. Additionally, the proposed method offers an alternate formulation for lifelong learning that supports both task and feature similarity matrices.

Efficient Batching Algorithm for Charging Slabs into a Continuous Furnace

An algorithm and a program have been created for efficiently batching the slab charge of a continuous heating furnace. The charge is composed with allowance for each slab’s heating group.

P2P Ranging‐Based Cooperative Localization Method for a Cluster of Mobile Nodes Containing IR‐UWB PHY

The problem of pedestrian localization using mobile nodes containing impulse radio ultra wideband (IR‐UWB) is considered. IEEE 802.15.4a‐based IR‐UWB can achieve accurate ranging. However, the coverage is as short as 30 m, owing to the restricted transmit power. This factor may cause a poor geometric relationship among the mobile nodes and anchor nodes in certain environments. To localize a group of pedestrians accurately, an enhanced cooperative localization method is proposed. We describe a sequential algorithm and define problems that may occur in the implementation of the algorithm. To solve these problems, a batch algorithm is proposed. The batch algorithm can be carried out after performing the sequential algorithm to linearize the nonlinear range equation. When a sequential algorithm cannot be performed due to a poor geometric relationship among nodes, a batch algorithm can be carried out directly. Herein, Monte Carlo simulations are presented to illustrate the proposed method and verify its performance.

A Novel Unconstrained Criterion for Blind Equalization Based on the Property of Vector Norm

This paper proposes a novel unconstrained criterion of blind equalization, which is based on minimizing the difference between two norms of the joint channel-equalizer impulse response. There is no need to impose any constraint on the equalizer coefficients when adjusting. The algorithm is convergent in the sense that a perfect equalization solution is achieved at every local optimum of the function. And we propose a batch algorithm and online-recursive method for implementation. Finally, the reliability, validity and good performance of the proposed algorithm are verified by evaluation of residual ISI through computer simulations.

Incremental graph pattern matching

Graph pattern matching is commonly used in a variety of emerging applications such as social network analysis. These applications highlight the need for studying the following two issues. First, graph pattern matching is traditionally defined in terms of subgraph isomorphism or graph simulation. These notions, however, often impose too strong a topological constraint on graphs to identify meaningful matches. Second, in practice a graph is typically large, and is frequently updated with small changes. It is often prohibitively expensive to recompute matches starting from scratch via batch algorithms when the graph is updated. This article studies these two issues. (1) We propose to define graph pattern matching based on a notion of bounded simulation , which extends graph simulation by specifying the connectivity of nodes in a graph within a predefined number of hops. We show that bounded simulation is able to find sensible matches that the traditional matching notions fail to catch. We also show that matching via bounded simulation is in cubic time, by giving such an algorithm. (2) We provide an account of results on incremental graph pattern matching, for matching defined with graph simulation, bounded simulation, and subgraph isomorphism. We show that the incremental matching problem is unbounded , that is, its cost is not determined alone by the size of the changes in the input and output, for all these matching notions. Nonetheless, when matching is defined in terms of simulation or bounded simulation, incremental matching is semibounded , that is, its worst-time complexity is bounded by a polynomial in the size of the changes in the input, output, and auxiliary information that is necessarily maintained to reuse previous computation, and the size of graph patterns. We also develop incremental matching algorithms for graph simulation and bounded simulation, by minimizing unnecessary recomputation. In contrast, matching based on subgraph isomorphism is neither bounded nor semibounded. (3) We experimentally verify the effectiveness and efficiency of these algorithms, and show that: (a) the revised notion of graph pattern matching allows us to identify communities commonly found in real-life networks, and (b) the incremental algorithms substantially outperform their batch counterparts in response to small changes. These suggest a promising framework for real-life graph pattern matching.

Online heterogeneous feature fusion machines for visual recognition

Heterogeneous Feature Fusion Machines (HFFM) is a kernel based logistic regression model that effectively fuses multiple features for visual recognition tasks. However, the batch mode solution for HFFM, ‘Block Coordinate Gradient Descent’ (BCGD) has the same low efficiency and poor scalability as the most batch algorithms do. In this paper, we describe a newly developed online learning algorithm in multiple Reproducing Kernel Hilbert Spaces for solving HFFM model. This new algorithm is called OLHFFM, i.e. Online HFFM. OLHFFM is novel combination of kernel-based learning technique with dual averaging gradient descent methods. In addition, group LASSO regularization technique is used in OLHFFM for finding important explanatory coefficients that are related to support samples in group manner. The effectiveness of OLHFFM has been demonstrated by a number of experiments that were conducted on public event, object dataset, as well as on large scale handwritten digital dataset. Using the OLHFFM approach, we have achieved almost equivalent recognition performance to that using batch-mode approach. Experiments conducted on both MIT Caltech-6 and challenging VOC2011 TrainVal object datasets show that OLHFFM is superior in performance to kernel based online learning approaches such as ILK or NORMA. In addition, the classification performance of OLHFFM approach as demonstrated by the experiments conducted on large scale MNIST dataset is comparable to or better than that of the current state-of-the-art online multiple kernel learning approaches such as OM-2, UFO-MKL, OMCL and OMKL. Extensive experiments on visual data classification demonstrate the effectiveness and robustness of the new OLHFFM approach.

Incremental learning of privacy-preserving Bayesian networks

Bayesian Networks (BNs) have received significant attention in various academic and industrial applications, such as modeling knowledge in image processing, engineering, medicine and bio-informatics. Preserving the privacy of sensitive data, owned by different parties, is often a critical issue. However, in many practical applications, BNs must train from data that gradually becomes available at different period of times, on which the traditional batch learning algorithms are not suitable or applicable. In this paper, an algorithm based on a new and efficient version of Sufficient Statistics is proposed for incremental learning with BNs. The standard K2 algorithm is also modified to be utilized inside the incremental learning algorithm. Next, some secure building blocks such as secure comparison, and factorial, which are resistant against colluding attacks and could be applied securely over public channels like internet, are presented to be used inside the main protocol. Then a privacy-preserving protocol is proposed for incremental learning of BNs, in which the structure and probabilities are estimated incrementally from homogeneously distributed and gradually available data among two or multi-parties. Finally, security and complexity analysis along with the experimental results are presented to compare with the batch algorithm and to show its performance and applicability in real world applications.

PETRELS: Parallel Subspace Estimation and Tracking by Recursive Least Squares From Partial Observations

Many real world data sets exhibit an embedding of low-dimensional structure in a high-dimensional manifold. Examples include images, videos and internet traffic data. It is of great significance to reduce the storage requirements and computational complexity when the data dimension is high. Therefore we consider the problem of reconstructing a data stream from a small subset of its entries, where the data is assumed to lie in a low-dimensional linear subspace, possibly corrupted by noise. We further consider tracking the change of the underlying subspace, which can be applied to applications such as video denoising, network monitoring and anomaly detection. Our problem can be viewed as a sequential low-rank matrix completion problem in which the subspace is learned in an on-line fashion. The proposed algorithm, dubbed Parallel Estimation and Tracking by REcursive Least Squares (PETRELS), first identifies the underlying low-dimensional subspace via a recursive procedure for each row of the subspace matrix in parallel with discounting for previous observations, and then reconstructs the missing entries via least-squares estimation if required. Numerical examples are provided for direction-of-arrival estimation and matrix completion, comparing PETRELS with state of the art batch algorithms.

On-line EM Variants for Multivariate Normal Mixture Model in Background Learning and Moving Foreground Detection

The unsupervised learning of multivariate mixture models from on-line data streams has attracted the attention of researchers for its usefulness in real-time intelligent learning systems. The EM algorithm is an ideal choice for iteratively obtaining maximum likelihood estimation of parameters in presumable finite mixtures, comparing to some popular numerical methods. However, the original EM is a batch algorithm that works only on fixed datasets. To endow the EM algorithm with the capability to process streaming data, two on-line variants are studied, including Titterington's method and a sufficient statistics-based method. We first prove that the two on-line EM variants are theoretically feasible for training the multivariate normal mixture model by showing that the model belongs to the exponential family. Afterward, the two on-line learning schemes for multivariate normal mixtures are applied to the problems of background learning and moving foreground detection. Experiments show that the two on-line EM variants can efficiently update the parameters of the mixture model and are capable of generating reliable backgrounds for moving foreground detection.

A Two-Stage Approach for Network Monitoring

A goal of network tomography is to infer the status (e.g. delay) of congested links internal to a network, through end-to-end measurements at boundary nodes (end-hosts) via insertion of probe signals. Because (a) probing constitutes traffic overhead, and (b) in any typical scenario, the number of congested links is a small fraction of the total number in the network, a desirable design objective is to identify those (few) congested links using a minimum number of probes. In this paper, we make a contribution to solving this problem, by proposing a new two-stage approach for this problem. First, we develop a binary observation model linking end-to-end observations with individual link statuses and derive necessary and sufficient conditions for whether at least one link in the network is congested. Stage I of the proposed method shows that achieving 1-identifiability with a minimum number of probes is equivalent to the familiar minimum set covering problem that can be efficiently solved via a greedy heuristic. A sequential algorithm is described, leading to a significantly lowered computational complexity vis-a-vis a batch algorithm. Next, a binary splitting algorithm originally developed in group testing is used to identify the location of the congested links. The proposed scheme is evaluated by simulations in OPNET and experiments on the PlanetLab testbed to validate the advantages of our 2-stage approach vis-a-vis a conventional (batch) algorithm.

Secure Key Agreement And Authentication Protocols

We consider several distributed collaborative key agreement and authentication protocols for dynamic peer groups. There are several important characteristics which make this problem different from traditional secure group communication. They are: 1) Distributed nature in which there is no centralized key server; 2) Collaborative nature in which the group key is contributory (i.e., each group member will collaboratively contribute its part to the global group key); and 3) Dynamic nature in which existing members may leave the group while new members may join. Instead of performing individual rekeying operations, i.e. recomputing the group key after every join or leave request, we discuss an interval-based approach of rekeying. We consider three intervalbased distributed rekeying algorithms, or interval-based algorithms for short, for updating the group key: 1) the Rebuild algorithm; 2) the Batch algorithm; and 3) the Queue-batch algorithm. Performance of these three interval-based algorithms under different settings, such as different join and leave probabilities, is analyzed. We show that the interval-based algorithms significantly outperform the individual rekeying approach and that the Queue-batch algorithm performs the best among the three interval-based algorithms. More importantly, the Queue-batch algorithm can substantially reduce the computation and communication workload in a highly dynamic environment. We further enhance the interval-based algorithms in two aspects: authentication and implementation. Authentication focuses on the security improvement, while implementation realizes the interval-based algorithms in real network settings. Our work provides a fundamental understanding about establishing a group key via a distributed and collaborative approach for a dynamic peer group.

Incremental threshold learning for classifier selection

Threshold-based classifier is a simple yet powerful pattern classification tool, which has been frequently used in applications of object detection and recognition. A threshold-based classifier is usually associated with a unique one-dimensional feature. A properly selected threshold and a binary sign corresponding to the feature govern the classifier. However, the learning process is usually done in a batch manner. The batch algorithms are not suitable for sequentially incoming data because of the limitation of storage and prohibitive computation cost. To deal with sequentially incoming data, this paper proposes an incremental algorithm for incrementally learning the threshold-based classifiers. The proposed method can not only incrementally model the features but also estimate the threshold and training error in a close form. The effectiveness of the proposed algorithm is evaluated in the applications of gender recognition, face detection, and human detection.

Accurate on-line -support vector learning

The ν-Support Vector Machine (ν-SVM) for classification proposed by Schölkopf et al. has the advantage of using a parameter ν on controlling the number of support vectors and margin errors. However, comparing to standard C-Support Vector Machine (C-SVM), its formulation is more complicated, up until now there are no effective methods on solving accurate on-line learning for it. In this paper, we propose a new effective accurate on-line algorithm which is designed based on a modified formulation of the original ν-SVM. The accurate on-line algorithm includes two special steps: the first one is relaxed adiabatic incremental adjustments; the second one is strict restoration adjustments. The experiments on several benchmark datasets demonstrate that using these two steps the accurate on-line algorithm can avoid the infeasible updating path as far as possible, and successfully converge to the optimal solution. It achieves the fast convergence especially on the Gaussian kernel and is faster than the batch algorithm.

An Effective Contributory Re-Keying Approach to Compute Conference Key

Problem statement: With the explosive growth of and internet and web applications many emerging group-oriented distributed applications such as tele/video-conferencing, multiplayer games are based on group communication model that need security services such as privacy and data integrity Hence a secure distributed group key agreement is required to establish and authenticate a common group key for secure and private communication. There is a need for security services to provide group-oriented communication privacy and data integrity. It is important that members of the group can establish a common secret key for encrypting group communication. A key tree approach has been proposed by many authors to distribute group key in such a way that the rekeying cost scales with the logarithm of the group size for a join or leave request. The efficiency of this key tree approach critically depends on whether the key tree remains balanced over time as members join or leaves Approach: Instead of performing individual re-keying operations, an interval-based approach of re keying is adopted in the proposed scheme. Results: In the proposed scheme Queue-merge algorithm is used for rekeying which substantially reduces the computation cost and communication cost. The comparison shows that queue merge algorithm performs better than Batch algorithm in terms of minimizing the key tree and presumes better node density thereby reducing the computation cost. Conclusion: Performance comparison also shows reduced number of renewed nodes for various rekeying interval which reduces the communication cost.