Style Algorithm Research Articles

Achieving Zero Constraint Violation for Constrained Reinforcement Learning via Conservative Natural Policy Gradient Primal-Dual Algorithm

We consider the problem of constrained Markov decision process (CMDP) in continuous state actions spaces where the goal is to maximize the expected cumulative reward subject to some constraints. We propose a novel Conservative Natural Policy Gradient Primal Dual Algorithm (CNPGPD) to achieve zero constraint violation while achieving state of the art convergence results for the objective value function. For general policy parametrization, we prove convergence of value function to global optimal upto an approximation error due to restricted policy class. We improve the sample complexity of existing constrained NPGPD algorithm. To the best of our knowledge, this is the first work to establish zero constraint violation with Natural policy gradient style algorithms for infinite horizon discounted CMDPs. We demonstrate the merits of proposed algorithm via experimental evaluations.

Reinforcement learning with dynamic convex risk measures

AbstractWe develop an approach for solving time‐consistent risk‐sensitive stochastic optimization problems using model‐free reinforcement learning (RL). Specifically, we assume agents assess the risk of a sequence of random variables using dynamic convex risk measures. We employ a time‐consistent dynamic programming principle to determine the value of a particular policy, and develop policy gradient update rules that aid in obtaining optimal policies. We further develop an actor–critic style algorithm using neural networks to optimize over policies. Finally, we demonstrate the performance and flexibility of our approach by applying it to three optimization problems: statistical arbitrage trading strategies, financial hedging, and obstacle avoidance robot control.

Co-Learning Bayesian Optimization.

Bayesian optimization (BO) is well known to be sample efficient for solving black-box problems. However, BO algorithms may get stuck in suboptimal solutions even with plenty of samples. Intrinsically, such a suboptimal problem of BO can attribute to the poor surrogate accuracy of the trained Gaussian process (GP), particularly that in the regions where the optimal solutions locate. Hence, we propose to build multiple GP models instead of a single GP surrogate to complement each other, thus resolving the suboptimal problem of BO. Nevertheless, according to the bias-variance tradeoff equation, the individual prediction errors can increase when increasing the diversity of models, which may lead even worse overall surrogate accuracy. On the other hand, based on the theory of the Rademacher complexity, it has been proven that exploiting the agreement of models on unlabeled information can reduce the complexity of hypothesis space, therefore achieving the required surrogate accuracy with fewer samples. Such value of model agreement has been extensively demonstrated for co-training style algorithms to boost model accuracy with a small portion of samples. Inspired by the above, we propose a novel BO algorithm labeled as co-learning BO (CLBO), which exploits both model diversity and agreement on unlabeled information to improve the overall surrogate accuracy with limited samples, therefore achieving more efficient global optimization. Through tests on five numerical toy problems and three engineering benchmarks, the effectiveness of the proposed CLBO has been well demonstrated.

A k-means method for trends of time series: An application to time series of COVID-19 cases in Japan.

A k-means method style clustering algorithm is proposed for trends of multivariate time series. The usual k-means method is based on distances or dissimilarity measures among multivariate data and centroids of clusters. Some similarity or dissimilarity measures are also available for multivariate time series. However, suitability of dissimilarity measures depends on the properties of time series. Moreover, it is not easy to define the centroid for time series. The k-medoid clustering method can be applied to time series using one of dissimilarity measures without using centroids. However, the k-medoid method becomes restrictive if appropriate medoids do not exist. In this paper, the centroid is defined as a common trend and a dissimilarity measure is also introduced for trends. Based on these centroids and dissimilarity measures, a k-means method style algorithm is proposed for a multivariate trend. The proposed method is applied to the time series of COVID-19 cases in each prefecture of Japan.

Dose calculation and reporting with a linear Boltzman transport equation solver in vertebral SABR.

Vertebral Stereotactic ablative body radiotherapy (SABR) involves substantial tumour density heterogeneities. We evaluated the impact of a linear Boltzmann transport equation (LBTE) solver dose calculation on vertebral SABR dose distributions. A sequential cohort of 20 patients with vertebral metastases treated with SABR were selected. Treatment plans were initially planned with a convolution style dose calculation algorithm. The plan was copied and recalculated with a LBTE algorithm reporting both dose to water (Dw) or dose to medium (Dm). Target dose as a function of CT number, and spinal cord dose was compared between algorithms. Compared with a convolution algorithm, there was minimal change in PTV D90% with LBTE. LBTE reporting Dm resulted in reduced GTV D50% by (mean, 95% CI) 2.2% (1.9-2.6%) and reduced Spinal Cord PRV near-maximum dose by 3.0% (2.0-4.1%). LBTE reporting Dw resulted in increased GTV D50% by 2.4% (1.8-3.0%). GTV D50% decreased or increased with increasing CT number with Dm or Dw respectively. LBTE, reporting either Dm or Dw resulted in decreased central spinal cord dose by 8.7% (7.1-10.2%) and 7.2% (5.7-8.8%) respectively. Reported vertebral SABR tumour dose when calculating with an LBTE algorithm depends on tumour density. Spinal cord near-maximum dose was lower when using LBTE algorithm reporting Dm, which may result in higher spinal cord doses being delivered than with a convolution style algorithm. Spinal cord central dose was significantly lower with LBTE, potentially reflecting LBTE transport approximations.

Data driven orthogonal basis selection for functional data analysis

Functional data analysis is typically performed in two steps: first, functionally representing discrete observations, and then applying functional methods, such as the functional principal component analysis, to the so-represented data. While the initial choice of a functional representation may have a significant impact on the second phase of the analysis, this issue has not gained much attention in the past. Typically, a rather ad hoc choice of some standard basis such as Fourier, wavelets, splines, etc. is used for the data transforming purpose. To address this important problem, we present its mathematical formulation, demonstrate its importance, and propose a data-driven method of functionally representing observations. The method chooses an initial functional basis by an efficient placement of the knots. A simple machine learning style algorithm is utilized for the knot selection and recently introduced orthogonal spline bases - splinets - are eventually taken to represent the data. The benefits are illustrated by examples of analyses of sparse functional data.

A novel sub-Kmeans based on co-training approach by transforming single-view into multi-view

Semi-supervised learning is dedicated to solving the problem of poor model performance caused by the scarcity of labeled samples. Co-training algorithm, as a representative of semi-supervised learning algorithms, through constructing the diversity of classifiers, is used to solve the shortage of label samples that leads to low classifier accuracy. However, traditional Co-training style algorithms have strong constraints: (1) there are multiple views of data, and (2) the randomness of data views is relatively large, which leads to the low stability of the Co-training style algorithms. To optimize and solve the shortcomings of traditional Co-training style algorithms, a novel Co-training method based on sub-Kmeans named Op-FSCO is proposed. The proposed approach uses an optimal subspace construct method to extend Co-training to single-view data, greatly improving the application field of Co-training style algorithm. The clustering subspace is generated by plotting the probability of the correlation between the feature and the mutual information measure between the feature and the class label. For the resulting cluster subspace, the top two are selected to construct different views. Finally, experiments on a synthetic dataset, UCI dataset, and real-world dataset demonstrate that the proposed approach is more effective than other Co-training style algorithms. On the UCI benchmark dataset, the performance of the Op-FSCO algorithm on 15 UCI datasets is significantly better than the traditional Co-training algorithm. And on the cere bral stroke dataset, our approach is 2%–5% higher than that of traditional Co-training style algorithms.

Periodic Communities Mining in Temporal Networks: Concepts and Algorithms

Periodicity is a frequently happening phenomenon for social interactions in temporal networks. Mining periodic communities are essential to understanding periodic group behaviors in temporal networks. Unfortunately, most previous studies for community mining in temporal networks ignore the periodic patterns of communities. In this paper, we study the problem of seeking periodic communities in a temporal network, where each edge is associated with a set of timestamps. We propose novel models, including <inline-formula><tex-math notation="LaTeX">$\sigma$</tex-math></inline-formula> -periodic <inline-formula><tex-math notation="LaTeX">$k$</tex-math></inline-formula> -core and <inline-formula><tex-math notation="LaTeX">$\sigma$</tex-math></inline-formula> -periodic <inline-formula><tex-math notation="LaTeX">$k$</tex-math></inline-formula> -clique, that represent periodic communities in temporal networks. Specifically, a <inline-formula><tex-math notation="LaTeX">$\sigma$</tex-math></inline-formula> -periodic <inline-formula><tex-math notation="LaTeX">$k$</tex-math></inline-formula> -core (or <inline-formula><tex-math notation="LaTeX">$\sigma$</tex-math></inline-formula> -periodic <inline-formula><tex-math notation="LaTeX">$k$</tex-math></inline-formula> -clique) is a <inline-formula><tex-math notation="LaTeX">$k$</tex-math></inline-formula> -core (or clique with size larger than <inline-formula><tex-math notation="LaTeX">$k$</tex-math></inline-formula> ) that appears at least <inline-formula><tex-math notation="LaTeX">$\sigma$</tex-math></inline-formula> times periodically in the temporal graph. The problem of searching periodic core is efficient but the resulting communities may be not enough cohesive; the problem of enumerating all periodic cliques is not efficient (NP-hard) but the resulting communities are very cohesive. To compute all of them efficiently, we first develop two effective graph reduction techniques to significantly prune the temporal graph. Then, we transform the temporal graph into a static graph and prove that mining the periodic communities in the temporal graph equals mining communities in the transformed graph. Subsequently, we propose a decomposition algorithm to search maximal <inline-formula><tex-math notation="LaTeX">$\sigma$</tex-math></inline-formula> -periodic <inline-formula><tex-math notation="LaTeX">$k$</tex-math></inline-formula> -core, a Bron-Kerbosch style algorithm to enumerate all maximal <inline-formula><tex-math notation="LaTeX">$\sigma$</tex-math></inline-formula> -periodic <inline-formula><tex-math notation="LaTeX">$k$</tex-math></inline-formula> -cliques, and a branch-and-bound style algorithm to find the maximum <inline-formula><tex-math notation="LaTeX">$\sigma$</tex-math></inline-formula> -periodic clique. The results of extensive experiments on five real-life datasets demonstrate the efficiency, scalability, and effectiveness of our algorithms.

Data Anomaly Detection for Internet of Vehicles Based on Traffic Cellular Automata and Driving Style.

The data validity of safe driving in the Internet of Vehicles (IoV) is the basis of improving the safety of vehicles. Different from a traditional information systems, the data anomaly analysis of vehicle safety driving faces the diversity of data anomaly and the randomness and subjectivity of the driver’s driving behavior. How to combine the characteristics of the IOV data with the driving style analysis to provide effective real-time anomaly data detection has become an important issue in the IOV applications. This paper aims at the critical safety data analysis, considering the large computing cost generated by the real-time anomaly detection of all data in the data package. We preprocess it through the traffic cellular automata model which is built to achieve the ideal abnormal detection effect with limited computing resources. On the basis of this model, the Anomaly Detection based on Driving style (ADD) algorithm is proposed to realize real-time and online detection of anomaly data related to safe driving. Firstly, this paper designs the driving coefficient and proposes a driving style quantization model to represent the driving style of the driver. Then, based on driving style quantization model and vehicle driving state information, a data anomaly detection algorithm is developed by using Gaussian mixture model (GMM). Finally, combining with the application scenarios of multi-vehicle collaboration in the Internet of Vehicles, this paper uses real data sets and simulation data sets to analyze the effectiveness of the proposed ADD algorithm.

Improving parallel efficiency for asynchronous graph analytics using Gauss‐Seidel‐based matrix computation

SummaryGraph analytics is extensively used in big‐data applications such as social networks, web analysis, bio‐informatics, etc. Most graph processing frameworks adopt vertex‐centric model due to its ease of use and programming. However, when dealing with asynchronous graph analytics, frameworks based on vertex programming perform inefficiently. The reason is that first, vertex programming must guarantee the sequential consistency, which means frequent use of locks or atomic operations, and second, the algorithms are parallelized in vertex level and latent parallelism of the algorithms cannot be exploited. To improve parallel efficiency of asynchronous graph processing, the Gauss‐Seidel style algorithms in particular, this paper proposes a scheduling model using Gauss‐Seidel‐based matrix computation, which converts the vertex programming into two main matrix operations and then algorithms are parallelized by row and column vectors. Compared to vertex programming, our model parallelizes algorithms in a finer way to exploit more latent parallelism, while retains the ease‐of‐programming advantage of vertex programming. Instead of using locks to guarantee the sequential consistency, our model uses a hybrid synchronization policy to reduce serializability among threads and overheads of context switching. Furthermore, this model strengthens locality of the program. Experiment results show that our model outperforms vertex‐centric asynchronous frameworks in both performance and scalability. Moreover, it even surpasses the matrix‐based synchronous framework GraphMat with some non‐Gauss‐Seidel style algorithms.

Improved Online Sequential Extreme Learning Machine: A New Intelligent Evaluation Method for AZ-Style Algorithms

Researches on computer games for Go, Chess, and Japanese Chess stand out as one of the notable landmarks in the progress of artificial intelligence. AlphaGo, AlphaGo Zero, and AlphaZero algorithms, which are called AlphaZero style (AZ-style) algorithms in some literature [1] , have achieved superhuman performance by using deep reinforcement learning (DRL). However, the unavailability of training details, expensive equipment used for model training, and the low evaluation accuracy resulted by slow self-play training without expensive computing equipment in practical applications have been the defects of AZ-style algorithms. To solve the problems to a certain extent, the paper proposes an improved online sequential extreme learning machine (IOS-ELM), a new evaluation method, to evaluate chess board positions for AZ-style algortihm. Firstly, the theoretical principles of IOS-ELM is given. Secondly, the study considers Gomoku as the application object and uses IOS-ELM as the evaluation method for AZ-style’s board positions to discuss the loss in the training process and hyperparameters affecting performance in detail. Under the same experimental conditions, the proposed method reduces the training parameters by 14 times, training time to 15%, and error of evaluation by 13% compared with the board evaluation network used in original AZ-style algorithms.

A Sparse Manifold Learning Approach to Robust Indoor Positioning Based on Wi-Fi RSS Fingerprinting

The emerging location-based applications depend on the fast and accurate positioning of mobile targets. Wi-Fi received signal strength (RSS) fingerprinting provides a promising solution to localize an object in indoor environments. Among the factors challenging the RSS fingerprinting based algorithms are the site survey cost and the time-varying environment, given the unreliable signal qualities. Here we present a novel approach to indoor object positioning using the manifold assumption on the radio map in RSS-location space. Thinking the measured RSS from one access point (AP) in different locations are randomly drawn from a nonlinear manifold (ground truth), we propose an expectation-maximization (EM) style algorithm to reconstruct the sparse representation of this manifold from the noisy RSS observations. Motivated by the observation that the radio map has a strong local correlation in the RSS-location space, we introduce a multi-scale constrained quadratic programming to approximate the manifold. Within limited iterations, we can estimate the ground truth RSS values and parameters simultaneously. As a result, the learned manifold is exploited to predict the object's position: we develop a positioning algorithm by minimizing the manifold distortion effort which integrates both measurement error and manifold shape preservation. We conducted extensive simulations and experiments in different settings, testing the datasets collected in a building in the last 8 months. The results showed that the proposed approach was adaptive to the varying environmental noise levels, presenting robust positioning performance.

A polynomial-time algorithm to compute generalized Hermite normal forms of matrices over [formula omitted

In this paper, we give the first polynomial time algorithm to compute the generalized Hermite normal form for a matrix F over Z[x], or equivalently, the reduced Gröbner basis of the Z[x]-module generated by the column vectors of F. The algorithm has polynomial bit size computational complexities and is also shown to be practically more efficient than existing algorithms. The algorithm is based on three key ingredients. First, an F4 style algorithm to compute the Gröbner basis is adopted, where a novel prolongation is designed such that the sizes of coefficient matrices under consideration are nicely controlled. Second, the complexity bound of the algorithm is achieved by a nice estimation for the degree and height bounds of the polynomials in the generalized Hermite normal form. Third, fast algorithms to compute Hermite normal forms of matrices over Z are used as the computational tool.

Multi-view clustering: A survey

In the big data era, the data are generated from different sources or observed from different views. These data are referred to as multi-view data. Unleashing the power of knowledge in multi-view data is very important in big data mining and analysis. This calls for advanced techniques that consider the diversity of different views, while fusing these data. Multi-view Clustering (MvC) has attracted increasing attention in recent years by aiming to exploit complementary and consensus information across multiple views. This paper summarizes a large number of multi-view clustering algorithms, provides a taxonomy according to the mechanisms and principles involved, and classifies these algorithms into five categories, namely, co-training style algorithms, multi-kernel learning, multiview graph clustering, multi-view subspace clustering, and multi-task multi-view clustering. Therein, multi-view graph clustering is further categorized as graph-based, network-based, and spectral-based methods. Multi-view subspace clustering is further divided into subspace learning-based, and non-negative matrix factorization-based methods. This paper does not only introduce the mechanisms for each category of methods, but also gives a few examples for how these techniques are used. In addition, it lists some publically available multi-view datasets. Overall, this paper serves as an introductory text and survey for multi-view clustering.

Dual Set Multi-Label Learning

In this paper, we propose a new learning framework named dual set multi-label learning, where there are two sets of labels, and an object has one and only one positive label in each set. Compared to general multi-label learning, the exclusive relationship among labels within the same set, and the pairwise inter-set label relationship are much more explicit and more likely to be fully exploited. To handle such kind of problems, a novel boosting style algorithm with model-reuse and distribution adjusting mechanisms is proposed to make the two label sets help each other. In addition, theoretical analyses are presented to show the superiority of learning from dual label sets to learning directly from all labels. To empirically evaluate the performance of our approach, we conduct experiments on two manually collected real-world datasets along with an adapted dataset. Experimental results validate the effectiveness of our approach for dual set multi-label learning.

P.3.b.043 - Two-year follow-up of patients with a first psychotic episode: comparison between affective and non-affective psychoses and predictors of functioning

Dense registration of different impressions of the same finger is beneficial to various fingerprint matching methods. This is a challenging problem due to elastic distortion of finger skin and sparsity of distinctive features (namely minutiae) in fingerprints. Most existing fingerprint registration algorithms produce only correspondences between minutiae, which are not sufficient for dense registration of fingerprints. In this paper, we proposed a novel dense fingerprint registration algorithm, which consists of a composite initial registration step and a dual-resolution block-based registration step. The dual-resolution block-based registration is approached in an energy minimization framework which consists of local search, energy function construction and global optimization. In local search step, a candidate set of transformations of every input image block are found using image correlation w.r.t. the corresponding reference image block. In energy function construction, two factors are considered: (1) the similarity between the transformed input block and the corresponding reference block, and (2) the compatibility between transformations of neighboring input blocks. In global optimization, a region growing style algorithm is proposed to minimize the energy function. Experimental results on three databases containing many distorted fingerprints, namely FVC2004 DB1, Tsinghua Distorted Fingerprint database and NIST SD27 latent fingerprint database, show that the proposed algorithm not only produces more accurate registration results but also improves the matching performance by fusion of minutiae matching and image correlation.

Software Physiognomics: Adorno's Radio Analytics Today

In “Radio Physiognomics” Theodor W. Adorno outlines the social features of music's radio broadcast, a still relatively new technology for music transmission and listening in the 1930s. The present article deploys Adorno's “physiognomic” analytics to diagnose the kinds of perceptual technics at work in the data-encoding methods associated with digital technologies today, from compression and decompression schemes for file formats to software for instrument and style simulation and algorithms for identifying signature grooves and metric beats in music. In the shift from autonomous listening (always already institutionalized in terms of habitus) to automatic listening (absolutizing habitus through executable algorithms), the article detects a link to technological misrecognition, whereby the sensuous/immediate and humanizing must—as a condition of its possibility—also foreclose what goes as human affect.

Dense registration of fingerprints

Dense registration of different impressions of the same finger is beneficial to various fingerprint matching methods. This is a challenging problem due to elastic distortion of finger skin and sparsity of distinctive features (namely minutiae) in fingerprints. Most existing fingerprint registration algorithms produce only correspondences between minutiae, which are not sufficient for dense registration of fingerprints. In this paper, we proposed a novel dense fingerprint registration algorithm, which consists of a composite initial registration step and a dual-resolution block-based registration step. The dual-resolution block-based registration is approached in an energy minimization framework which consists of local search, energy function construction and global optimization. In local search step, a candidate set of transformations of every input image block are found using image correlation w.r.t. the corresponding reference image block. In energy function construction, two factors are considered: (1) the similarity between the transformed input block and the corresponding reference block, and (2) the compatibility between transformations of neighboring input blocks. In global optimization, a region growing style algorithm is proposed to minimize the energy function. Experimental results on three databases containing many distorted fingerprints, namely FVC2004 DB1, Tsinghua Distorted Fingerprint database and NIST SD27 latent fingerprint database, show that the proposed algorithm not only produces more accurate registration results but also improves the matching performance by fusion of minutiae matching and image correlation.

Whole genome sequencing of Streptococcus pneumoniae: development, evaluation and verification of targets for serogroup and serotype prediction using an automated pipeline.

Streptococcus pneumoniae typically express one of 92 serologically distinct capsule polysaccharide (cps) types (serotypes). Some of these serotypes are closely related to each other; using the commercially available typing antisera, these are assigned to common serogroups containing types that show cross-reactivity. In this serotyping scheme, factor antisera are used to allocate serotypes within a serogroup, based on patterns of reactions. This serotyping method is technically demanding, requires considerable experience and the reading of the results can be subjective. This study describes the analysis of the S. pneumoniae capsular operon genetic sequence to determine serotype distinguishing features and the development, evaluation and verification of an automated whole genome sequence (WGS)-based serotyping bioinformatics tool, PneumoCaT (Pneumococcal Capsule Typing). Initially, WGS data from 871 S. pneumoniae isolates were mapped to reference cps locus sequences for the 92 serotypes. Thirty-two of 92 serotypes could be unambiguously identified based on sequence similarities within the cps operon. The remaining 60 were allocated to one of 20 ‘genogroups’ that broadly correspond to the immunologically defined serogroups. By comparing the cps reference sequences for each genogroup, unique molecular differences were determined for serotypes within 18 of the 20 genogroups and verified using the set of 871 isolates. This information was used to design a decision-tree style algorithm within the PneumoCaT bioinformatics tool to predict to serotype level for 89/94 (92 + 2 molecular types/subtypes) from WGS data and to serogroup level for serogroups 24 and 32, which currently comprise 2.1% of UK referred, invasive isolates submitted to the National Reference Laboratory (NRL), Public Health England (June 2014–July 2015). PneumoCaT was evaluated with an internal validation set of 2065 UK isolates covering 72/92 serotypes, including 19 non-typeable isolates and an external validation set of 2964 isolates from Thailand (n = 2,531), USA (n = 181) and Iceland (n = 252). PneumoCaT was able to predict serotype in 99.1% of the typeable UK isolates and in 99.0% of the non-UK isolates. Concordance was evaluated in UK isolates where further investigation was possible; in 91.5% of the cases the predicted capsular type was concordant with the serologically derived serotype. Following retesting, concordance increased to 99.3% and in most resolved cases (97.8%; 135/138) discordance was shown to be caused by errors in original serotyping. Replicate testing demonstrated that PneumoCaT gave 100% reproducibility of the predicted serotype result. In summary, we have developed a WGS-based serotyping method that can predict capsular type to serotype level for 89/94 serotypes and to serogroup level for the remaining four. This approach could be integrated into routine typing workflows in reference laboratories, reducing the need for phenotypic immunological testing.

Improving semi-supervised self-training with embedded manifold transduction

Semi-supervised learning aims to utilize both labelled and unlabelled data to improve learning performance. This paper shows a distinct way to exploit unlabelled data for traditional semi-supervised learning methods, such as self-training. Self-training is a well-known semi-supervised learning algorithm which iteratively trains a classifier by bootstrapping from unlabelled data. Standard self-training barely selects unlabelled examples for training set augmentation according to the current classifier model, which is trained only on the labelled data. This could be problematic since the underlying classifier is not strong enough, especially when initial labelled data is sparse. Consequently, self-training suffers from too much classification noise accumulated in the training set. In this paper, we propose a novel self-training style algorithm, which exploits a manifold assumption to optimize the self-labelling process. Unlike standard self-training, our algorithm utilizes labelled and unlabelled data as a whole to label and select unlabelled examples for training set augmentation. In detail, two measures are employed to minimize the effect of noise introduced to the labelled training set: a transductive method based on controlled graph random walk is incorporated to generate reliable predictions on unlabelled data; secondly, the mechanism is adopted to sequentially augment the training set. Empirical results suggest that the proposed method can effectively improve classification performance.