Continuous Space Model Research Articles

Continuous space models for CLIR

We present and evaluate a novel technique for learning cross-lingual continuous space models to aid cross-language information retrieval (CLIR). Our model, which is referred to as external-data composition neural network (XCNN), is based on a composition function that is implemented on top of a deep neural network that provides a distributed learning framework. Different from most existing models, which rely only on available parallel data for training, our learning framework provides a natural way to exploit monolingual data and its associated relevance metadata for learning continuous space representations of language. Cross-language extensions of the obtained models can then be trained by using a small set of parallel data. This property is very helpful for resource-poor languages, therefore, we carry out experiments on the English-Hindi language pair. On the conducted comparative evaluation, the proposed model is shown to outperform state-of-the-art continuous space models with statistically significant margin on two different tasks: parallel sentence retrieval and ad-hoc retrieval.

Cross-domain deception detection using support vector networks

Our motivation is to assess the effectiveness of support vector networks (SVN) on the task of detecting deception in texts, as well as to investigate to which degree it is possible to build a domain-independent detector of deception in text using SVN. We experimented with different feature sets for training the SVN: a continuous semantic space model source represented by the latent Dirichlet allocation topics, a word-space model, and dictionary-based features. In this way, a comparison of performance between semantic information and behavioral information is made. We tested several combinations of these features on different datasets designed to identify deception. The datasets used include the DeRev dataset (a corpus of deceptive and truthful opinions about books obtained from Amazon), OpSpam (a corpus of fake and truthful opinions about hotels), and three corpora on controversial topics (abortion, death penalty, and a best friend) on which the subjects were asked to write an idea contrary to what they really believed. We experimented with one-domain setting by training and testing our models separately on each dataset (with fivefold cross-validation), with mixed-domain setting by merging all datasets into one large corpus (again, with fivefold cross-validation), and with cross-domain setting: using one dataset for testing and a concatenation of all other datasets for training. We obtained an average accuracy of 86% in one-domain setting, 75% in mixed-domain setting, and 52 to 64% in cross-domain setting.

Advanced neural network prediction and system identification of liquid-liquid flow patterns in circular microchannels with varying angle of confluence

The flow pattern map for liquid-liquid system in a circular microchannel of 600μm was experimentally investigated for a varying confluence angle (10–170 degree) of inlet fluids. The experimental results showing distinguishing nature of transition boundaries were established using graphical interpretation. Investigations were carried out to find a better objective flow pattern indicator for a vast flow pattern observations. Studies on advanced feed-forward back-propagation networks and radial basis networks such as CFN, ANN-FF, ANN-PR, PNN, GRNN and ANFIS showed that GRNN (Generalized Regression Neural Network) showed a better prediction over other prediction techniques with a R2 value of 0.988. The relative impact of input variables such as confluence angle, superficial velocity of dodecane and superficial velocity of water on flow pattern formation was found to be 17.78%, 43.30% and 38.92% respectively. The discrete and continuous time state space models for the system was also developed and interpreted in detail using system identification technique for the better understanding of the system.

Stabilization of species coexistence in spatial models through the aggregation–segregation effect generated by local dispersal and nonspecific local interactions

Spatial interactions are widely acknowledged to play a significant role in sustaining diversity in ecological communities. However, theoretical work on this topic has focused on how spatial processes affect coexistence of species that differ in their strategies, with less attention to how spatial processes matter when competitors are equivalent. Furthermore, though it is recognized that models with local dispersal and local competition may sustain higher diversities of equivalent competitors than models in which these are not both localized, there is debate as to whether this reflects merely equalizing effects or whether there is also a stabilizing component. In this study, we explore how dispersal limitation and nonspecific local competition influence the outcome of species coexistence in communities driven by stochastic drift. We demonstrate that space alone acts as a stabilizing factor in a continuous space model with local dispersal and competition, as individuals of rare species on average experience lower total neighborhood densities, causing per capita reproductive rates to decrease systematically with increasing abundance. These effects prolong time to extinction in a closed system and enhance species diversity in an open system with constant immigration. Fundamentally, these stabilizing effects are obtained when dispersal limitation interacts with local competition to generate fluctuations in population growth rates. Thus this effect can be considered a fluctuating mechanism similar to spatial or temporal storage effects, but generated purely endogenously without requiring any exogenous environmental variability or species dissimilarities.

Linked Lexical Knowledge Bases: Foundations and Applications

This book conveys the fundamentals of Linked Lexical Knowledge Bases (LLKB) and sheds light on their different aspects from various perspectives, focusing on their construction and use in natural language processing (NLP). It characterizes a wide range of both expert-based and collaboratively constructed lexical knowledge bases. Only basic familiarity with NLP is required and this book has been written for both students and researchers in NLP and related fields who are interested in knowledge-based approaches to language analysis and their applications. Lexical Knowledge Bases (LKBs) are indispensable in many areas of natural language processing, as they encode human knowledge of language in machine readable form, and as such, they are required as a reference when machines attempt to interpret natural language in accordance with human perception. In recent years, numerous research efforts have led to the insight that to make the best use of available knowledge, the orchestrated exploitation of different LKBs is necessary. This allows us to not only extend the range of covered words and senses, but also gives us the opportunity to obtain a richer knowledge representation when a particular meaning of a word is covered in more than one resource. Examples where such an orchestrated usage of LKBs proved beneficial include word sense disambiguation, semantic role labeling, semantic parsing, and text classification. This book presents different kinds of automatic, manual, and collaborative linkings between LKBs. A special chapter is devoted to the linking algorithms employing text-based, graph-based, and joint modeling methods. Following this, it presents a set of higher-level NLP tasks and algorithms, effectively utilizing the knowledge in LLKBs. Among them, you will find advanced methods, e.g., distant supervision, or continuous vector space models of knowledge bases (KB), that have become widely used at the time of this book's writing. Finally, multilingual applications of LLKB's, such as cross-lingual semantic relatedness and computer-aided translation are discussed, as well as tools and interfaces for exploring LLKBs, followed by conclusions and future research directions.

Recent advances in machine translation using comparable corpora

AbstractThis paper highlights some of the recent developments in the field of machine translation using comparable corpora. We start by updating previous definitions of comparable corpora and then look at bilingual versions of continuous vector space models. Recently, neural networks have been used to obtain latent context representations with only few dimensions which are often called word embeddings. These promising new techniques cannot only be applied to parallel but also to comparable corpora. Subsequent sections of the paper discuss work specifically targeting at machine translation using comparable corpora, as well as work dealing with the extraction of parallel segments from comparable corpora. Finally, we give an overview on the design and the results of a recent shared task on measuring document comparability across languages.

Continuous-time identification of periodically parameter-varying state space models

This paper presents a new frequency domain identification technique to estimate multivariate Linear Parameter-Varying (LPV) continuous-time state space models, where a periodic variation of the parameters is assumed or imposed. The main goal is to obtain an LPV state space model suitable for control, from a single parameter-varying experiment. Although most LPV controller synthesis tools require continuous time state space models, the identification of such models is new. The proposed identification method designs a periodic input signal, taking the periodicity of the parameter variation into account. We show that when an integer number of periods is observed for both the input and the scheduling, the state space model representation has a specific, sparse structure in the frequency domain, which is exploited to speed up the estimation procedure. A weighted non-linear least squares algorithm then minimizes the output error. Two initialization methods are explored to generate starting values. The first approach uses a Linear Time-Invariant (LTI) approximation. The second estimates a Linear Time-Variant (LTV) input–output differential equation, from which a corresponding state space realization is computed.

Anderson localization in optical lattices with correlated disorder

We study the Anderson localization of atomic gases exposed to simple-cubic optical lattices with a superimposed disordered speckle pattern. The two mobility edges in the first band and the corresponding critical filling factors are determined as a function of the disorder strength, ranging from vanishing disorder up to the critical disorder intensity where the two mobility edges merge and the whole band becomes localized. Our theoretical analysis is based both on continuous-space models which take into account the details of the spatial correlation of the speckle pattern, and also on a simplified tight-binding model with an uncorrelated distribution of the on-site energies. The mobility edges are computed via the analysis of the energy-level statistics, and we determine the universal value of the ratio between consecutive level spacings at the mobility edge. We analyze the role of the spatial correlation of the disorder, and we also discuss a qualitative comparison with available experimental data for interacting atomic Fermi gases measured in the moderate interaction regime.

Exponential Language Modeling Using Morphological Features and Multi-Task Learning

For languages with fast vocabulary growth and limited resources, data sparsity leads to challenges in training a language model. One strategy for addressing this problem is to leverage morphological structure as features in the model. This paper explores different uses of unsupervised morphological features in both the history and prediction space for three word-based exponential models (maximum entropy, logbilinear, and recurrent neural net (RNN)). Multi-task training is introduced as a regularizing mechanism to improve performance in the continuous-space approaches. The models are compared to non-parametric baselines. From using the RNN with morphological features and multi-task learning, experiments with conversational speech from four languages show we can obtain consistent gains of 7–11% in perplexity reduction in a limited-resource scenario (10 hrs speech), and 12–18% when the training size is increased ( 80 hrs ). Results are mixed for all other approaches, compared to a modified Kneser-Ney baseline, but morphology is useful in continuous-space models compared to their word-only baseline. Multi-task learning improves both continuous-space models.

Introduction to the Special Section on Continuous Space and Related Methods in Natural Language Processing

The articles in this special section discuss some latest findings on research problems related to the application of continuous space and related models in Natural Language Processing (NLP).

Adequacy–Fluency Metrics: Evaluating MT in the Continuous Space Model Framework

This work extends and evaluates a two-dimensional automatic evaluation metric for machine translation, which is designed to operate at the sentence level. The metric is based on the concepts of adequacy and fluency, aiming at decoupling both semantic and syntactic components of the translation process to provide a more balanced view on translation quality. These two elements are independently evaluated by using continuous space and <formula formulatype="inline" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex Notation="TeX">$n$</tex> </formula> -gram language modeling frameworks, respectively. Two different implementations are evaluated: a monolingual version that fully operates on the target language side, and a cross-language version that has the main advantage of not requiring reference translations. Both implementations are evaluated by comparing their performance with state-of-the-art automatic metrics over a dataset involving five different European languages.

Reasoning with Vectors: A Continuous Model for Fast Robust Inference.

This paper describes the use of continuous vector space models for reasoning with a formal knowledge base. The practical significance of these models is that they support fast, approximate but robust inference and hypothesis generation, which is complementary to the slow, exact, but sometimes brittle behavior of more traditional deduction engines such as theorem provers. The paper explains the way logical connectives can be used in semantic vector models, and summarizes the development of Predication-based Semantic Indexing, which involves the use of Vector Symbolic Architectures to represent the concepts and relationships from a knowledge base of subject-predicate-object triples. Experiments show that the use of continuous models for formal reasoning is not only possible, but already demonstrably effective for some recognized informatics tasks, and showing promise in other traditional problem areas. Examples described in this paper include: predicting new uses for existing drugs in biomedical informatics; removing unwanted meanings from search results in information retrieval and concept navigation; type-inference from attributes; comparing words based on their orthography; and representing tabular data, including modelling numerical values. The algorithms and techniques described in this paper are all publicly released and freely available in the Semantic Vectors open-source software package.

Continuous model in dq frame of Thyristor Controlled Reactors for stability analysis of high power electrical systems

This paper illustrates a formulation of a continuous state space model in dq frame of the Thyristor Controlled Reactor (TCR), which can be used for stability studies by eigenvalue analysis. Differently from the previous approaches, this model is just addressed to reproduce the operation of the non-linear part of the TCR system, without considering shunted capacitor or control system, thus obtaining a characterization of the TCR independent from the application. A typical problem of the continuous models in dq frame of the TCR system is the presence of an un-damped homogenous current component in the solution, which is inconsistent with the real operation; the proposed model solves this problem improving the accuracy. The model performance has been also evaluated in the frequency domain comparing the obtained results with a numerical simulation of the TCR implemented by PSIM program.

Continuous space maximal coverage: Insights, advances and challenges

Siting facilities in continuous space to maximize coverage of regional demand is an important planning problem. It is challenging to solve because demand is continuously distributed in a region and facilities may be sited anywhere in the region, suggesting that an infinite number of locations must be considered both as demand for service as well as a potential facility. An ingenious approach for solving continuous space coverage problems is abstracting continuously distributed demand as discrete spatial objects (points, lines or polygons), generating a finite dominating set from continuous space as candidate facility sites, and applying a discrete location coverage model. However, abstracting continuous demand and potential facility sites often introduces errors and uncertainties in coverage of regional demand, rendering findings (service and optimal facility locations) less than definitive. In this paper we examine the continuous space maximal coverage problem. Studies focused on locating antennas in Paradise Valley, Arizona and emergency warning sirens in Dublin, Ohio, are used to highlight issues and challenges in continuous space modeling. Potential paths forward are suggested based on empirical insights.

A continuous phenotype space model of RNA virus evolution within a host

Due to their very high replication and mutation rates, RNA viruses can serve as an excellent testing model for verifying hypothesis and addressing questions in evolutionary biology. In this paper, we suggest a simple deterministic mathematical model of the within-host viral dynamics, where a possibility for random mutations incorporates. This model assumes a continuous distribution of viral strains in a one-dimensional phenotype space where random mutations are modelled by Brownian motion (that is, by diffusion). Numerical simulations show that random mutations combined with competition for a resource result in evolution towards higher Darwinian fitness: a stable pulse traveling wave of evolution, moving towards higher levels of fitness, is formed in the phenotype space. The advantage of this model, compared with the previously constructed models, is that this model is mechanistic and is based on commonly accepted model of virus dynamics within a host, and thus it allows an incorporation of features of the real-life host-virus system such as immune response, antiviral therapy, etc.

Racing under uncertainty: Boundary value problem approach

This paper explores the continuous time and continuous space model of racing under uncertainty put forward by Budd, Harris, and Vickers [4] and allows for potentially asymmetric players. To prove the existence of Markov perfect equilibria, I use a boundary value problem formulation which is novel to the dynamic competition literature. In addition, by providing a new and intuitive definition of the pivot of an equilibrium, I show that equilibrium strategies exhibit the discouragement effect similar to that of Harris and Vickers [10] but under a more general class of the cost functions.

K-Capture in multiagent pursuit evasion, or the lion and the hyenas

We consider the following generalization of the classical pursuit–evasion problem, which we call k-capture. A group of n pursuers (hyenas) wish to capture an evader (lion) who is free to move in an m-dimensional Euclidean space, the pursuers and the evader can move with the same maximum speed, and at least k pursuers must simultaneously reach the evaderʼs location to capture it. If fewer than k pursuers reach the evader, then those pursuers get destroyed by the evader. Under what conditions can the evader be k-captured? We study this problem in the discrete time, continuous space model and prove that k-capture is possible if and only if there exists a time when the evader lies in the interior of the pursuersʼ k-Hull. When the pursuit occurs inside a compact, convex subset of the Euclidean space, we show through an easy constructive strategy that k-capture is always possible.

Design of a Novel Fuzzy PI Controller Based on Artificial Bee Colony Algorithm for VSC-HVDC Transmission Link

This paper presents a continuous time state space model for a VSC-HVDC transmission link in the d-q reference frame. An Artificial Bee Colony (ABC) algorithm based Fuzzy proportional plus integral (FPI) controller is proposed for a decoupled back to back VSC-HVDC transmission link single machine power system. Here, the ABC approach is used for selecting the nonlinear variable gain parameters of scaling factors of Fuzzy controller to enhance its performance. Simulation result has been presented to show and validate the efficiency of new approach for damping oscillations over a wide range of operating conditions and compared with the conventional PI controller tuned by Ziggler-Nichol technique, and Fuzzy PI controller without optimization. It has been found that the proposed controller is quite effective in enhancing the power system dynamic stability significantly.

Difference in modes of firing rate modulation between cortical areas

Neurons in vivo exhibit irregular firing, and some aspect of their firing pattern such as the degree of firing irregularity does not fluctuate greatly in time for individual neurons, while it differs among neurons [1]. By analyzing spike trains recorded from a variety of cortical areas, it has been revealed that neurons in motor-related cortical areas fire regularly while neurons in visual and prefrontal cortex fire randomly [2]. Here, the firing irregularity is defined as a variability of consecutive interspike intervals, independent of the firing rate. In those studies, the firing rate has been ignored, because it may fluctuate greatly in time depending on external stimuli or behavioral contexts of an animal. Though the neuronal firing rate may fluctuate with external stimuli, this does not necessarily mean that the rate is extrinsic and perfectly controllable; there could be some uncontrollable intrinsic aspect in the firing rate fluctuation. In this contribution, we go contrary to the above-mentioned studies, and attempt to characterize neuronal spike trains solely in terms of the firing rate, ignoring the firing irregularity. The key feature of neuronal spike trains we now pay attention to is dynamic mode of rate fluctuation, such that the rate either fluctuates smoothly in time or changes abruptly between several stationary states. We compare two different models of a fluctuating rate process for their goodness-of-fit with respect to spike trains recorded from single neurons in several cortical areas of rats. The first model is the state space model constructed under the Empirical Bayes method [3,4]. The generative inhomogeneous Poisson process is defined as the conditional distribution of the spike times ti≡t1,t2,...,tn given the rateλ(t), ptiλ(t)= ∏i=1nλ(ti)exp-∫ 0Tλ(t)dt. We give the prior distribution of λ(t) by incorporating the tendency of the rate to be flat, pλ(t)∝exp-β∫ 0Tdλ/dt2dt. The hyperparameter β is selected by the principle of maximizing the marginal likelihood, and then we obtain the maximum a posteriori (MAP) estimate for the rate λ⌢(t). The second alternative model is the Hidden Markov model (HMM) whose hidden states represent different rate states [5-7]. The number of states is selected with a variation Bayes method. In this way, two different estimations of the firing rate are obtained using the continuous state space model and the discrete HMM. We compare them for each spike train obtained from a neuron, and examine whether the continuous or discrete representation of the state transition is more plausible. Then, we examine how the different preferences are distributed in different cortical areas.

Numerical simulation of a susceptible–exposed–infectious space-continuous model for the spread of rabies in raccoons across a realistic landscape

We introduce a numerical model for the spread of a lethal infectious disease in wildlife. The reference model is a Susceptible–Exposed–Infectious system where the spatial component of the dynamics is modelled by a diffusion process. The goal is to develop a model to be used for real geographical scenarios, so we do not rely upon simplifying assumptions on the shape of the region of interest. For this reason, space discretization is carried out with the finite element method on an unstructured triangulation. A diffusion term is designed to take into account landscape heterogeneities such as mountains and waterways. Numerical simulations are carried out for rabies epidemics among raccoons in New York state. A qualitative comparison of numerical results to available data from real-world epidemics is discussed.