Incomplete Observations Research Articles

A mathematical model and inference method for bacterial colonization in hospital units applied to active surveillance data for carbapenem-resistant enterobacteriaceae.

Widespread use of antibiotics has resulted in an increase in antimicrobial-resistant microorganisms. Although not all bacterial contact results in infection, patients can become asymptomatically colonized, increasing the risk of infection and pathogen transmission. Consequently, many institutions have begun active surveillance, but in non-research settings, the resulting data are often incomplete and may include non-random testing, making conventional epidemiological analysis problematic. We describe a mathematical model and inference method for in-hospital bacterial colonization and transmission of carbapenem-resistant Enterobacteriaceae that is tailored for analysis of active surveillance data with incomplete observations. The model and inference method make use of the full detailed state of the hospital unit, which takes into account the colonization status of each individual in the unit and not only the number of colonized patients at any given time. The inference method computes the exact likelihood of all possible histories consistent with partial observations (despite the exponential increase in possible states that can make likelihood calculation intractable for large hospital units), includes techniques to improve computational efficiency, is tested by computer simulation, and is applied to active surveillance data from a 13-bed rehabilitation unit in New York City. The inference method for exact likelihood calculation is applicable to other Markov models incorporating incomplete observations. The parameters that we identify are the patient–patient transmission rate, pre-existing colonization probability, and prior-to-new-patient transmission probability. Besides identifying the parameters, we predict the effects on the total prevalence (0.07 of the total colonized patient-days) of changing the parameters and estimate the increase in total prevalence attributable to patient–patient transmission (0.02) above the baseline pre-existing colonization (0.05). Simulations with a colonized versus uncolonized long-stay patient had 44% higher total prevalence, suggesting that the long-stay patient may have been a reservoir of transmission. High-priority interventions may include isolation of incoming colonized patients and repeated screening of long-stay patients.

Economic Analysis of a Power System Network using Optimal Placement of PMUs and DULRs Through Complete and Incomplete Observability Analysis

For power system operation and control State estimation has become essential in recent days. To evaluate this State estimation, to reach out and recorder system status Supervisory Control and Data Acquisition and Energy Management System are intended due to which are operated at specified time. Phasor Measurement Unit is one of the crucial techniques for auditing a system. To make system performance is honest, keeping Phasor Measurement Units in a system is very much helpful, and that will be intended to help operating time reduced by very much. Due to this another problem may arise, that is substation updating cost may increase due to these Phasor Measurement Units. In this paper an optimal set of Phasor Measurement Units along with optimal locations Dual Use Line Relays are achieved for complete economic analysis of a power network using both complete and incomplete observability analysis. No of Phasor Measurement Units required to attain the complete perceptibility of a network are considerably diminished using incomplete observability analysis. To test IEEE 14, 24, 30, 57 and both SR and sub systems 1, 2 with a size of 270 and 444 buses, the recommended complete and incomplete observability analysis are used to place Phasor Measurement Units. Similarly IEEE 14, 30 and 57 bus systems are considered for both Phasor Measurement Units and Dual Use Line Relays placement. Also single Phasor Measurement Unit loss condition is considered by using complete observability analysis.

Fast stable parameter estimation for linear dynamical systems

Dynamical systems describe changes in processes that arise naturally from their underlying physical principles, such as the laws of motion or the conservation of mass, energy or momentum. These models facilitate a causal explanation for the drivers and impediments of the processes. Extracting these governing equations from data is a central challenge in many diverse areas of science and engineering. A methodology for estimating the solution; and the parameters of linear dynamical systems from incomplete and noisy observations of the processes is introduced. Building on the parameter cascading approach, where a linear combination of basis functions approximates the implicitly defined solution of the dynamical system. The systems’ parameters are estimated so that this approximating solution adheres to the data. By taking advantage of the linearity of the system, the parameter cascading estimation procedure is simplified, and by developing a new iterative scheme, fast and stable computation is achieved. An illustrative example obtains a linear differential equation that represents real data from biomechanics. A comparison of the proposed approach with popular methods for estimating the parameters of linear dynamical systems, namely, the non-linear least-squares approach, simulated annealing, parameter cascading and smooth functional tempering reveals a considerable reduction in computation and an improved bias and sampling variance.

Convective rear-flank downdraft as driver for meteotsunami along English Channel and North Sea coasts 28–29 May 2017

We examine the physical processes that led to the meteotsunami observed along the English Channel and North Sea coasts on 29 May 2017. It was most notably reported along the Dutch coast, but also observed on tide gauges from the Channel Islands to the coast of Germany, and also those in eastern England. From an assessment of multiple observations, including rain radar, LIDAR, satellite, surface observations and radiosonde reports we conclude that the event was driven by a rear flank downdraft in association with a mesoscale convective system (MCS). This downdraft, from a medium level or elevated MCS, led to a hydrostatically forced internal or ducted gravity wave below the MCS. The gravity wave was manifested by a marked rise and fall in pressure, a meso-high, which then interacted with the sea surface through Proudman resonance causing a measured wave of close to 0.9 m in amplitude, and an estimated wave run-up on Dutch beaches of 2 m. Through examination of existing research, we show that the basic assumptions here relating to the formation of the Dutch meteotsunami are consistent with previously described physical processes, and confirm the correlation between the speed of the ocean wave and medium level steering winds. This raises the possibility that high-resolution, coupled, weather-ocean numerical weather prediction (NWP) models can be utilised to predict future events. However, deterministic high-resolution NWP models still struggle with modelling convective systems with sufficient precision because of the chaotic nature of the atmosphere and incomplete observations. A way forward is proposed here to improve forecasting through post-processing of NWP model output by overlaying medium level wind fields with ocean bathymetry.

Uncovering ecological state dynamics with hidden Markov models.

Ecological systems can often be characterised by changes among a finite set of underlying states pertaining to individuals, populations, communities or entire ecosystems through time. Owing to the inherent difficulty of empirical field studies, ecological state dynamics operating at any level of this hierarchy can often be unobservable or ‘hidden’. Ecologists must therefore often contend with incomplete or indirect observations that are somehow related to these underlying processes. By formally disentangling state and observation processes based on simple yet powerful mathematical properties that can be used to describe many ecological phenomena, hidden Markov models (HMMs) can facilitate inferences about complex system state dynamics that might otherwise be intractable. However, HMMs have only recently begun to gain traction within the broader ecological community. We provide a gentle introduction to HMMs, establish some common terminology, review the immense scope of HMMs for applied ecological research and provide a tutorial on implementation and interpretation. By illustrating how practitioners can use a simple conceptual template to customise HMMs for their specific systems of interest, revealing methodological links between existing applications, and highlighting some practical considerations and limitations of these approaches, our goal is to help establish HMMs as a fundamental inferential tool for ecologists.

Towards Dynamic Uncertain Causality Graphs for the Intelligent Diagnosis and Treatment of Hepatitis B

Hepatitis B is a widespread epidemic in the world, but so far no single drug has been shown to kill or eliminate the Hepatitis B virus and heal people with chronic Hepatitis B virus infection. Based on comprehensive investigations to relevant characteristics of Hepatitis B, a diagnostic modelling and reasoning methodology using Dynamic Uncertain Causality Graph is proposed. The symptoms, physical signs, examinations results, medical histories, etiology, pathogenesis and other factors were included in the diagnosis model. In order to reduce the difficulty of building the model, a modular modeling scheme is proposed, which provides multi-perspectives and arbitrary granularity for the expression of disease causality. The chain reasoning algorithm and weighted logic operation mechanism are introduced to ensure the correctness and effectiveness of diagnostic reasoning under incomplete and uncertain information. In addition, the causal view of the potential interactions between diseases and symptoms visually shows the reasoning process in a graphical way. In the relevant model, the model of the diagnostic process and the model of the therapeutic process are symmetrical. The results show that, even with incomplete observations, the proposed methodology achieves encouraging diagnostic accuracy and effectiveness, providing a promising assistance tool for physicians in the diagnosis of Hepatitis B.

System Identification and LQR Controller Design with Incomplete State Observation for Aircraft Trajectory Tracking

This paper presents a controller design process for an aircraft tracking problem when not all states are available. In the study, a nonlinear-transport aircraft simulation model was used and identified through Maximum Likelihood Principle and Extended Kalman Filter. The obtained mathematical model was used to design a Linear–Quadratic Regulator (LQR) with optimal weighting matrices when not all states are measured. The nonlinear aircraft simulation model with LQR controller tracking abilities were analyzed for multiple experiments with various noise levels. It was shown that the designed controller is robust and allows for accurate trajectory tracking. It was found that, in ideal atmospheric conditions, the tracking errors are small, even for unmeasured variables. In wind presence, the tracking errors were proportional to the wind velocity and acceptable for small and moderate disturbances. When turbulence was present in the experiment, state variable oscillations occurred that were proportional to the turbulence intensity and acceptable for small and moderate disturbances.

Learning Elastic Constitutive Material and Damping Models

AbstractCommonly used linear and nonlinear constitutive material models in deformation simulation contain many simplifications and only cover a tiny part of possible material behavior. In this work we propose a framework for learning customized models of deformable materials from example surface trajectories. The key idea is to iteratively improve a correction to a nominal model of the elastic and damping properties of the object, which allows new forward simulations with the learned correction to more accurately predict the behavior of a given soft object. Space‐time optimization is employed to identify gentle control forces with which we extract necessary data for model inference and to finally encapsulate the material correction into a compact parametric form. Furthermore, a patch based position constraint is proposed to tackle the challenge of handling incomplete and noisy observations arising in real‐world examples. We demonstrate the effectiveness of our method with a set of synthetic examples, as well with data captured from real world homogeneous elastic objects.

Bridging Data Science and Dynamical Systems Theory

This short review describes mathematical techniques for statistical analysis and prediction in dynamical systems. Two problems are discussed, namely (i) the supervised learning problem of forecasting the time evolution of an observable under potentially incomplete observations at forecast initialization; and (ii) the unsupervised learning problem of identification of observables of the system with a coherent dynamical evolution. We discuss how ideas from from operator-theoretic ergodic theory combined with statistical learning theory provide an effective route to address these problems, leading to methods well-adapted to handle nonlinear dynamics, with convergence guarantees as the amount of training data increases.

A simulation study of approaches for handling disease progression in dose-finding clinical trials

ABSTRACT In traditional dose-finding studies, dose-limiting toxicity (DLT) is determined within a fixed time observation window where DLT is often defined as a binary outcome. In the setting of oncology dose-finding trials, often patients in advanced stage of diseases are enrolled. Therefore, disease progression may occur within the DLT observation window leading to treatment discontinuation and rendering the patient unevaluable for DLT assessment. As a result, additional patients have to be enrolled, increasing the sample size. We propose and compare several practical approaches for handling disease progression which occurs within the DLT observation window, while in the framework of the time-to-event continual reassessment method (TITE-CRM) which allows using partial observations. The approaches differ on the way they define an evaluable patient and in the way incomplete observations are included. The practical approaches, which we call strategies A, B and C, are illustrated and contrasted in the context of a single simulated trial, and compared via simulations under various scenarios of dose-progression relationship, in the setting of advanced soft-tissue sarcoma.

Calculation Method of Incubation Period of Infectious Diseases and Its Epidemiological Significance

Based on the practical application, this paper introduced the basic calculation conditions, methods and epidemiological significance of incubation period. The real data were used for calculations of the incubation period by lognormal, gamma, Weibull and Erlang distribution methods. Both of the complete and incomplete observation data were demonstrated.

Reproducibility challenges for biomarker detection with uncertain but informative experimental data.

Recent studies have revealed that circulating microRNAs are promising biomarkers for detecting toxicity or disease. Quantitative real-time polymerase chain reaction (qPCR) is often used to measure the levels of microRNAs. Besides complete and certain data, investigators inevitably have observed technically incomplete or uncertain qPCR data. Investigators usually set incomplete observations equal to the maximum quality number of qPCR cycles, apply the complete-observation method, or choose not to analyze targets with incomplete observations. Using biostatistical knowledge and published studies, we show that three commonly applied methods tend to cause biased inference and decrease reproducibility in biomarker detection. More efforts are needed to address the challenges to identify and detect reliable, novel circulating biomarkers in liquid biopsies.

A Unified Tensor Framework for Clustering and Simultaneous Reconstruction of Incomplete Imaging Data

Incomplete observations in the data are always troublesome to data clustering algorithms. In fact, most of the well-received techniques are not designed to encounter such imperative scenarios. Hence, clustering of images under incomplete samples is an inquisitive yet unaddressed area of research. Therefore, the aim of this article is to design a single-stage optimization procedure for clustering as well as simultaneous reconstruction of images without breaking the intrinsic spatial structure. The method employs the self-expressiveness property of submodules, and images are stacked as the lateral slices of a three-dimensional tensor. The proposed optimization method is designed to extract a sparse t -linear combination tensor with low multirank constraint, consisting of a unique set of linear coefficients in the form of mode-3 fibers and the spectral clustering is performed on these fibers. Simultaneously, the recovery of lost samples is accomplished by twisting the entire lateral slices of the data tensor and applying a low-rank approximation on each slice. The prominence of the proposed method lies in the simultaneous execution of data clustering and reconstruction of incomplete observations in a single step. Experimental results reveal the excellence of the proposed method over state-of-the-art clustering algorithms in the context of incomplete imaging data.

Multi-sensor prognostics modeling for applications with highly incomplete signals

Multi-stream degradation signals have been widely used to predict the residual useful lifetime of partially degraded systems. To achieve this goal, most of the existing prognostics models assume that degradation signals are complete, i.e., they are observed continuously and frequently at regular time grids. In reality, however, degradation signals are often (highly) incomplete, i.e., containing missing and corrupt observations. Such signal incompleteness poses a significant challenge for the parameter estimation of prognostics models. To address this challenge, this article proposes a prognostics methodology that is capable of using highly incomplete multi-stream degradation signals to predict the residual useful lifetime of partially degraded systems. The method first employs multivariate functional principal components analysis to fuse multi-stream signals. Next, the fused features are regressed against time-to-failure using (log)-location-scale regression. To estimate the fused features using incomplete multi-stream degradation signals, we develop two computationally efficient algorithms: subspace detection and signal recovery. The performance of the proposed prognostics methodology is evaluated using simulated datasets and a degradation dataset of aircraft turbofan engines from the NASA repository.

CHANGE DETECTION AND DEFORMATION ANALYSIS BASED ON MOBILE LASER SCANNING DATA OF URBAN AREAS

Abstract. Change detection is an important tool for processing multiple epochs of mobile LiDAR data in an efficient manner, since it allows to cope with an otherwise time-consuming operation by focusing on regions of interest. State-of-the-art approaches usually either do not handle the case of incomplete observations or are computationally expensive. We present a novel method based on a combination of point clouds and voxels that is able to handle said case, thereby being computationally less expensive than comparable approaches. Furthermore, our method is able to identify special classes of changes such as partially moved, fully moved and deformed objects in addition to the appeared and disappeared objects recognized by conventional approaches. The performance of our method is evaluated using the publicly available TUM City Campus datasets, showing an overall accuracy of 88 %.

Comparison of partial least square algorithms in hierarchical latent variable model with missing data

Missing data is almost inevitable for various reasons in many applications. For hierarchical latent variable models, there usually exist two kinds of missing data problems. One is manifest variables with incomplete observations, the other is latent variables which cannot be observed directly. Missing data in manifest variables can be handled by different methods. For latent variables, there exist several kinds of partial least square (PLS) algorithms which have been widely used to estimate the value of latent variables. In this paper, we not only combine traditional linear regression type PLS algorithms with missing data handling methods, but also introduce quantile regression to improve the performances of PLS algorithms when the relationships among manifest and latent variables are not fixed according to the explored quantile of interest. Thus, we can get the overall view of variables’ relationships at different levels. The main challenges lie in how to introduce quantile regression in PLS algorithms correctly and how well the PLS algorithms perform when missing manifest variables occur. By simulation studies, we compare all the PLS algorithms with missing data handling methods in different settings, and finally build a business sophistication hierarchical latent variable model based on real data.

Nonlinear sparse Bayesian learning for physics-based models

This paper addresses the issue of overfitting while calibrating unknown parameters of over-parameterized physics-based models with noisy and incomplete observations. A semi-analytical Bayesian framework of nonlinear sparse Bayesian learning (NSBL) is proposed to identify sparsity among model parameters during Bayesian inversion. NSBL offers significant advantages over machine learning algorithm of sparse Bayesian learning (SBL) for physics-based models, such as 1) the likelihood function or the posterior parameter distribution is not required to be Gaussian, and 2) prior parameter knowledge is incorporated into sparse learning (i.e. not all parameters are treated as questionable). NSBL employs the concept of automatic relevance determination (ARD) to facilitate sparsity among questionable parameters through parameterized prior distributions. The analytical tractability of NSBL is enabled by employing Gaussian ARD priors and by building a Gaussian mixture-model approximation of the posterior parameter distribution that excludes the contribution of ARD priors. Subsequently, type-II maximum likelihood is executed using Newton's method whereby the evidence and its gradient and Hessian information are computed in a semi-analytical fashion. We show numerically and analytically that SBL is a special case of NSBL for linear regression models. Subsequently, a linear regression example involving multimodality in both parameter posterior pdf and model evidence is considered to demonstrate the performance of NSBL in cases where SBL is inapplicable. Next, NSBL is applied to identify sparsity among the damping coefficients of a mass-spring-damper model of a shear building frame. These numerical studies demonstrate the robustness and efficiency of NSBL in alleviating overfitting during Bayesian inversion of nonlinear physics-based models.

Linear censored regression models with skew scale mixtures of normal distributions

A special source of difficulty in the statistical analysis is the possibility that some subjects may not have a complete observation of the response variable. Such incomplete observation of the response variable is called censoring. Censorship can occur for a variety of reasons, including limitations of measurement equipment, design of the experiment, and non-occurrence of the event of interest until the end of the study. In the presence of censoring, the dependence of the response variable on the explanatory variables can be explored through regression analysis. In this paper, we propose to examine the censorship problem in context of the class of asymmetric, i.e., we have proposed a linear regression model with censored responses based on skew scale mixtures of normal distributions. We develop a Monte Carlo EM (MCEM) algorithm to perform maximum likelihood inference of the parameters in the proposed linear censored regression models with skew scale mixtures of normal distributions. The MCEM algorithm has been discussed with an emphasis on the skew-normal, skew Student-t-normal, skew-slash and skew-contaminated normal distributions. To examine the performance of the proposed method, we present some simulation studies and analyze a real dataset.

Individual reserving and nonparametric estimation of claim amounts subject to large reporting delays

Thanks to nonparametric estimators coming from machine learning, microlevel reserving has become more and more popular for actuaries. Recent research focused on how to integrate the whole information one can have on claims to predict individual reserves, with varying success due to incomplete observations. Using the CART algorithm, we develop new results that allow us to deal with large reporting delays and partially observed explanatory variables. Statistically speaking, we extend CART to take into account truncation of the data and introduce plug-in estimators. Our applications are based on real-life insurance portfolios embedding Income Protection and Third-Party Liability guarantees. The full knowledge of the claim lifetime is shown to be crucial to predict the individual reserves efficiently.

Ajuste de modelos de fragilidade e riscos proporcionais aplicados a dados de retinopatia diabética

Atualmente a análise de sobrevivência é uma das áreas que mais crescem no campo da análise estatística, com uma sólida teoria para ajustar modelos de regressão para estudar certos fenômenos, os quais têm, em sua estrutura, a característica de ter observações incompletas na amostra denominada censura. Embora esses modelos possam representar eficientemente o fenômeno em estudo em muitas situações, alguns deles não levam em consideração a existência de uma variável não observável presente na maioria dos estudos, denominada fragilidade. Essa fragilidade denota a suscetibilidade do evento a ocorrer por um indivíduo ou objeto determinado sob investigação. O objetivo deste trabalho foi mostrar que, em situações em que a fragilidade está presente, o uso de modelos que capturam a variabilidade dessa variável é mais viável para a análise desses dados quando comparado aos modelos convencionais em estudos de sobrevivência. Para tanto, foi realizada uma análise comparativa entre esses modelos, ajustada para um conjunto de dados de pacientes diagnosticados com retinopatia diabética, e também foi realizado um estudo de simulação para o modelo de fragilidade gama com diferentes porcentagens de censura e heterogeneidade. Após o ajuste dos modelos, observa-se que os modelos de fragilidade tiveram melhor desempenho quando comparados ao modelo de Cox, com ênfase no modelo de fragilidade gama, que gerou o menor valor para AIC e BIC. O estudo de simulação mostrou que altas taxas de censura prejudicam o grau de previsibilidade do modelo de fragilidade e que altas taxas de heterogeneidade contribuem para estimativas de parâmetros.