Expectation Maximization Approach Research Articles

Maximum Likelihood Estimation for a Progressively Type II Censored Generalized Inverted Exponential Distribution via EM Algorithm

The Generalized Inverted Exponential (GIE) distribution is a mixed lifetime model used in a number of fields such as queuing theory, testing of products or components and modelling the speed of winds. The study aims to focus on the determination of maximum likelihood estimates of GIE distribution when the test units are progressively (type II) censored. The scheme permits the withdrawal of units from the life test at stages during failure. This may be due to cost and time constraints. Both Expectation- Maximization (EM) and Newton-Raphson (NR) methods have been used to obtain the maximum likelihood estimates of the GIE parameters. Also, the variance-covariance matrix of the obtained estimators has been derived. The performance of the obtained MLEs via EM method is compared with those obtained using NR method in terms of bias and root mean squared errors and confidence interval widths for different progressive type II censoring schemes at fixed parameter values of λ and θ Simulation results reveal that estimates obtained via EM approach are more robust compared to those obtained via NR algorithm. It's also noted that the bias, root mean squared errors and confidence interval widths decrease with an increase in the sample size for a fixed number of failures. A similar trend in results is observed with increase in number of failures for a fixed sample size. The results of the obtained estimators are finally illustrated on two real data sets.

Unsupervised sorting of retinal vessels using locally consistent Gaussian mixtures

Background and Objectives: Retinal blood vessels classification into arterioles and venules is a major task for biomarker identification. Especially, clustering of retinal blood vessels is a challenging task due to factors affecting the images such as contrast variability, non-uniform illumination etc. Hence, a high performance automatic retinal vessel classification system is highly prized. In this paper, we propose a novel unsupervised methodology to classify retinal vessels extracted from fundus camera images into arterioles and venules.Methods: The proposed method utilises the homomorphic filtering (HF) to preprocess the input image for non-uniform illumination and denoising. In the next step, an unsupervised multiscale line operator segmentation technique is used to segment the retinal vasculature before extracting the discriminating features. Finally, the Locally Consistent Gaussian Mixture Model (LCGMM) is utilised for unsupervised sorting of retinal vessels.Results: The performance of the proposed unsupervised method was assessed using three publicly accessible databases: INSPIRE-AVR, VICAVR, and MESSIDOR. The proposed framework achieved 90.14%,90.3% and 93.8% classification rate in zone B for the three datasets respectively.Conclusions: The proposed clustering framework provided high classification rate as compared to conventional Gaussian mixture model using Expectation-Maximisation (GMM-EM) approach, thus have a great capability to enhance computer assisted diagnosis and research in field of biomarker discovery.

Predictive Deployment of UAV Base Stations in Wireless Networks: Machine Learning Meets Contract Theory

In this paper, a novel framework is proposed to enable a predictive deployment of unmanned aerial vehicles (UAVs) as temporary base stations (BSs) to complement ground cellular systems in face of downlink traffic overload. First, a novel learning approach, based on the weighted expectation maximization (WEM) algorithm, is proposed to estimate the user distribution and the downlink traffic demand. Next, to guarantee a truthful information exchange between the BS and UAVs, using the framework of contract theory, an offload contract is developed, and the sufficient and necessary conditions for having a feasible contract are analytically derived. Subsequently, an optimization problem is formulated to deploy an optimal UAV onto the hotspot area in a way that the utility of the overloaded BS is maximized. Simulation results show that the proposed WEM approach yields a prediction error of around 10%. Compared with the expectation maximization and k-mean approaches, the WEM method shows a significant advantage on the prediction accuracy, as the traffic load in the cellular system becomes spatially uneven. Furthermore, compared with two event-driven deployment schemes based on the closest-distance and maximal-energy metrics, the proposed predictive approach enables UAV operators to provide efficient communication service for hotspot users in terms of the downlink capacity, energy consumption and service delay. Simulation results also show that the proposed method significantly improves the revenues of both the BS and UAV networks, compared with two baseline schemes.

Smart tracking algorithm for multi‐static sonar based on expectation maximisation

This study proposes an intelligent tracking method based on the expectation maximisation approach. To obtain the consistent tracking performance, the observer should maintain contact with the target and continuously update its tracking data. Unfortunately, a significant problem arises from the uncertainty of the target motion. The reasons behind this uncertainty include losses, time-varying noises, external inputs, and others. Above all, a sharp manoeuvre by the arbitrary acceleration input mainly degrades the tracking performance. A variety of techniques were needed and studied to enhance performance levels. This study focuses on acceleration as the cause of the uncertainty and a consistent approximation of the acceleration, leading to highly efficient tracking performance. The EM makes it possible to separate the noise term from the acceleration inputs and the noise components. Finally, a manoeuvring submarine with three-dimensional coordinates is provided as an example to show the effectiveness of the proposed algorithm.

Collaborative Semantic Understanding and Mapping Framework for Autonomous Systems

Performing collaborative semantic mapping is a critical challenge for cooperative robots to enhance their comprehensive contextual understanding of the surroundings. This article bridges the gap between the advances in collaborative geometry mapping that relies on pure geometry information fusion, and single robot semantic mapping that focuses on integrating continuous raw sensor data. In this article, a novel hierarchical collaborative probabilistic semantic mapping framework is proposed, where the problem is formulated in a distributed setting. The key novelty of this work is the modeling of the hierarchical semantic map fusion framework and its mathematical derivation of its probability decomposition. At the single robot level, the semantic point cloud is obtained by combining information from heterogeneous sensors and used to generate local semantic maps. At the collaborative robots level, local maps are shared among robots for global semantic map fusion. Since the voxel correspondence is unknown between local maps, an expectation-maximization approach is proposed to estimate the hidden data association. Then, Bayesian rule is applied to perform semantic and occupancy probability update. Experimental results on the unmanned aerial vehicle and the unmanned ground vehicle platforms show the high quality of global semantic maps, demonstrating the accuracy and utility in practical missions.

EM algorithms for ordered and censored system lifetime data under a proportional hazard rate model

In this paper, we consider maximum likelihood estimation of the proportional parameter in a proportional hazard rate (PHR) model based on single and multiply censored order statistics, and progressively Type-II censored order statistics from lifetimes that follow the PHR model. The expectation-maximization (EM) algorithm is proposed for computing the maximum likelihood estimate (MLE) by utilizing the relationship between order statistics and lifetimes of components in a system. The existence and uniqueness of the MLE under different data structures are discussed. We also propose a reliable initial value for the iterative algorithm based on approximating the likelihood function using Taylor series expansion. Monte Carlo simulation studies are used to study the performance of the proposed EM algorithms and the proposed initial value. The proposed EM approach is shown to be a good alternative to the Newton-Raphson method in terms of computational efficient and robustness to the initial value. We also show that the proposed initial value works well with the EM algorithm in obtaining the MLE.

A New Supervised Clustering Framework Using Multi Discriminative Parts and Expectation–Maximization Approach for a Fine-Grained Animal Breed Classification (SC-MPEM)

Fine-grained image classification is active research in the field of computer vision. Specifically, animal breed classification is an arduous task due to the challenges in camera traps images like occlusion, camouflage, poor illumination, pose variation, etc. In this paper, we propose a fine-grained animal breed classification model using supervised clustering based on Multi Part-Convolutional Neural Network (MP-CNN) and Expectation–Maximization (EM) clustering. The proposed model follows a straightforward pipeline that combines the deep feature extraction using the CNN pre-trained on ImageNet and classifies unsupervised data using EM clustering. Further, we also propose a multi discriminative part selection and detection for the precise classification of animal breeds without using bounding box and annotations on both training and testing phases. The model is tested on several benchmark datasets for animals, including the largest camera trap Snapshot Serengeti dataset and has achieved a cumulative accuracy of 98.4%. The results from the proposed model strengthen the belief that supervised training of deep CNN on a large and versatile dataset, extracts better features than most of the traditional approaches, even for the unsupervised tasks.

Learning nonlinear turbulent dynamics from partial observations via analytically solvable conditional statistics

Learning nonlinear turbulent dynamics from partial observations is an important and challenging topic. In this article, an efficient learning algorithm based on the expectation-maximization approach is developed for a rich class of complex nonlinear turbulent dynamics using short training data. Despite the significant nonlinear and non-Gaussian features in these models, the analytically solvable conditional statistics allows the development of an exact and accurate nonlinear optimal smoother for recovering the hidden variables, which facilitates an efficient learning of these fully nonlinear models with extreme events. Then three additional ingredients are incorporated into the basic algorithm for improving the learning process. First, the physics constraint that requires the conservation of energy in the quadratic nonlinear terms is taken into account. It plays an important role in preventing the finite-time blowup of the solution and various pathological behavior of the recovered model. Second, a judicious block decomposition is applied to many large-dimensional nonlinear systems. It greatly accelerates the calculation of high-dimensional conditional covariance matrix and provides an extremely cheap parallel computation for learning the model parameters. Third, sparse identification of the complex turbulent models is combined with the learning algorithm that leads to parsimonious models. Numerical tests show the skill of the algorithm in learning the nonlinear dynamics and non-Gaussian statistics with extreme events in both perfect model and model error scenarios. It is also shown that in the presence of noise and partial observations, the model is not uniquely identified. Different nonlinear models all perfectly capture the key non-Gaussian features and obtain the same ensemble forecast skill of the observed variables as the perfect model, but they may have distinct model responses to external perturbations.

Gap-filling continuously-measured soil respiration data: A highlight of time-series-based methods

Soil respiration is an important ecosystem process that releases carbon dioxide into the atmosphere. While soil respiration can be measured continuously at high temporal resolutions, gaps in the dataset are inevitable, leading to uncertainties in carbon budget estimations. Therefore, robust methods used to fill the gaps are needed. The process-based non-linear least squares (NLS) regression is the most widely used gap-filling method, which utilizes the established relationship between the soil respiration and temperature. In addition to NLS, we also implemented three other methods based on: 1) artificial neural networks (ANN), driven by temperature and moisture measurements, 2) singular spectrum analysis (SSA), relying only on the time series itself, and 3) the expectation-maximization (EM) approach, referencing to parallel flux measurements in the spatial vicinity. Six soil respiration datasets (2017–2019) from two boreal forests were used for benchmarking. Artificial gaps were randomly introduced into the datasets and then filled using the four methods. The time-series-based methods, SSA and EM, showed higher accuracies than NLS and ANN in small gaps (<1 day). In larger gaps (15 days), the performance was similar among NLS, SSA and EM; however, ANN showed large errors in gaps that coincided with precipitation events. Compared to the observations, gap-filled data by SSA showed similar degree of variances and those filled by EM were associated with similar first-order autocorrelation coefficients. In contrast, data filled by both NLS and ANN exhibited lower variance and higher autocorrelation than the observations. For estimations of the annual soil respiration budget, NLS, SSA and EM resulted in errors between −3.7% and 5.8% given the budgets ranged from 463 to 1152 g C m−2 year−1, while ANN exhibited larger errors from −11.3 to 16.0%. Our study highlights the two time-series-based methods which showed great potential in gap-filling carbon flux data, especially when environmental variables are unavailable.

Unsupervised Bayesian Subpixel Mapping of Hyperspectral Imagery Based on Band-Weighted Discrete Spectral Mixture Model and Markov Random Field

Although accurate training and initialization information is difficult to acquire, unsupervised hyperspectral subpixel mapping (SPM) without relying on this predefined information is an insufficiently addressed research issue. This letter presents a novel Bayesian approach for unsupervised SPM of hyperspectral imagery (HSI) based on the Markov random field (MRF) and a band-weighted discrete spectral mixture model (BDSMM), with the following key characteristics. First, this is an unsupervised approach that allows adjustment of abundance and endmember information adaptively for less relying on algorithm initialization. Second, this approach consists of the BDSMM for accommodating the noise heterogeneity and the hidden label field of subpixels in HSI. The BDSMM also integrates SPM into the spectral mixture analysis and allows enhanced SPM by fully exploring the endmember-abundance patterns in HSI. Third, the MRF and BDSMM are integrated into a Bayesian framework to use both the spatial and spectral information efficiently, and an expectation-maximization (EM) approach is designed to solve the model by iteratively estimating the endmembers and the label field. Experiments on both simulated and real HSI demonstrate that the proposed algorithm can yield better performance than traditional methods.

ECM Algorithm for Estimating Vector ARMA Model with Variance Gamma Distribution and Possible Unbounded Density

The simultaneous analysis of several financial time series is important in portfolio setting and risk management. This paper proposes a novel alternating Expectation conditional Maximisation (AECM) algorithm to estimate the vector autoregressive moving average (VARMA) model with variance gamma (VG) error distribution in the multivariate skewed setting. We explain why VARMA-VG model is suitable for high frequency returns (HFRs) because VG distribution provides thick tails to capture the high kurtosis in the data and the unbounded central density further captures the majority of near zero HFRs. The distribution can also be expressed in normal mean-variance mixtures which facilitate model implementation using Bayesian and expectation maximisation (EM) approaches. We adopt the EM approach which avoids the time consuming Markov chain Monto Carlo sampling and solve the unbounded density problem in the classical maximum likelihood estimation. We discuss some properties of VARMA model, conduct extensive simulation studies to evaluate the accuracy of the proposed AECM estimator and apply the models to analyse the dependency between several HFR series.

Mining an "anti-knowledge base" from Wikipedia updates with applications to fact checking and beyond

We introduce the problem of anti-knowledge mining. Our goal is to create an "anti-knowledge base" that contains factual mistakes. The resulting data can be used for analysis, training, and benchmarking in the research domain of automated fact checking. Prior data sets feature manually generated fact checks of famous misclaims. Instead, we focus on the long tail of factual mistakes made by Web authors, ranging from erroneous sports results to incorrect capitals.We mine mistakes automatically, by an unsupervised approach, from Wikipedia updates that correct factual mistakes. Identifying such updates (only a small fraction of the total number of updates) is one of the primary challenges. We mine anti-knowledge by a multi-step pipeline. First, we filter out candidate updates via several simple heuristics. Next, we correlate Wikipedia updates with other statements made on the Web. Using claim occurrence frequencies as input to a probabilistic model, we infer the likelihood of corrections via an iterative expectation-maximization approach. Finally, we extract mistakes in the form of subject-predicate-object triples and rank them according to several criteria. Our end result is a data set containing over 110,000 ranked mistakes with a precision of 85% in the top 1% and a precision of over 60% in the top 25%. We demonstrate that baselines achieve significantly lower precision. Also, we exploit our data to verify several hypothesis on why users make mistakes. We finally show that the AKB can be used to find mistakes on the entire Web.

Direct prediction of regulatory elements from partial data without imputation.

Genome segmentation approaches allow us to characterize regulatory states in a given cell type using combinatorial patterns of histone modifications and other regulatory signals. In order to analyze regulatory state differences across cell types, current genome segmentation approaches typically require that the same regulatory genomics assays have been performed in all analyzed cell types. This necessarily limits both the numbers of cell types that can be analyzed and the complexity of the resulting regulatory states, as only a small number of histone modifications have been profiled across many cell types. Data imputation approaches that aim to estimate missing regulatory signals have been applied before genome segmentation. However, this approach is computationally costly and propagates any errors in imputation to produce incorrect genome segmentation results downstream. We present an extension to the IDEAS genome segmentation platform which can perform genome segmentation on incomplete regulatory genomics dataset collections without using imputation. Instead of relying on imputed data, we use an expectation-maximization approach to estimate marginal density functions within each regulatory state. We demonstrate that our genome segmentation results compare favorably with approaches based on imputation or other strategies for handling missing data. We further show that our approach can accurately impute missing data after genome segmentation, reversing the typical order of imputation/genome segmentation pipelines. Finally, we present a new 2D genome segmentation analysis of 127 human cell types studied by the Roadmap Epigenomics Consortium. By using an expanded set of chromatin marks that have been profiled in subsets of these cell types, our new segmentation results capture a more complex picture of combinatorial regulatory patterns that appear on the human genome.

Deterministic and Bayesian Sparse Signal Processing Algorithms for Coherent Multipath Directions-of-Arrival (DOAs) Estimation

In this paper, we propose novel deterministic and Bayesian methods to identify the directions-of-arrival (DOAs) of coherent multipath signals assuming that a very few number of multipath signals are present and multiple snapshots are available. For both types of methods, we exploit both sparsity and the underlying structure of coherent signal propagation. For the deterministic case, we formulate the problem of estimating the DOAs of coherent multipath signals as a biconvex optimization problem and then solve it by an alternating convex search approach. This is in contrast to the widely used 11 sparse signal recovery convex optimization problem, which only exploits sparsity of the signal under consideration. For the Bayesian case, we propose two novel algorithms based on the mean field variational Bayesian expectation maximization approach. The first algorithm assumes that the true scenario DOAs of the multipath signals are exactly aligned with the angular grids. We next extend the on-grid model to deal with the off-grid problem, which is the second proposed Bayesian algorithm of this paper. These two Bayesian algorithms are in contrast to the widely used sparse Bayesian learning relevance vector machine algorithm, which only exploits sparsity in spatial domain. A simulation study is carried out in terms of root-mean-squared error in the DOA estimates to compare the performances of different algorithms. Finally, we demonstrate the application of the proposed off-grid Bayesian algorithm by analyzing data from the shallow water HF97 ocean acoustic experiment.

Development of the Hemispherical Rotational Modulation Collimator Imaging System

A rotational modulation collimator (RMC) imager consists of two collimator masks rotating ahead of a non-position-sensitive detector. It requires fewer detectors than other radiation imagers, which offers the possibility of reducing the instrument complexity and cost. A hemispherical RMC (H-RMC) is capable of imaging radiation with a significantly better field of view (FOV) than conventional RMCs with cylindrical geometry. In this study, a novel hemispherical collimator design was developed in order to extend the FOV to nearly $2\pi $ . We designed the hemispherical mask considering several parameters, demonstrated the measured performance of the instrument, and showed the two-dimensional (2-D) reconstructed images of radiation distribution. The H-RMC imager described herein featured a symmetric design that leads to degenerate reconstruction of two possible source locations for a single true source location. Nevertheless, it has been possible to establish that the maximum likelihood expectation maximization (MLEM) approach with regularization technique significantly reduced the artifact of misestimated source positions and improved the values of signal-to-noise ratio (SNR) and the structural similarity (SSIM) index compared to the conventional MLEM method.

Worldwide human mitochondrial haplogroup distribution from urban sewage

Community level genetic information can be essential to direct health measures and study demographic tendencies but is subject to considerable ethical and legal challenges. These concerns become less pronounced when analyzing urban sewage samples, which are ab ovo anonymous by their pooled nature. We were able to detect traces of the human mitochondrial DNA (mtDNA) in urban sewage samples and to estimate the distribution of human mtDNA haplogroups. An expectation maximization approach was used to determine mtDNA haplogroup mixture proportions for samples collected at each different geographic location. Our results show reasonable agreement with both previous studies of ancient evolution or migration and current US census data; and are also readily reproducible and highly robust. Our approach presents a promising alternative for sample collection in studies focusing on the ethnic and genetic composition of populations or diseases associated with different mtDNA haplogroups and genotypes.

Robust optimization of a pharmaceutical freeze-drying process under non-Gaussian parameter uncertainties

Model-based design of pharmaceutical manufacturing processes has received much interest in academia and industry. Model parameter uncertainties, however, might deteriorate the predicted process performance. Probability-based robust process design concepts as a countermeasure against uncertainties might be implemented. Here, parameter uncertainties are typically limited to Gaussian parameter distributions. However, parameter uncertainties derived with experimental data can be correlated and arbitrarily distributed. In our previous work, transformation techniques were combined with the point estimate method (PEM) to address non-Gaussian and correlated parameter distributions, but at the cost of additional nonlinearities and approximation errors. In this work, we take advantage of Gaussian mixture distributions (GMD) and decompose the parameter distribution into a finite set of Gaussian distributions using the Expectation-Maximization approach. Combining the GMD with the PEM ensures a proper and effective uncertainty quantification. The improved PEM algorithm is applied to a freeze-drying process (lyophilization) aiming for high-quality products with minimum processing time. Results obtained suggest that the novel GMD-PEM algorithm has the potential to outperform conventional robustification concepts regarding credibility and efficiency.

Edge2vec: Representation learning using edge semantics for biomedical knowledge discovery

BackgroundRepresentation learning provides new and powerful graph analytical approaches and tools for the highly valued data science challenge of mining knowledge graphs. Since previous graph analytical methods have mostly focused on homogeneous graphs, an important current challenge is extending this methodology for richly heterogeneous graphs and knowledge domains. The biomedical sciences are such a domain, reflecting the complexity of biology, with entities such as genes, proteins, drugs, diseases, and phenotypes, and relationships such as gene co-expression, biochemical regulation, and biomolecular inhibition or activation. Therefore, the semantics of edges and nodes are critical for representation learning and knowledge discovery in real world biomedical problems.ResultsIn this paper, we propose the edge2vec model, which represents graphs considering edge semantics. An edge-type transition matrix is trained by an Expectation-Maximization approach, and a stochastic gradient descent model is employed to learn node embedding on a heterogeneous graph via the trained transition matrix. edge2vec is validated on three biomedical domain tasks: biomedical entity classification, compound-gene bioactivity prediction, and biomedical information retrieval. Results show that by considering edge-types into node embedding learning in heterogeneous graphs, edge2vec significantly outperforms state-of-the-art models on all three tasks.ConclusionsWe propose this method for its added value relative to existing graph analytical methodology, and in the real world context of biomedical knowledge discovery applicability.

Robust Model Selection and Estimation for Censored Survival Data with High Dimensional Genomic Covariates.

When relating genomic data to survival outcomes, there are three main challenges that are the censored survival outcomes, the high-dimensionality of the genomic data, and the non-normality of data. We propose a method to tackle these challenges simultaneously and obtain a robust estimation of detecting significant genes related to survival outcomes based on Accelerated Failure Time (AFT) model. Specifically, we include a general loss function to the AFT model, adopt model regularization and shrinkage technique, cope with parameters tuning and model selection, and develop an algorithm based on unified Expectation-Maximization approach for easy implementation. Simulation results demonstrate the advantages of the proposed method compared with existing methods when the data has heavy-tailed errors and correlated covariates. Two real case studies on patients are provided to illustrate the application of the proposed method.

An Integrated Framework for Mining Temporal Logs from Fluctuating Events

The importance of mining time lags of hidden temporal dependencies from sequential data is highlighted in many domains including system management, stock market analysis, climate monitoring, and more. Mining time lags of temporal dependencies provides useful insights into the understanding of sequential data and predicting its evolving trend. Traditional methods mainly utilize the predefined time window to analyze the sequential items, or employ statistical techniques to identify the temporal dependencies from a sequential data. However, it is a challenging task for existing methods to find the time lag of temporal dependencies in the real world, where time lags are fluctuating, noisy, and interleaved with each other. In order to identify temporal dependencies with time lags in this setting, this paper comes up with an integrated framework from both system and algorithm perspectives. Specifically, a novel parametric model is introduced to model the noisy time lags for temporal dependencies discovery between events. Based on the parametric model, an efficient expectation maximization approach is proposed for time lag discovery with maximum likelihood. Furthermore, this paper also contributes an approximation method for learning time lag to improve the scalability in terms of the number of events, without incurring significant loss of accuracy.