Markov Random Field Research Articles

Protein language models learn evolutionary statistics of interacting sequence motifs

Protein language models (pLMs) have emerged as potent tools for predicting and designing protein structure and function, and the degree to which these models fundamentally understand the inherent biophysics of protein structure stands as an open question. Motivated by a finding that pLM-based structure predictors erroneously predict nonphysical structures for protein isoforms, we investigated the nature of sequence context needed for contact predictions in the pLM Evolutionary Scale Modeling (ESM-2). We demonstrate by use of a "categorical Jacobian" calculation that ESM-2 stores statistics of coevolving residues, analogously to simpler modeling approaches like Markov Random Fields and Multivariate Gaussian models. We further investigated how ESM-2 "stores" information needed to predict contacts by comparing sequence masking strategies, and found that providing local windows of sequence information allowed ESM-2 to best recover predicted contacts. This suggests that pLMs predict contacts by storing motifs of pairwise contacts. Our investigation highlights the limitations of current pLMs and underscores the importance of understanding the underlying mechanisms of these models.

MicroNet-MIMRF: A Microbial Network Inference Approach Based on Mutual Information and Markov Random Fields

Abstract Motivation The human microbiome, comprises complex associations and communication networks among microbial communities, which are crucial for maintaining health. The construction of microbial networks is vital for elucidating these associations and co-occurrence patterns. However, existing microbial networks inference methods cannot solve the issues of zero-inflation and non-linear associations. Therefore, necessitating novel methods to improve the accuracy of microbial networks inference. Results In this study, we introduce the Microbial Network based on Mutual Information and Markov Random Fields (MicroNet-MIMRF) as a novel approach for inferring microbial networks. Abundance data of microbes are modeled through the zero-inflated Poisson distribution, and the discrete matrix is estimated for further calculation. Markov random fields based on mutual information are used to construct accurate microbial networks. MicroNet-MIMRF excels at estimating pairwise associations between microbes, effectively addressing zero-inflation and non-linear associations in microbial abundance data. It outperforms commonly used techniques in simulation experiments, achieving area under the curve values exceeding 0.75 for all parameters. A case study on inflammatory bowel disease data further demonstrates the method’s ability to identify insightful associations. Conclusively, MicroNet-MIMRF is a powerful tool for microbial network inference that handles the biases caused by zero-inflation and overestimation of associations. This novel tool has abundant applications in health research. Availability and implementation The MicroNet-MIMRF is provided at https://github.com/Fionabiostats/MicroNet-MIMRF.

Reference to Patients in Nurse Shift Handover Meetings: Exploring the Dynamics of Referring Expressions

ABSTRACT This paper investigates the dynamics of referring expressions in hospital nurse handover meetings when discussing patients. We apply the methods of the Variable Length Markov Chain (VLMC) and network analyses to model the use of referring expressions and evaluate relationships between them. The models reveal second-order dependencies emerging for metonymy and noun phrases. Specifically, metonymy shows a greater association with the beginning of a reference, particularly in the context of other metonymies. In contrast, noun phrases tend to be more strongly associated with later points in the reference. Further, we introduce the notion of referential typicality, which measures the conformity of sequences of referring expressions to anticipated patterns. We show, for example, that consecutive noun phrases fall outside the typical pattern, whereas metonymical sequences and sequences of pronouns are highly typical. The transitions from metonymy or nouns to pronouns also closely align with a highly typical pattern. Using a Generalized Additive Model (GAM), we then track the overall evolution of referential typicality throughout the duration of handover meetings, from their beginning to end. The study reveals a subtle increase in referential typicality towards the end of these sessions, indicating a trend towards more consistent referencing as the discourse unfolds.

A real scale application of a novel set of spatial and similarity features for detection and classification of natural seismic sources from Distributed Acoustic Sensing data

Summary Distributed Acoustic Sensing (DAS) turns a fiber optic into a very dense network of equally-distributed seismic sensors. We focused on the high-density sampling of the seismic wavefield, expressed in strain rates, measured by DAS. Classical approaches used to identify seismic signals rely on the recorded features at one station, but it is difficult to include spatial information in case of dense seismic station networks. This work aims at introducing new spatial and similarity features for seismic event classification suitable to analyze DAS observations. We propose a processing chain based on the XGBoost algorithm and the use of specifically designed spatio-temporal and similarity features for the event classification, and Markov Random Field for the spatial clustering. The methodology is designated to be applied on a continuous stream of DAS observations. We tested our processing chain to detect earthquakes and quarry blasts recorded in the region by permanent seismic networks and included in the RENASS catalog. These events are part of a strain-rate seismic survey carried out during a 3 weeks campaign of DAS measurements along à 91 km fiber optic cable deployed in the central Pyrenees mountains (France). Despite the high anthropogenic activities along the fiber optic path, the proposed method succeeded in detecting earthquakes of magnitude >0.4 and quarry blasts of magnitude >1.0 while limiting the number of false alarms. This performance is particularly noteworthy for low-magnitude events, where detection is accomplished despite a lower signal-to-noise ratio compared to traditional seismometers. The methodology opens the door to real time detection and classification of seismic events measured with long-distance fiber optic systems.

AI-Powered Approaches for Hypersurface Reconstruction in Multidimensional Spaces

The present article explores the possibilities of using artificial neural networks to solve problems related to reconstructing complex geometric surfaces in Euclidean and pseudo-Euclidean spaces, examining various approaches and techniques for training the networks. The main focus is on the possibility of training a set of neural networks with information about the available surface points, which can then be used to predict and complete missing parts. A method is proposed for using separate neural networks that reconstruct surfaces in different spatial directions, employing various types of architectures, such as multilayer perceptrons, recursive networks, and feedforward networks. Experimental results show that artificial neural networks can successfully approximate both smooth surfaces and those containing singular points. The article presents the results with the smallest error, showcasing networks of different types, along with a technique for reconstructing geographic relief. A comparison is made between the results achieved by neural networks and those obtained using traditional surface approximation methods such as Bézier curves, k-nearest neighbors, principal component analysis, Markov random fields, conditional random fields, and convolutional neural networks.

Remote Sensing Image Segmentation Based on Probabilistic Fuzzy Local Information Clustering

Abstract. In this paper, a remote sensing image segmentation based on probabilistic fuzzy local information clustering algorithm is proposed. First, assuming that the spectral measure within the same ground object follows the same probability distribution. The dissimilarity between pixel and object is modeled by the negative logarithm of the Gaussian probability density function. It can improve the noise sensitive problem caused by the Euclidean distance which can only describe the data with isotropic distribution. Then, in order to consider the effect of local spatial constraint, on the one hand, the probability measure is used to modify the local fuzzy factor to establish the dissimilarity measure with spatial constraints. On the other hand, the hidden Markov random field is used to model the prior probability model of pixel membership. Next, the entropy regularization term of the objective function is built by combining the Kullback-Leibler(KL) maximum entropy model to further improve the robustness and noise resistance. The qualitative and quantitative analysis of simulated image and different types of real remote sensing images show that the proposed algorithm can effectively overcome the above problems and further improve the accuracy of image segmentation to over 95%.

Improved MRF rail surface defect segmentation method based on clustering features

Aiming at the characteristics of small number and many types of rail surface defect samples, as well as the problems of unstable transfer learning effect and threshold segmentation being easily affected by environmental factors in real scenes, an improved Markov defect segmentation method with zero samples is proposed. Firstly, the collected data is processed by Gabor function to highlight the defect features and reduce the data dimension to obtain the reduced dimension feature map; Kmeans clustering is performed on the processed feature map to reduce the distribution of data and reduce the influence of reflection and shadow, and the clustering result is used as the pre-classification matrix; an improved Markov random field two-layer graph model is constructed and inferred through the reduced dimension feature map and the pre-classification matrix; the local geometric structure of the defect part is analyzed according to the eigenvalues of the classification matrix inferred by the model; finally, the defect area is marked and the defect segmentation is completed. The experimental part uses a self-sampling data set, and the final conclusion is drawn based on the comparative experiment and ablation experiment. The experimental results show that the pixel accuracy, average pixel accuracy, weighted intersection-over-union ratio, and average intersection-over-union ratio of this method on the self-sampling data set are respectively 93.6%、80.7%、89.4%、68.2% , which exceeds the accuracy of other comparative detection algorithms.

Multi-stage reasoning on introspecting and revising bias for visual question answering

Visual Question Answering (VQA) is a task that involves predicting an answer to a question depending on the content of an image. However, recent VQA methods have relied more on language priors between the question and answer rather than the image content. To address this issue, many debiasing methods have been proposed to reduce language bias in model reasoning. However, the bias can be divided into two categories: good bias and bad bias. Good bias can benefit to the answer prediction, while the bad bias may associate the models with the unrelated information. Therefore, instead of excluding good and bad bias indiscriminately in existing debiasing methods, we proposed a bias discrimination module to distinguish them. Additionally, bad bias may reduce the model’s reliance on image content during answer reasoning and thus attend little on image features updating. To tackle this, we leverage Markov theory to construct a Markov field with image regions and question words as nodes. This helps with feature updating for both image regions and question words, thereby facilitating more accurate and comprehensive reasoning about both the image content and question. To verify the effectiveness of our network, we evaluate our network on VQA v2 and VQA cp v2 datasets and conduct extensive quantity and quality studies to verify the effectiveness of our proposed network. Experimental resu- lts show that our network achieves significant performance against the previous state-of-the-art methods.

Environmental Quality, Extreme Heat, and Healthcare Expenditures

Although the effects of the environment on human health are well-established, the literature on the relationship between the quality of the environment and expenditures on healthcare is relatively sparse and disjointed. In this study, the Environmental Quality Index developed by the Environmental Protection Agency and heatwave days were compared against per capita Medicare spending at the county level. A general additive model with a Markov Random Field smoothing term was used for the analysis to ensure that spatial dependence did not undermine model results. The Environmental Quality Index was found to hold a statistically significant (p < 0.05), multifaceted nonlinear association with spending, as was the average seasonal maximum heat index. The same was not true of heatwave days, however. In a secondary analysis on the individual domains of the index, the social and built environment components were significantly related to spending, but the air, water, and land domains were not. These results provide initial support for the simultaneous benefits of healthcare financing systems to mitigate some dimensions of poor environmental quality and consistently high air temperatures.

A formal goodness-of-fit test for spatial binary Markov random field models.

Binary spatial observations arise in environmental and ecological studies, where Markov random field (MRF) models are often applied. Despite the prevalence and the long history of MRF models for spatial binary data, appropriate model diagnostics have remained an unresolved issue in practice. A complicating factor is that such models involve neighborhood specifications, which are difficult to assess for binary data. To address this, we propose a formal goodness-of-fit (GOF) test for diagnosing an MRF model for spatial binary values. The test statistic involves a type of conditional Moran's I based on the fitted conditional probabilities, which can detect departures in model form, including neighborhood structure. Numerical studies show that the GOF test can perform well in detecting deviations from a null model, with a focus on neighborhoods as a difficult issue. We illustrate the spatial test with an application to Besag's historical endive data as well as the breeding pattern of grasshopper sparrows across Iowa.

Gauss Markov and Flow Balanced Vector Radial Learning network traffic classification on IoT with SDN.

Recent evolution in connected devices modelled a massive stipulation for network traffic resources and classification. Software-defined networking (SDN) enables ML techniques with the Internet of Things (IoT) to automate network traffic. This helps to reduce accuracy and improves latency. Problems by conventional techniques to categorize network traffic acquired from IoT and assign resources can be resolved through SDN solutions. This manuscript proposes a proposed network traffic classification technique on IoT with SDN called Gauss Markov and Flow-balanced Vector Radial Learning (GM-FVRL). With the network traffic features acquired from the IoT devices, SDN-enabled Gauss Markov Correlation-based IoT Network Traffic Feature Extraction is applied to extort relevant network aspects. Next, the flow-balanced radial-based ML model for network traffic categorization uses the relevant extracted network traffic features. With the aid of flow, the balanced radial basis function reduces the influence of noise due to distinct network flow. This helps to improve accuracy and minimize latency. Due to this, better precision and recall is ensured. Performance of our method has been evaluated utilizing a scheme using an SDN traffic dataset. The results show that our method classifies the network traffic with high classification accuracy and minimum latency, ensuring better precision and recall.

Statistical batch-aware embedded integration, dimension reduction, and alignment for spatial transcriptomics.

Spatial transcriptomics (ST) technologies provide richer insights into the molecular characteristics of cells by simultaneously measuring gene expression profiles and their relative locations. However, each slice can only contain limited biological variation, and since there are almost always non-negligible batch effects across different slices, integrating numerous slices to account for batch effects and locations is not straightforward. Performing multi-slice integration, dimensionality reduction, and other downstream analyses separately often results in suboptimal embeddings for technical artifacts and biological variations. Joint modeling integrating these steps can enhance our understanding of the complex interplay between technical artifacts and biological signals, leading to more accurate and insightful results. In this context, we propose a hierarchical hidden Markov random field model STADIA to reduce batch effects, extract common biological patterns across multiple ST slices, and simultaneously identify spatial domains. We demonstrate the effectiveness of STADIA using five datasets from different species (human and mouse), various organs (brain, skin, and liver), and diverse platforms (10x Visium, ST, and Slice-seqV2). STADIA can capture common tissue structures across multiple slices and preserve slice-specific biological signals. In addition, STADIA outperforms the other three competing methods (PRECAST, fastMNN, and Harmony) in terms of the balance between batch mixing and spatial domain identification, and it demonstrates the advantage of joint modeling when compared to STAGATE and GraphST. The source code implemented by R is available at https://github.com/zhanglabtools/STADIA and archived with version 1.01 on Zenodo https://zenodo.org/records/13637744.

IoT Sensor Based Cross-Basin Natural Ecological Environment QualityMonitoring and Modeling Simulation with Artificial Intelligence RemoteSensing and GIS

This study presents an integrated approach for monitoring and modelling the quality of crossbasin natural ecological environments using advanced techniques such as Markov Random Field Clustering Classification (MRF-CC), Geographic Information Systems (GIS), Hidden Markov Models (HMM), and remote sensing. With key ecological parameters including water quality, biodiversity, habitat suitability, and land cover types across various scenarios and locations within a study area. The simulation results from MRF-CC revealed the significant impacts of different environmental scenarios and management actions, with restoration efforts showing improvements in ecological quality, while pollution mitigation and urbanization pressures led to declines. The simulation results from MRF-CC revealed the significant impacts of different environmental scenarios and management actions. For example, restoration efforts (Scenario 1) improved water quality (pH 7.5), biodiversity index (0.88), and habitat suitability (0.78) compared to the baseline values (pH 7.2, biodiversity index 0.85, habitat suitability 0.75). Conversely, pollution mitigation (Scenario 2) and urbanization pressure (Scenario 8) resulted in declines, with water quality dropping to pH 7.0 and 6.9, biodiversity indices to 0.82 and 0.81, and habitat suitability to 0.72 and 0.71, respectively. GIS estimation provided spatial insights into ecological parameters, revealing variability across different locations. For instance, Point A (forest) exhibited a water quality index of 0.78, while Point D (urban) showed a lower index of 0.54. The integration of HMM offered probabilistic predictions of land cover dynamics, with probabilities ranging from 0.85 for forest at Point A to 0.45 for urban land cover at Point D.

Implicit Is Not Enough: Explicitly Enforcing Anatomical Priors inside Landmark Localization Models.

The task of localizing distinct anatomical structures in medical image data is an essential prerequisite for several medical applications, such as treatment planning in orthodontics, bone-age estimation, or initialization of segmentation methods in automated image analysis tools. Currently, Anatomical Landmark Localization (ALL) is mainly solved by deep-learning methods, which cannot guarantee robust ALL predictions; there may always be outlier predictions that are far from their ground truth locations due to out-of-distribution inputs. However, these localization outliers are detrimental to the performance of subsequent medical applications that rely on ALL results. The current ALL literature relies heavily on implicit anatomical constraints built into the loss function and network architecture to reduce the risk of anatomically infeasible predictions. However, we argue that in medical imaging, where images are generally acquired in a controlled environment, we should use stronger explicit anatomical constraints to reduce the number of outliers as much as possible. Therefore, we propose the end-to-end trainable Global Anatomical Feasibility Filter and Analysis (GAFFA) method, which uses prior anatomical knowledge estimated from data to explicitly enforce anatomical constraints. GAFFA refines the initial localization results of a U-Net by approximately solving a Markov Random Field (MRF) with a single iteration of the sum-product algorithm in a differentiable manner. Our experiments demonstrate that GAFFA outperforms all other landmark refinement methods investigated in our framework. Moreover, we show that GAFFA is more robust to large outliers than state-of-the-art methods on the studied X-ray hand dataset. We further motivate this claim by visualizing the anatomical constraints used in GAFFA as spatial energy heatmaps, which allowed us to find an annotation error in the hand dataset not previously discussed in the literature.

Approximate cross-validated mean estimates for Bayesian hierarchical regression models

We introduce a novel procedure for obtaining cross-validated predictive estimates for Bayesian hierarchical regression models (BHRMs). BHRMs are popular for modeling complex dependence structures (e.g., Gaussian processes and Gaussian Markov random fields) but can be computationally expensive to run. Cross-validation (CV) is, therefore, not a common practice to evaluate the predictive performance of BHRMs. Our method circumvents the need to re-run computationally costly estimation methods for each cross-validation fold and makes CV more feasible for large BHRMs. We shift the CV problem from probability-based sampling to a familiar and straightforward optimization problem by conditioning on the variance-covariance parameters. Our approximation applies to leave-one-out CV and leave-one-cluster-out CV, the latter of which is more appropriate for models with complex dependencies. In many cases, this produces estimates equivalent to full CV. We provide theoretical results, demonstrate the efficacy of our method on publicly available data and in simulations, and compare the model performance with several competing methods for CV approximation. Code and other supplementary materials available online.

An Underwater Stereo Matching Method: Exploiting Segment-Based Method Traits without Specific Segment Operations

Stereo matching technology, enabling the acquisition of three-dimensional data, holds profound implications for marine engineering. In underwater images, irregular object surfaces and the absence of texture information make it difficult for stereo matching algorithms that rely on discrete disparity values to accurately capture the 3D details of underwater targets. This paper proposes a stereo method based on an energy function of Markov random field (MRF) with 3D labels to fit the inclined plane of underwater objects. Through the integration of a cross-based patch alignment approach with two label optimization stages, the proposed method demonstrates features akin to segment-based stereo matching methods, enabling it to handle images with sparse textures effectively. Through experiments conducted on both simulated UW-Middlebury datasets and real deteriorated underwater images, our method demonstrates superiority compared to classical or state-of-the-art methods by analyzing the acquired disparity maps and observing the three-dimensional reconstruction of the underwater target.

A Markov random field model for change points detection

Detecting Change Points (CPs) in data sequences is a challenging problem that arises in a variety of disciplines, including signal processing and time series analysis. While many methods exist for PieceWise Constant (PWC) signals, relatively fewer address PieceWise Linear (PWL) signals due to the challenge of preserving sharp transitions. This paper introduces a Markov Random Field (MRF) model for detecting changes in slope. The number of CPs and their locations are unknown. The proposed method incorporates PWL prior information using MRF framework with an additional boolean variable called Line Process (LP), describing the presence or absence of CPs. The solution is then estimated in the sense of maximum a posteriori. The LP allows us to define a non-convex non-smooth energy function that is algorithmically hard to minimize. To tackle the optimization challenge, we propose an extension of the combinatorial algorithm DPS, initially designed for CP detection in PWC signals. Also, we present a shared memory implementation to enhance computational efficiency. Numerical studies show that the proposed model produces competitive results compared to the state-of-the-art methods. We further evaluate the performance of our method on three real datasets, demonstrating superior and accurate estimates of the underlying trend compared to competing methods.

Spatial Variability in Relationships between Early Childhood Lead Exposure and Standardized Test Scores in Fourth Grade North Carolina Public School Students (2013-2016).

Exposure to lead during childhood is detrimental to children's health. The extent to which the association between lead exposure and elementary school academic outcomes varies across geography is not known. Estimate associations between blood lead levels (BLLs) and fourth grade standardized test scores in reading and mathematics in North Carolina using models that allow associations between BLL and test scores to vary spatially across communities. We link geocoded, individual-level, standardized test score data for North Carolina public school students in fourth grade (2013-2016) with detailed birth records and blood lead testing data retrieved from the North Carolina childhood blood lead state registry on samples typically collected at 1-6 y of age. BLLs were categorized as: (reference), , and . We then fit spatially varying coefficient models that incorporate information sharing (smoothness), across neighboring communities via a Gaussian Markov random field to provide a global estimate of the association between BLL and test scores, as well as census tract-specific estimates (i.e., spatial coefficients). Models adjusted for maternal- and child-level covariates and were fit separately for reading and math. The average BLL across the 91,706 individuals in the analysis dataset was . Individuals were distributed across 2,002 (out of 2,195) census tracts in North Carolina. In models adjusting for child sex, birth weight percentile for gestational age, and Medicaid participation as well as maternal race/ethnicity, educational attainment, marital status, and tobacco use, BLLs of , and were associated with overall lower reading test scores of [95% confidence interval (CI): , ], (, ), and (, ), respectively. For BLLs of , , and , spatial coefficients-that is, tract-specific adjustments in reading test score relative to the "global" coefficient-ranged from to 2.52, to 3.90, to 7.85, and to 4.33, respectively. Results for mathematics were similar to those for reading. The association between lead exposure and reading and mathematics test scores exhibits considerable heterogeneity across North Carolina communities. These results emphasize the need for prevention and mitigation efforts with respect to lead exposures everywhere, with special attention to locations where the cognitive impact is elevated. https://doi.org/10.1289/EHP13898.

Estimating the effects of smolt size and migration timing on salmon marine survival using a multivariate mixed‐effect model

Abstract When the conditions encountered by a migratory species are highly variable, the timing of migration and migrant condition are critical to survival. Spring/summer‐run Chinook salmon in the Snake River of the Northwestern United States are listed as threatened under the US Endangered Species Act because of decades of poor adult returns; juvenile smolt‐to‐adult survival is often 0.5% or less. Juveniles pass through eight dams en route to the ocean, and despite changes to the hydropower system, there has been little recent improvement in survival. We analysed the effect of migration date, fish length and their interaction on the survival to adulthood of 409,747 wild juvenile Chinook salmon as they migrated through a large, dynamic riverine ecosystem and through the ocean, measured over 20 years. Parametric 2D smoothers are often used to quantify unexplained heterogeneity across space; however, these same statistical models can be applied to 2D datasets to uncover patterns of covariance of two biological processes important to management actions. We employ Gaussian Markov random fields (GMRF) and a 2D smoother to estimates of fish survival to identify the nature of variation in survival with respect to length and date. We found significant effects of fish length, migration date and the date‐by‐length interaction on resulting riverine and marine survival. Unlike the gradual shifts in survival in relation to length and timing that marginal estimates demonstrate, we show that within year, changes in survival with length and timing are often more abrupt than previously thought. Synthesis and applications: Inter‐annually, we identified distinct combinations of fish length and migration timing leading to elevated smolt‐to‐adult survival. Often these ‘hotspots’ are compressed into fish sizes greater than 100 mm and passage timings of only a week or two, demonstrating that even modest shifts in body size and migration timing can have a significant effect on survival. Further research can concentrate on these distinct annual periods to identify sources of mortality that have previously been elusive.

Computing marginal and conditional divergences between decomposable models with applications in quantum computing and earth observation

AbstractThe ability to compute the exact divergence between two high-dimensional distributions is useful in many applications, but doing so naively is intractable. Computing the $$\alpha \beta $$ α β -divergence—a family of divergences that includes the Kullback–Leibler divergence and Hellinger distance—between the joint distribution of two decomposable models, i.e., chordal Markov networks, can be done in time exponential in the treewidth of these models. Extending this result, we propose an approach to compute the exact $$\alpha \beta $$ α β -divergence between any marginal or conditional distribution of two decomposable models. In order to do so tractably, we provide a decomposition over the marginal and conditional distributions of decomposable models. We then show how our method can be used to analyze distributional changes by first applying it to the benchmark image dataset QMNIST and a dataset containing observations from various areas at the Roosevelt Nation Forest and their cover type. Finally, based on our framework, we propose a novel way to quantify the error in contemporary superconducting quantum computers.