Markov Theory Research Articles

Multi-stage reasoning on introspecting and revising bias for visual question answering

Visual Question Answering (VQA) is a task that involves predicting an answer to a question depending on the content of an image. However, recent VQA methods have relied more on language priors between the question and answer rather than the image content. To address this issue, many debiasing methods have been proposed to reduce language bias in model reasoning. However, the bias can be divided into two categories: good bias and bad bias. Good bias can benefit to the answer prediction, while the bad bias may associate the models with the unrelated information. Therefore, instead of excluding good and bad bias indiscriminately in existing debiasing methods, we proposed a bias discrimination module to distinguish them. Additionally, bad bias may reduce the model’s reliance on image content during answer reasoning and thus attend little on image features updating. To tackle this, we leverage Markov theory to construct a Markov field with image regions and question words as nodes. This helps with feature updating for both image regions and question words, thereby facilitating more accurate and comprehensive reasoning about both the image content and question. To verify the effectiveness of our network, we evaluate our network on VQA v2 and VQA cp v2 datasets and conduct extensive quantity and quality studies to verify the effectiveness of our proposed network. Experimental resu- lts show that our network achieves significant performance against the previous state-of-the-art methods.

Prediction of building carbon emission based on grey Markov theory

The increase in carbon emissions in the construction industry is one of the factors that lead to environmental problems and hinder the sustainable development of society and the construction industry. Therefore, there is an urgent need to rationally control carbon emissions by formulating scientific and effective energy-saving and emission reduction policies so as to safeguard the development of the industry. However, because of the uncertainty and complexity of CO2 emissions, more reliable prediction and assessment tools are needed to comprehensively analyze and predict CO2 emissions. This study combines the gray system theory and Markov principle to give full play to the advantages of the two methods, using Markov theory to determine the state transition probability, find out the characteristics of carbon emissions in the construction industry, and form a suitable gray Markov prediction model. The feasibility of the prediction model is demonstrated by calculating and examining the prediction model using data related to carbon emissions from the U.S. buildings from 2002 to 2022, which greatly improves the accuracy of CO2 emissions prediction.

Reinforcement learning-based energy management for hybrid electric vehicle enhanced by velocity prediction

Abstract This paper proposes a novel approach for optimal energy management of engine-battery hybrid electric systems. It addresses two limitations in existing research, namely inaccurate speed prediction and low control accuracy. The proposed method integrates Markov theory and clustering theory to achieve high-precision prediction of future velocity under diverse driving styles. Furthermore, a learning-based energy management strategy is developed using a deep deterministic strategy gradient algorithm. This strategy considers the predicted velocity and aims to minimize engine fuel consumption, battery state fluctuations, and engine start-stop occurrences. Simulation results demonstrate that the proposed algorithm achieves near-optimal performance under specific operating conditions.

Shortage Policies for a Jump Process with Positive and Negative Batch Arrivals in a Random Environment

We study a continuous-review stock management of a retailer for a single item in a limited storage (buffer) in a random environment. The stock level fluctuates according to two independent compound Poisson processes with discrete amounts of items (batches) that enter and leave the storage facility. The storage facility is controlled by a three-parameter base-stock replenishment policy. All items exceeding the storage capacity are transferred to an unlimited foreign facility. In addition, a restricted backlogging possibility is permitted; additional demands for items are lost sales. We further assume a random shelf life, the possibility of total inventory collapse, and a random lead time. Applying Markov theory, we derive the optimal control parameters minimizing the long-run expected total cost. A sensitivity analysis is conducted focusing on the comparison between the pure lost-sales policy and a partial backordering policy. Accordingly, we identify cases where one policy is cost effective compared to the other, particularly with respect to the batch patterns (sign, rate, average, and variability), and the associated costs.

Optimization model of battery electric vehicle charging facility layout towards embedded system and data mining algorithm

Abstract Battery electric vehicles have been developing rapidly in recent years due to their advantages of energy saving and emission reduction. However, the development of battery electric vehicles is still in its infancy due to the relatively immature charging technology and the lack of infrastructure development. In many researches, there have been many solutions to adopt remote monitoring systems in battery electric vehicles to monitor and collect them in real time for statistical analysis and fault diagnosis. However, the current monitoring systems are carried out statistically and it is difficult to extract much valuable information from the data. In this paper, after analyzing the construction mode of battery electric vehicle charging facilities, the battery electric vehicle charging facility system platform using embedded system was analyzed. Several functions were simulated to meet the problems encountered by users in using it, studying the battery electric vehicle charging facility layout optimization model based on data mining algorithm. Focusing on the coverage problem in the siting problem, the State-of-Charge (SOC) of the battery charge state was analyzed. Taking residential areas as an example and 4 weeks as a cycle, it was found that during the first to fourth weeks, the negative power at 1:00, 20:00 and 24:00 was higher. At the same type of time point, different charging thresholds had a significant impact on the charging load demand of users. The larger the charging threshold of battery electric vehicles appeared, the earlier the peak moment and the higher the peak load. The battery state of the battery electric vehicle in the work area is 60 % at 1:00, and it reaches a relatively full state of 90 % at 8:00. In different regions, the allocation of charging thresholds for battery electric vehicles was different, and the Markov theory was derived from the analysis of the coverage problem to enable fast and effective siting of charging piles while satisfying charging convenience and high utilization of charging facilities to complement the research on the optimization model of battery electric vehicle charging facility layout.

Prediction of machining errors for free-form surfaces based on an improved grey model (1,N)

An improved grey model (GM) for predicting free-form surface machining errors was established to address the low measuring efficiencies of coordinate measuring machines (CMMs) and the low prediction accuracy of the GM(1,1) model. A number of points on a free-form surface are measured with a CMM, and machining errors are obtained. Ideas from metabolic methods and the GM(1,N) prediction model were combined. To reduce the impact of random fluctuations of the machining errors, a Markov prediction model was then used to correct the fitted results for the residuals and obtain the predicted results of the machining errors for free-form surfaces, thus improving the prediction accuracy of the model. The predicted results of the metabolic GM(1,1) and metabolic GM(1,N) models were then compared. The experimental results showed that the combination of metabolic theory, a grey model, and Markov theory effectively improved the prediction accuracy.

Stochastic operation model for readiness assessment of small wind turbines based on Markov theory

AbstractWind energy is now one of the most important alternative sources of renewable energy. The development of wind power generation is a prerequisite to meet environmental and sustainable development requirements while achieving energy security goals. The condition for the effectiveness of the proper operation of a wind power plant is its ability to perform the intended functions, which can be expressed by the coefficient of technical readiness. This article presents an original analysis of the readiness of small wind turbines. As part of the research, an original stochastic operation model was developed based on the application of the Markov process theory. Based on the collected data, a 3‐state phase space of the studied process was identified. Various analyses were carried out to calculate the operating coefficients of wind power plants. The ergodic (boundary) probabilities were calculated using three methods. The conditional probabilities were then determined based on the initial distribution. The resulting readiness factors, not exceeding 30%, indicate the low operational efficiency of the small wind turbines in question. Furthermore, an analysis of the sensitivity of the performance factors was carried out for the analysed objects.

Multiaxial fatigue life prediction of notched specimens based on multidimensional grey Markov theory

It is always a challenge to accurately predict the fatigue life of notched specimens under multiaxial loading, which is a guarantee of structural integrity. Therefore, in this paper, multiple factors affecting fatigue life are considered and a multidimensional grey Markov model is used for fatigue life prediction. Firstly, the plane of maximum shear strain amplitude is determined as the critical plane. Meanwhile, the stress–strain components on the critical plane are extracted, and the relevant parameters affecting the fatigue life are calculated. Secondly, due to the defects of the grey theory itself, the prediction error is larger. Therefore, the grey model is modified with Markov theory to get the grey Markov model. The correlation degree analysis between relevant factors and fatigue life is carried out with grey theory to analyze the degree of influence of different factors on fatigue life. Finally, the grey Markov model is analyzed and verified using experimental data from Q355(D), GH4169 and Al7050 notched specimens and the results show that the grey Markov model can provide an economical and effective method for multiaxial fatigue life prediction.

A systematic strategy to analyse individual‐based models reveals pathways of degradation and recovery in Patagonian rangelands

One of the central problems in ecology is how to scale from small‐scale observations and experiments to large‐scale patterns and processes. One approach to such upscaling is to use dynamic simulation models, but their application to large scales relevant for management is limited by computational costs, and their outputs are difficult to analyse without a systematic strategy. Our general objective is to propose such a strategy. The idea is to approximate the dynamics of detailed simulation models through a set of states, external drivers, and transition matrices, and then use Markov chain and network analysis of the resulting transition matrices to gain insights into the dynamics of the underlying detailed model. We used the individual‐based model COIRON, which simulates the dynamics of semiarid grass steppes in Patagonia (Argentina) under alternative grazing management, as example. Our specific objectives are to identify pathways of degradation and rehabilitation, as well as critical grazing thresholds and early‐warning vegetation states to guide sustainable grazing management in these steppes. Our results indicate nonlinear effects of stocking rate and grazing season on steppe dynamics. Markov chain analysis suggests benefits of seasonal over continuous grazing at intermediate stocking rates, and network analysis of recovery and degradation trajectories shows that intermediate stocking rates maximize differences between grazing seasons. Finally, our analysis identified specific vegetation states as early warning signals that indicate a high risk of irreversible vegetation changes. Patagonian grass steppes should ideally be managed with multi‐paddock grazing at moderate stocking rates around 0.5 sheep ha−1. The transition matrices summarize the relevant key features of the detailed model for larger scales, and applying Markov and network theory provides a systematic strategy to analyse its dynamics to respond to biological questions, both are often difficult to obtain by direct analysis of the detailed model.

State-based transition probabilities for systems subject to periodic inspection interval

ABSTRACT In this paper, we present a general formula to calculate transition probabilities for six different types of systems based on their redundancy strategy and the status of the components. The systems are under periodic inspection policy and their components are repairable. The investigated systems include system I – active redundancy without any component to be replaced or repaired; system II – active with the component(s) to be replaced and repaired; system III – active with the component(s) to be repaired; system IV – standby without any component(s) to be replaced or repaired; system V – standby with the component(s) to be replaced and repaired; and system VI – standby with the component(s) to be repaired. In addition, all components in a system are considered non-identical. To calculate the transition probabilities for systems IV, V, and VI, we first consider a system with n non-identical components and cold standby configuration (NIC-CSC) and calculate the system state probabilities using Markov theory. Then, we present the general formula transition probabilities for systems IV, V, and VI using the results of the NIC-CSC system inspection interval. We demonstrate how to calculate the transition probabilities and matrixes for the six above-mentioned systems using the provided formulas. Moreover, we use the provided formulas to optimize the inspection interval of these systems using a modified full enumeration technique.

Optimal Discrete Estimation of Reference Points of a Discrete-Continuous Markov Process against the Background of Correlated Markov Noise

In this paper, we solve the problem of synthesizing optimal and quasi-optimal algorithms for estimating the reference points of continuous components of a vector discrete-continuous Markov random process with allowance for the known statistical characteristics of additive Markovian correlated noise using the methods of the Markov theory of estimation of random processes. The method of difference measurements was used while synthesizing the algorithms. A block diagram of a quasi-optimal digital filter is given. Using a simple example, simulation shows the performance of a quasi-optimal algorithm in statistically uncertain situations.

DTE-RR: Dynamic Topology Evolution-Based Reliable Routing in VANET

The dynamics of Vehicular Ad-hoc Network (VANET) brings tremendous challenges to reliable routing. In the existing topology evolution schemes, the link connectivity and the interaction between links have yet to be well investigated, resulting in the inability to accurately describe the evolution of routing topology. In this letter, we propose a Dynamic Topology Evolution (DTE) based reliable routing algorithm to improve transmission reliability. The DTE model is constructed to formulate the evolution of network topology. Especially, we analyze the network topology changes from the perspective of nodes and links. Meanwhile, the Dynamic Bond Percolation (DBP) model is extended to describe link interaction. Then, the DTE model is completed by deriving the state transition matrix of the network topology based on the link change using Markov theory. Finally, the DTE-based Reliable Routing (DTE-RR) algorithm is designed to optimize routing paths. The experimental results demonstrate that DTE-RR outperforms the existing protocols regarding packet delivery ratio and end-to-end delay.

Temporal network modeling with online and hidden vertices based on the birth and death process

Complex networks have played an important role in describing real complex systems since the end of the last century. Recently, research on real-world data sets reports intermittent interaction among social individuals. In this paper, we pay attention to this typical phenomenon of intermittent interaction by considering the state transition of network vertices between online and hidden based on the birth and death process. By continuous-time Markov theory, we show that both the number of each vertex’s online neighbors and the online network size are stable and follow the homogeneous probability distribution in a similar form, inducing similar statistics as well. In addition, all propositions are verified via simulations. Moreover, we also present the degree distributions based on small-world and scale-free networks and find some regular patterns by simulations. The application in fitting real networks is discussed.

The Road of Post-Industrialization Transformation in Developing Countries Based on Weighted Markov and Grey Correlation Theory, Taking the Change of Industrial Structure in Heilongjiang Province of China as a Case Study

In the context of economic globalization, the comparative advantages of developing countries and developed countries tend to be more complex. After the economic crisis in 2008, the challenges of developing countries are intensified, and the game is more cyclical; Especially in the critical period when the “Fourth Industrial Revolution” is pending, many countries have increased their efforts in policy formulation and efficiency improvement, focusing on industrial transformation and upgrading, and closely combining structural reform with the industrialization process. Therefore, it is particularly important to analyze, change and forecast the industrial structure of post-industrial regions in developing countries based on data science algorithms, and to reshape the understanding of the adjustment of the world economic order and the evolution of the international trade division system. Based on the proportion of gross regional product and output value of three industries in Heilongjiang Province of China from 2001 to 2020, the minimum deviation model of industrial structure in Heilongjiang Province is constructed through Markov theory. The Lingo software is used to obtain the transition probability matrix of the industrial structure state, and combined with the autocorrelation coefficient of each order and the transfer weight, the change trend of the proportion of the output value of the three industries and the change of the contribution rate of the three industries (to GDP) in Heilongjiang Province in the next 10 years are obtained; At the same time, through the grey correlation index and spss software, this paper analyzes the correlation changes between the three industrial adjustments and economic development in Heilongjiang Province in the past 20 years, discusses the new growth points of economic development in Heilongjiang Province and puts forward corresponding suggestions for the adjustment of different industrial structures in Heilongjiang Province, and finally extends the general rules of the development of post-industrialization in the world. This article believes that it is necessary to adjust the structure of the first, second, and third industries reasonably based on different historical and natural endowments, contemporary backgrounds, and other practical factors, in accordance with local conditions, circumstances, and each with its own emphasis. At the same time, it also requires the support and inclination of government policies; Adapting industrial structure to local economic development while actively leading productivity and local economic development.

A reinforcement learning‐based energy management strategy for a battery–ultracapacitor electric vehicle considering temperature effects

SummaryThe design of energy management strategy (EMS) plays a vital role in the power performance and economy of battery–ultracapacitor for electric vehicles. A reinforcement learning (RL)‐based EMS is proposed to obtain an optimal power allocation strategy for battery–ultracapacitor electric vehicle, and its robustness is verified at different temperatures. First of all, the dynamic characteristic experiments of the battery and ultracapacitor were performed at 10°C, 25°C, and 40°C to obtain mechanism characteristics at different temperatures. Secondly, a genetic algorithm is selected to identify the parameters of the battery and ultracapacitor model. Next, the RL‐based strategy takes the minimum energy loss of the hybrid energy storage system as the reward function and solves the optimal policy based on Markov theory. The simulation results show that the economy of the RL‐based strategy correspondingly improved by 3.05%, 3.20%, and 3.15% at different temperatures in comparison with the fuzzy‐based strategy, and the economic gap between the RL‐based strategy and the DP‐based strategy is further narrowed down to 7.30%, 3.88%, and 8.40% at different temperatures, respectively. Finally, the proposed strategy is validated under different driving conditions, which indicate that the RL‐based strategy can effectively reduce energy consumption and has good robustness at different temperatures.

Prediction of Liaoning Maritime Traffic Accidents Based on Improved SCGM(1,1)c Model

In order to ensure the safety of maritime traffic in Liaoning sea area and improve the prediction accuracy of maritime traffic accidents. In this paper, the number of ship collision accidents in Liaoning sea area from 2012 to 2021 have been statistically analyzed, and the number of accidents have predicted by using the modified SCGM(1,1)C model modified by Markov chain, and have solved by Matlab programming. The results show that Markov theory can effectively overcome the shortage of volatility prediction of cloud grey system. Compared with SCGM(1,1)C model only, the average relative error of the improved model is reduced by 6.349 %, which provides a strong reference for the establishment of ship collision traffic warning mechanism in Liaoning sea area.

THEORETICAL FOUNDATIONS OF FORECASTING THE RESOURCE OF HYDRAULIC STRUCTURES

It is established that in terms of the discrete Markov process, the problem is reduced to the search for unconditional probabilities of the system S at an arbitrary step k in state Si, that is, obtaining a transition probabilities matrix. In this formulation, the model is used for assessing the technical condition of the element; assessing the level of safety of operation of structural elements; ranking elements according to the need for repairs, reconstruction or replacement; in strategic planning of repair or reconstruction costs in conditions of limited funding and forecasting the remaining resource of elements. It is established that the theoretical basis of the study, which aims to predict the resource of hydraulic structures in operation, is the Markov theory of random processes. For a mathematical description of the process of element degradation, the most successful is the mathematical apparatus of the Markov random processes. Determination of the failure intensity parameter is the dominant feature of the Markov phenomenological model of damage accumulation to hydraulic structures' elements. The only parameter of lifecycle management is the failure rate . In the model under consideration, the parameter is determined based on the initial conditions for an individual element obtained from the survey results. Because the parameter λ is determined for an individual element and must be specified each time after the next survey, the accuracy of the model will increase. The proposed model is integral. It does not contain an explicit theoretical apparatus for a material-sensitive element, its static scheme, construction technology, environmental conditions, etc. On the other hand, all these factors and many other secondary ones are taken into account in the model at the moment the state of the element is determined using classification tables containing physical and mechanical signs of degradation. In the theory of structures, the statistical approach to formulating the transition matrix is widespread and is based on historical data from the structure operation system. It is believed that the transition matrix based on the data of the operating system is a more realistic basis for predicting the processes of structures degradation. A large number of foreign studies are devoted to the practical application of the transition matrix based on statistical data, which consider the features of transition matrices related to the bridge operation system in different countries. In this formulation, each element of the transition probability matrix P is the probability that the system in the state will move to state j in one step (i.e., in one year). At the same time, it is considered that there are no operational interventions, so the sub-diagonal elements are zero. As before, the sum of elements of the same line is 1 and the element pjj = 1 because state j is absorbing. For the implementation algorithm of the Markov chain model for forecasting the technical condition of hydraulic structures in general, the initial data are: statistical data of the distribution of structures by the state at the time of the forecast, the rating assessment of the structure is calculated by an expert according to the scale and the forecast time in years. It is established that the degradation properties of structural designs are described by two parameters: the degradation criterion and the failure rate. Any factor of the stress-strain state can be taken as a degradation criterion: reliability, internal forces, or deformations. The degradation criterion can be an arbitrary rating assessment. In our case, the reliability of the element is taken as the degradation criterion, as the most general factor of the stress-strain state.

Robust networked power system load frequency control against hybrid cyber attack

AbstractModern network and communication technologies are essential for the implementation and operation of load frequency control (LFC) systems. The measurements of crucial LFC system parameters will be compromised by attackers, rendering data received by the defence inaccurate and causing frequency fluctuations or even system collapse. To detect the potential attack on measured data and keep the LFC performance, an adaptive event‐triggered scheme with fractional order global sliding mode control scheme is proposed in this paper. Furthermore, Markov theory is employed for the modelling process with energy storage to present a multi‐area LFC power system considering renewable energy and hybrid cyber attacks. Stability and stabilisation criteria are built by employing improved Lyapunov stability theory and second‐order Bessel‐Legendre inequality. Finally, a two‐area LFC system under hybrid cyber attacks and a modified IEEE 39‐bus New England test power system with 3 wind farms are simulated to explore the efficacy of the proposed method.

Modeling comorbidity of chronic diseases using coupled hidden Markov model with bivariate discrete copula.

A range of chronic diseases have a significant influence on each other and share common risk factors. Comorbidity, which shows the existence of two or more diseases interacting or triggering each other, is an important measure for actuarial valuations. The main proposal of the study is to model parallel interacting processes describing two or more chronic diseases by a combination of hidden Markov theory and copula function. This study introduces a coupled hidden Markov model with the bivariate discrete copula function in the hidden process. To estimate the parameters of the model and deal with the numerical intractability of the log-likelihood, we use a variational expectation maximization algorithm. To perform the variational expectation maximization algorithm, a lower bound of the model's log-likelihood is defined, and estimators of the parameters are computed in the M-part. A possible numerical underflow occurring in the computation of forward-backward probabilities is solved. The simulation study is conducted for two different levels of association to assess the performance of the proposed model, resulting in satisfactory findings. The proposed model was applied to hospital appointment data from a private hospital. The model defines the dependency structure of unobserved disease data and its dynamics. The application results demonstrate that the model is useful for investigating disease comorbidity when only population dynamics over time and no clinical data are available.

Optimum Complexing of Measurements when Maintaining a Maneuvering Object in Statistically Uncertain Situations

The problem of synthesizing optimal and quasi-optimal algorithms for complex information processing is solved using the methods of Markov theory for estimating random processes when maintaining a maneuvering object and two-channel vector observation with violations in statistically uncertain situations. The problem is solved in relation to a discrete-continuous Markov process for the case when its continuous part is a vector Markov sequence, and the discrete part is characterized by a three-component discrete Markov process, each component of which is described by a Markov chain to several positions. A block diagram of quasi-optimal complex information processing is given. Using a simple example, simulation modeling shows the performance of a quasi-optimal algorithm in statistically uncertain situations.