State Transition Probability Matrix Research Articles

Construction of Cybersecurity and Risk Prediction Model for New Energy Power Plants under Machine Learning Algorithm

Abstract Addressing the complex equipment and network challenges in new energy power plant industrial control systems, this study introduces a Markov time-varying machine learning algorithm, leveraging classification-constrained Boltzmann machines for real-time network security risk prediction. By employing a hybrid training mode for innovative feature extraction and classification, the algorithm forecasts future security risk states through an up-to-date state transition probability matrix. Integrating with the Markov time-varying model enhances the efficiency over traditional Boltzmann machines, facilitating nuanced network state analyses. The proposed model demonstrates high effectiveness against various network attacks, with average precision, recall, and F1 scores of 0.96, 0.93, and 0.94, respectively, and maintains over 80% accuracy under noise levels up to 40 dB. This research provides a solid foundation for proactive security defense mechanisms in industrial control systems.

Finite-state Markov modeling of Gamma-Gamma fading FSO links with pointing errors

The knowledge of atmospheric channel is quite imperative for free-space optical (FSO) communications. To capture the time-varying essence of intensity fluctuations in an FSO connection, we propose a finite-state Markov (FSM) model for FSO links affected by Gamma-Gamma distributed atmospheric turbulence and pointing errors. First, the level crossing rate (LCR) of the unified intensity fluctuation is derived considering the non-generic channel autocorrelation characteristics, based on which the state transition probability (STP) matrix of the FSM model is calculated. Then, numerical simulations are presented for different strength parameters and frequency parameters of the channel. The results show that the misalignment strength does not affect the ratio of the level where the LCR reaches its maximum to the mean level. Finally, a series of well-designed experiments are carried out to verify the theory. As a result, the theoretical predictions conform to the experimental data, with averaged STP relative errors below 6.7%.

APPLICATION OF EXPECTED CREDIT LOSS MODEL AND MARKOV CHAIN TO CALCULATE NET SINGLE PREMIUM OF UNSECURED CREDIT INSURANCE

Transferring credit risk to an insurance company is a way to mitigate risk. Premiums should be calculated accurately to attain economic value for both the lender and the guarantor. The aim of this study was to determine the net single premium (NSP) values for an unsecured credit insurance product using the expected credit loss (ECL) method from IFRS 9. This study used data generated through simulation of insurance policies issued in 2015 or 2016. Their state classifications were monthly observed from 2016 to 2020. The probability of disbursed claim (PDC) parameter replaced the probability of default parameter on the ECL model, whereas the PDC model was constructed based on the components of a state-transition probability matrix, obtained with the Markov chain approach using the cohort method: = 0.999181, = 0.000130, and = 0.000689. The PDC model validation showed relatively decent results, whereas MSE = 2.457% and zs = 0.608 with a = 5%. These results indicated that the PDC model was a good fit to calculate ECL. 5,000 iterations were done as part of the cash flow simulation process, whereas debtors’ loan amounts were randomly generated during each iteration, and the average NPV of these iterations was -Rp564.419.305. Based on model sensitivity analysis, cash flow values were most sensitive to the variable used to construct the PDC model (). Thus, the 5,000-iteration process was repeated with the newly adjusted PDC value, which were = 0.998924 and = 0.000946. The new average NPV of these iterations was Rp409,877,840, indicating that the constructed ECL model was a good fit to calculate NSP values for unsecured credit insurance products.

Optimal online energy management strategy of a fuel cell hybrid bus via reinforcement learning

An energy management strategy (EMS) based on reinforcement learning is proposed in this study to enhance the fuel economy and durability of a fuel cell hybrid bus (FCHB). Firstly, a comprehensive powertrain system model for the FCHB is established, mainly including the FCHB’s power balance, fuel cell system (FCS) efficiency, and aging models. Secondly, the state–action space, state transition probability matrix (TPM), and multi-objective reward function of Q-learning algorithm are designed to improve the fuel economy and the durability of power sources. The FCHB’s demand power and battery state of charge (SOC) serve as the state variables and the FCS output power is used as the action variable. Using the demonstration FCHB data, a state TPM is created to represent the overall operation. Finally, an EMS employing Q-learning is formulated to optimize the fuel economy of FCHB, maintain battery SOC, suppress FCS power fluctuations, and enhance FCS lifetime. The proposed EMS is tested and verified through hardware-in-the-loop (HIL) tests. The simulation results demonstrate the effectiveness of the proposed strategy. Compared to a rule-based EMS, the Q-learning-based EMS can improve the energy economy by 7.8%. Furthermore, it is only a 3.7% difference to the best energy economy under dynamic optimization, while effectively reducing the decline and enhancing the durability of the FCS.

A Quantitative Framework for Propagation Paths of Natech Domino Effects in Chemical Industrial Parks: Part I—Failure Analysis

Along with global climate change and industrialization, domino effects caused by Natech events occurred frequently in chemical industrial parks over the past decades. Previous research has not yet proposed a reliable method to obtain all possible paths of Natech domino effects, and moreover, a risk assessment and mitigation system has not been established. The present work aims to develop a quantitative framework for propagation paths of Natech domino effects, which can effectively safeguard the sustainable development of chemical industrial parks. The presentation of this work is divided into two parts: Part I (current paper) proposes a path probability calculation method that can simultaneously consider multiple primary accident scenarios and multi-level domino effects triggered by natural disasters. The proposed method transforms the propagation paths of domino effects into the paths of directed graph by constructing the equipment failure state transition matrix and the equipment failure state transition probability matrix. The depth-first traversal algorithm is used to obtain all possible propagation paths and their propagation probabilities, providing data support for the quantitative risk assessment and prevention and control measures presented in the accompanying paper (Part II). The case study shows that the probability of equipment failure caused by multi-level domino effects triggered by Natech accidents is higher than that of conventional accidents. However, the present work only considers the spatial propagation of domino effects, while their spatio-temporal propagation remains as a further direction for this area of inquiry.

Driving cycle prediction based on Markov chain combined with driving information mining

Driving cycle prediction (DCP) is of great importance for vehicle’s awareness of surrounding environment and optimization of control strategy. It has won much attention around the world. However, the used driving database could not reflecting the great diversity in multi driving cycles in real world. The characteristic parameters could not represent the complex and diverse driving conditions. Therefore, DCP models’ adaptability and generalization ability are limited. Moreover, prediction methods in existing researches always take large computation burden, difficult to apply in practice. To tackle these issues, the present research focus on DCP based on Markov chain combined with driving information mining (DIM). Firstly, many standard test cycles and real cycles are collected to construct an database of great diversity. Secondly, DIM technologies are studied to extract the optimal parameter set reflecting driving characteristics and determine the driving cycle categories. Thirdly, prediction method based on recursive self-learning Markov chain is proposed. Recursive equation of Markov state transition probability matrix (TPM) is deduced to save computation time. The key parameter of prediction method is adjusted to achieve optimal prediction performance. Synthesized standard test driving cycles (SSTDC) and real driving cycles (RDC) in database, as well as a new real driving cycle (NRDC) are tested. Driving pattern prediction accuracy of the proposed method is 99.25% for SSTDC, 100.00% for RDC, and 99.84% for NRDC, respectively. Compared with the benchmark method, the prediction accuracy is improved by 18.87% for SSTDC, 10.07% for RDC and 23.51% for NRDC, respectively. In addition, processing time of the prediction program is significantly reduced by 2.23 s (36.14%) for SSTDC, 12.30 s (51.25%) for RDC and 3.99 s (52.16%) for NRDC, respectively. It can be verified that the proposed method has high precision, short response time, as well as good adaptability and charming generalization ability for new cycles.

Assessment of Vigilance Level during Work: Fitting a Hidden Markov Model to Heart Rate Variability

This study aimed to enhance the real-time performance and accuracy of vigilance assessment by developing a hidden Markov model (HMM). Electrocardiogram (ECG) signals were collected and processed to remove noise and baseline drift. A group of 20 volunteers participated in the study. Their heart rate variability (HRV) was measured to train parameters of the modified hidden Markov model for a vigilance assessment. The data were collected to train the model using the Baum-Welch algorithm and to obtain the state transition probability matrix A^ and the observation probability matrix B^. Finally, the data of three volunteers with different transition patterns of mental state were selected randomly and the Viterbi algorithm was used to find the optimal state, which was compared with the actual state. The constructed vigilance assessment model had a high accuracy rate, and the accuracy rate of data prediction for these three volunteers exceeded 80%. Our approach can be used in wearable products to improve their vigilance level assessment functionality or in other fields that have key positions with high concentration requirements and monotonous repetitive work.

A Loosely Coupled Model for Simulating and Predicting Land Use Changes

The analysis and modeling of spatial and temporal changes in land use can reveal changing urban spatial patterns and trends. In this paper, we introduce a linear transformation optimization Markov (LTOM) model that can be exploited to estimate the state transition probability matrix of land use, building a loosely coupled ANN-CA-LTOM model for simulating and predicting land use changes. The advantages of this model are that it is flexible and high expansibility; it can maintain semantic coupling between the Artificial Neural Networks (ANN), Cellular Automata (CA), and LTOM model and enhance their functions; and it can break the limitation of requiring two periods of land use data when calculating the transition probability matrix. We also construct a suitability atlas of land use as the transition rules into the CA-LTOM model, taking into account the regional natural and socioeconomic driver factors, by exploiting the ANN model. The ANN-CA-LTOM model is employed to simulate the distribution of the three major types of land use, i.e., construction land, agricultural land, and unused land, in the Nansha District, China, in 2018 and 2020. The results show that the model performs well and the overall accuracy of the land use simulation was 97.72%, with a kappa coefficient of 0.962761. Furthermore, the simulated and predicted results of land use changes from 2021 to 2023 in Nansha District show changing trends in construction, agricultural, and unused land use. This study provides an approach for estimating a Markov transition probability matrix and a coupled mode of the models for simulating and predicting land use changes.

Research on the co-evolution of competitive public opinion and intervention strategy based on Markov process

Under Omni-media environment, Online Social Networks (OSN) have gradually become the most momentous platform for information propagation. Considering the interaction and coexistence of both positive and negative public opinion information (referred to as public opinion), it is of great significance for social development and economic stability to understand the co-evolution process of competitive public opinion and compress the spreading space of negative public opinion. Allowing for this point, this paper constructed a two-stage spreading model of competitive public opinion combing with the actual case of public opinion propagation, analysed the main factors influencing the co-evolution process, such as netizens’ intimacy, network literacy, and so on, and redefined netizens’ state transition probability matrix with the help of Markov process. Then, the effectiveness of the spreading model was verified and the propagation rule of public opinion was discussed in open and closed OSN through simulation experiments. Finally, the intervention strategies were proposed and optimised with the limitation of cost. The results show that the propagation of public opinion mainly depends on netizens’ behaviour with low literacy and presents difference characteristics in two types of OSN. During the intervention process of public opinion propagation, there exists an effective intervention interval and the best intervention strategy varies with the change of network topology. Our research provided a cornerstone for further understanding of the co-evolution process of competitive public opinion and the research conclusions also provided a certain decision-making reference for enterprises, governments and other regulators to reasonably respond to the propagation of public opinion.

The impact of customer clubs on lifetime value of banking customers

Today, with the abundance of private banks, state-owned banks and financial and credit institutions, banks are trying to satisfy customers more than ever in order to be preferred by them. In this regard, they use customer clubs as a communication network to improve their up-selling, cross-selling and word of mouth marketing. Therefore, this study aims to investigate the short-term impact of customer clubs on the lifetime value of customers as one of the most important indicators that reflects customer satisfaction, loyalty and thereby financial gain. To this end, the RFM model was used for clustering customers based on their behaviours. Then customers' lifetime value for each cluster was calculated in four periods. The state transition probability matrix of a Markov chain was also used to analyse the customers' transitions and change in their values through the seasons. The results indicated that customers' lifetime value and the number of profitable customers increased in the last period in which the customer club was used.

Reliability evaluation method of PMFSM system based on hidden Markov

To solve the complex calculation in the reliability evaluation of the motor system with the Markov model, this study developed a reliability evaluation method based on the hidden Markov model (HMM). Next, the developed method was employed in a type of permanent magnet hybrid segmented flux-switching machine (PMFSM). By analyzing and simplifying the components of the PMFSM system, the reliability measurement standard and the failure modes of the respective component were examined, and the relationships between the failure modes of each component and the system parameters were built. The optimal parameter observation sequence was determined by calculating the Minkowski distance, and the HMM was iteratively calculated by using the Baum–Welch algorithm. Moreover, the state transition probability matrix of the system was built, and the system reliability was evaluated. As revealed from the calculation results, the reliability of the PMFSM system is significantly higher than that of the conventional structure at the same time, and the mean time to failure (MTTF) of the PMFSM system is 14.8% higher than that of the conventional structure. Compared with the conventional evaluation method based on the Markov model, the two methods have similar reliability results for conventional and novel PMFSM systems, and the variation trend is the same. It shows that the proposed reliability evaluation method is feasible and effective, and the calculation is simple. The work in this study provides a basis for the design of the novel PMFSM system, and a practical engineering method to efficiently evaluate the reliability of other motor systems.

Optimal Path Planning of Autonomous Marine Vehicles in Stochastic Dynamic Ocean Flows Using a GPU-Accelerated Algorithm

Autonomous marine vehicles play an essential role in many ocean science and engineering applications. Planning time and energy optimal paths for these vehicles to navigate in stochastic dynamic ocean environments are essential to reduce operational costs. In some missions, they must also harvest solar, wind or wave energy (modeled as a stochastic scalar field) and move in optimal paths that minimize net energy consumption. Markov decision processes (MDPs) provide a natural framework for sequential decision making for robotic agents in such environments. However, building a realistic model and solving the modeled MDP becomes computationally expensive in large-scale real-time applications, warranting the need of parallel algorithms and efficient implementation. In this article, we introduce an efficient end-to-end graphical processing unit (GPU)-accelerated algorithm that 1) builds the MDP model (computing transition probabilities and expected one-step rewards) and 2) solves the MDP to compute an optimal policy. We develop methodical and algorithmic solutions to overcome the limited global memory of GPUs by 1) using a dynamic reduced-order representation of the ocean flows; 2) leveraging the sparse nature of the state transition probability matrix; 3) introducing a neighboring subgrid concept; and 4) proving that it is sufficient to use only the stochastic scalar field’s mean to compute the expected one-step rewards for missions involving energy harvesting from the environment, thereby saving memory and reducing the computational effort. We demonstrate the algorithm on a simulated stochastic dynamic environment and highlight that it builds the MDP model and computes the optimal policy 600–1000 times faster than conventional CPU implementations, making it suitable for real-time use.

Research on future parallel power grid architecture

Abstract This paper introduces the six distribution levels of physical object layer, object perception layer, storage layer, operation layer, analysis layer and parallel control layer of the future parallel power grid architecture. On this basis, a more targeted analysis model is proposed. Starting from the resource operation level of parallel power grid, the abstract model method is used to analyse the changes of the architecture of resource nodes in different virtual agent sets. The detailed changes of the resource system are described through multi-layer nodes, and the random matrix is used to systematically analyse the operation data of the parallel power grid in the future so as to obtain the architecture mode with the highest adaptability when the actual space and virtual space are associated. Considering that the virtual community in the future parallel power grid is the core part of the architecture, the guidance information received by the virtual community in the power grid is regarded as a three-dimensional state transition probability matrix, which captures the real-time changes of the virtual community. On this basis, taking the future parallel power grid macro-economy as the development prospect, the supply and demand balance conditions of the future power grid are obtained to help the power grid achieve strong development. The experimental results show that the parallel power grid architecture optimised by this method is more in line with the actual development needs; the resource utilisation rate and operation income have a positive growth trend and finally achieve an impressive result of approximately 100%, which is of great help to the development of parallel power grid in the future.

An M-Estimation-Based Improved Interacting Multiple Model for INS/DVL Navigation Method

This paper presents an M-estimation-based Improved Interacting Multiple Model (IIMM) algorithm for inertial navigation system (INS)/Doppler velocity logs (D VL) integrated method to directly apply in complex underwater environment without empirical parameters. Considering the time-varying measurement noise covariance, a variable model structure based on Bayesian estimation law is proposed to adapt to the environment and an adaptive state transition probability matrix is generated to promote the model adaption. Meanwhile, M-estimation-based filter is employed as primary model to save IIMM from the effect of outliers. The performance of the IIMM algorithm is evaluated on simulation, land vehicle, and Yangtze River test, where the state-of-the-art is compared adequately. The robustness simulation test is also performed. It is highlighted that IIMM approach can adapt to actual model rapidly and can resist outliers efficiently. The proposed IIMM method can improve the accuracy and robustness of the navigation system in a severe and unknown environment without increasing the computational load.

Steady-State Availability Evaluation for Heterogeneous Edge Computing-Enabled WSNs with Malware Infections

To evaluate the steady-state availability of heterogeneous edge computing-enabled wireless sensor networks (HECWSNs) with malware infections, we first propose a Stackelberg attack-defence game to predict the optimal strategies of malware and intrusion detection systems (IDSs) deployed in heterogeneous sensor nodes (HSNs). Next, we present a new malware infection model—heterogeneous susceptible-threatened-active-recovered-dead (HSTARD) based on epidemic theory. Then, considering the heterogeneity of sink sensor nodes and common sensor nodes and the malware attack correlation, we derive the state transition probability matrix of an HSN based on a semi-Markov process (SMP), as well as the steady-state availability of an HSN. Furthermore, based on a data flow analysis of HSNs, we deduce the steady-state availability of HECWSNs with various topologies, including the star topology, cluster topology, and mesh topology. Finally, numerical analyses illustrate the influence of the IDS parameters on the optimal infection probability of malware and reveal the effect of multiple factors on the steady-state availability of HSNs, including the initial infection rate, the infection change rate, and the malware attack correlation. In addition, we present data analyses of the steady-state availability of HECWSNs with various topologies, including the star topology, cluster topology, and mesh topology, which provide a theoretical basis for the design, deployment, and maintenance of high-availability HECWSNs.

Point-kinetics neutron noise modeling and analysis via probabilistic finite state automata

This paper addresses two issues for safety & performance analysis of nuclear reactors; these issues are: (i) Modeling of the neutron noise process, which is constructed as stochastic differential equations by augmenting deterministic dynamic models of point kinetics, thermal hydraulics, and mechanical vibrations in the reactor core, and (ii) Time-series analysis of nuclear reactor internals for early-stage anomaly detection by making use of the neutron noise model. The model is used to generate an ensemble of time series of neutron noise under both normal and anomalous operating conditions, where an anomaly in the postulated probability distribution of neutron noise is caused by abnormal vibration of the fuel assembly. A symbolic time series analysis (STSA) tool, called probabilisitc finite state automata (PFSA), has been selected to detect the anomalies, if any. The PFSA method symbolizes a time-series and then encodes the resulting symbol string into a state transition probability matrix, which is both stochastic and ergodic by construction. The discrimination function for anomaly detection is built upon the residual error of the principal (left) eigenvector of the state transition probability matrix, which is also the state probability vector of the PFSA. The accuracy of the proposed analysis and its capability for online anomaly detection is demonstrated by simulation on a low-order dynamic model of a 2,500 MWt PWR of the Three-Mile-Island type.

Study on the preventive maintenance of the cabin structure based on the markov chain

In view of the complex structure and the different damage situations of the cabin, the Markov state transition model was established considering the crack propagation at the damaged sites. The state transition probability matrix of related crack propagation was constructed using the equation of linear elastic facture mechanics to simulate the deterioration process of the main structural parts during navigation. Secondly, the maintenance matrix was added to the state transition model, and the distribution probability of the system state was predicted by the periodicity of Markov chain, formulating a reasonable maintenance strategy. Finally, the practicability of the preventive maintenance strategy was verified taking the marine oil FPSO 116 as the example. The maintenance plan formulated according to the calculation results could reduce the damage of the hull theoretically, preventing the occurrence of unexpected deterioration states, prolonging the service life of the hull, and providing a strategic reference for the formulation of the maintenance plans of the hull.

A Predictive Energy Management Strategy for Multi-Energy Source Vehicles Based on Full-Factor Trip Information.

To achieve the real-time application of a dynamic programming (DP) control strategy, we propose a predictive energy management strategy (PEMS) based on full-factor trip information, including vehicle speed, slip ratio and slope. Firstly, the prediction model of the full-factor trip information is proposed, which provides an information basis for global optimization energy management. To improve the prediction’s accuracy, the vehicle speed is predicted based on the state transition probability matrix generated in the same driving scene. The characteristic parameters are extracted by a feature selection method taken as the basis for the driving condition’s identification. Similar to speed prediction, regarding the uncertain route at an intersection, the slope prediction is modelled as a Markov model. On the basis of the predicted speed and the identified maximum adhesion coefficient, the slip ratio is predicted based on a neural network. Then, a predictive energy management strategy is developed based on the predictive full-factor trip information. According to the statistical rules of DP results under multiple standard driving cycles, the reference SOC trajectory is generated to ensure global sub-optimality, which determines the feasible state domain at each prediction horizon. Simulations are performed under different types of driving conditions (Urban Dynamometer Driving Schedule, UDDS and World Light Vehicle Test Cycle, WLTC) to verify the effectiveness of the proposed strategy.

Ergodicity of the probabilistic converter, a serial connection of two automata

Abstract The paper provides necessary and sufficient conditions for the the ergodicity of a serial connection of automata under which the output sequence of a substitution Mealy automaton is fed to the input of an output-free substitution automaton. It is shown that the condition of complete indecomposability of the state transition probability matrix of a Mealy automaton provides a sufficient condition for ergodicity of the probabilistic converter as a serial connection of automata. It is also shown that if the partial state transition functions of a Mealy automaton commute, then the condition of ergodicity of a serial connection is equivalent to that of both original probabilistic converters.

Variational Beta Process Hidden Markov Models with Shared Hidden States for Trajectory Recognition.

Hidden Markov model (HMM) is a vital model for trajectory recognition. As the number of hidden states in HMM is important and hard to be determined, many nonparametric methods like hierarchical Dirichlet process HMMs and Beta process HMMs (BP-HMMs) have been proposed to determine it automatically. Among these methods, the sampled BP-HMM models the shared information among different classes, which has been proved to be effective in several trajectory recognition scenes. However, the existing BP-HMM maintains a state transition probability matrix for each trajectory, which is inconvenient for classification. Furthermore, the approximate inference of the BP-HMM is based on sampling methods, which usually takes a long time to converge. To develop an efficient nonparametric sequential model that can capture cross-class shared information for trajectory recognition, we propose a novel variational BP-HMM model, in which the hidden states can be shared among different classes and each class chooses its own hidden states and maintains a unified transition probability matrix. In addition, we derive a variational inference method for the proposed model, which is more efficient than sampling-based methods. Experimental results on a synthetic dataset and two real-world datasets show that compared with the sampled BP-HMM and other related models, the variational BP-HMM has better performance in trajectory recognition.