State Transition Probability Matrix Research Articles

A novel operational reliability and state prediction method based on degradation hidden Markov model with random threshold

AbstractBearings are widely used as a common mechanical component and significantly impacts the reliability and safety of various engineering equipment. In the field of large‐scale equipment such as wind turbines, statistical data for reliability and state prediction of bearings is usually limited, leading to the inapplicability of traditional statistical methods. Besides, individual differences commonly exist among bearings, and the appropriate failure threshold for a specific bearing is difficult to be determined due to the individual uncertainty, which may induce prediction errors. To address these issues, a novel operational reliability and state prediction method based on random threshold degradation hidden Markov model was proposed in the study. The traditional hidden Markov model was improved by considering the effect of performance degradation on state transition probability matrix. Moreover, random failure thresholds of performance degradation were employed to describe individual differences between bearings. The operational reliability and state of bearings was obtained by comprehensive analysis of prediction results under different failure thresholds. Verification of the proposed operational reliability and state prediction method were conducted using PHM2012 bearing operation data. The results underscored the effectiveness of the proposed method in achieving a comparatively high accuracy in operational reliability prediction without the prerequisite of an exact failure threshold, when contrasted with several established reliability and life prediction techniques.

A Distributed Non-Intrusive Load Monitoring Method Using Karhunen–Loeve Feature Extraction and an Improved Deep Dictionary

In recent years, the non-invasive load monitoring (NILM) method based on sparse coding has shown promising research prospects. This type of method learns a sparse dictionary for each monitoring target device, and it expresses load decomposition as a problem of signal reconstruction using dictionaries and sparse vectors. The existing NILM methods based on sparse coding have problems such as inability to be applied to multi-state and time-varying devices, single-load characteristics, and poor recognition ability for similar devices in distributed manners. Using the analysis above, this paper focuses on devices with similar features in households and proposes a distributed non-invasive load monitoring method using Karhunen–Loeve (KL) feature extraction and an improved deep dictionary. Firstly, Karhunen–Loeve expansion (KLE) is used to perform subspace expansion on the power waveform of the target device, and a new load feature is extracted by combining singular value decomposition (SVD) dimensionality reduction. Afterwards, the states of all the target devices are modeled as super states, and an improved deep dictionary based on the distance separability measure function (DSM-DDL) is learned for each super state. Among them, the state transition probability matrix and observation probability matrix in the hidden Markov model (HMM) are introduced as the basis for selecting the dictionary order during load decomposition. The KL feature matrix of power observation values and improved depth dictionary are used to discriminate the current super state based on the minimum reconstruction error criterion. The test results based on the UK-DALE dataset show that the KL feature matrix can effectively reduce the load similarity of devices. Combined with DSM-DDL, KL has a certain information acquisition ability and acceptable computational complexity, which can effectively improve the load decomposition accuracy of similar devices, quickly and accurately estimating the working status and power demand of household appliances.

Comparison of Fuzzy Grey Markov Model (1,1) and Fuzzy Grey Markov Model (2,1) in Forecasting Gold Prices in Indonesia

Currently, gold investment is considered promising despite the ever-changing price of gold. However, obtaining optimal profits is a challenge for investors. Therefore, a proper forecasting method is needed to forecast the gold price so investors can know the best transaction time. This study used two forecasting methods: the Fuzzy Grey Markov Model (1,1) and a new, never-before-used approach, the Fuzzy Grey Markov Model (2,1). The Fuzzy Grey Markov Model (2,1) approach is interesting because it can be considered for forecast data that shows varying increases and decreases, such as the gold price data used in this study. Both methods are combined models that utilize fuzzy logic to handle uncertainty in data; the Grey model forms a forecasting model, and the Markov chain determines the state transition probability matrix. Next, the error rates of the two methods are compared based on the Mean Absolute Percentage Error (MAPE) value to obtain the best forecasting method. As a result of this study, the Fuzzy Grey Markov Model (1,1) was chosen as the best forecasting method with a MAPE value of 0.28%.

Research on hydrological survey and forecasting model based on rolling sampling Markov chain

Abstract Hydrological survey is of great significance in flood disaster warning, hydropower engineering construction, and efficient utilization of watershed resources. Due to the seasonal and random fluctuations of hydraulic resources, it is difficult to accurately predict waterway runoff. Based on this, this article proposes a hydrological survey and prediction method based on rolling sampling Markov chain. Firstly, based on historical hydrological and meteorological data, establish a watershed runoff model and summarize and analyze the characteristics of the patterns; Then, based on the first-order Markov chain model, the relationship between adjacent days is considered. According to seasonal and meteorological factors, the historical data is rolling sampled in a 10 day sampling interval, and a multi state transition probability matrix is established to construct a year-round time-series runoff forecasting model; Finally, taking the Jinsha River Basin as an example, the annual runoff situation of the waterway was simulated. The calculation results have verified the effectiveness of the proposed method, indicating that it can simulate runoff well and provide certain guidance for watershed management and construction.

A Reliability Assessment Method for Complex Systems Based on Non-Homogeneous Markov Processes.

The Markov method is a common reliability assessment method. It is often used to describe the dynamic characteristics of a system, such as its repairability, fault sequence and multiple degradation states. However, the "curse of dimensionality", which refers to the exponential growth of the system state space with the increase in system complexity, presents a challenge to reliability assessments for complex systems based on the Markov method. In response to this challenge, a novel reliability assessment method for complex systems based on non-homogeneous Markov processes is proposed. This method entails the decomposition of a complex system into multilevel subsystems, each with a relatively small state space, in accordance with the system function. The homogeneous Markov model or the non-homogeneous Markov model is established for each subsystem/system from bottom to top. In order to utilize the outcomes of the lower-level subsystem models as inputs to the upper-level subsystem model, an algorithm is proposed for converting the unavailability curve of a subsystem into its corresponding 2×2 dynamic state transition probability matrix (STPM). The STPM is then employed as an input to the upper-level system's non-homogeneous Markov model. A case study is presented using the reliability assessment of the Reactor Protection System (RPS) based on the proposed method, which is then compared with the models based on the other two contrast methods. This comparison verifies the effectiveness and accuracy of the proposed method.

Construction of Cybersecurity and Risk Prediction Model for New Energy Power Plants under Machine Learning Algorithm

Abstract Addressing the complex equipment and network challenges in new energy power plant industrial control systems, this study introduces a Markov time-varying machine learning algorithm, leveraging classification-constrained Boltzmann machines for real-time network security risk prediction. By employing a hybrid training mode for innovative feature extraction and classification, the algorithm forecasts future security risk states through an up-to-date state transition probability matrix. Integrating with the Markov time-varying model enhances the efficiency over traditional Boltzmann machines, facilitating nuanced network state analyses. The proposed model demonstrates high effectiveness against various network attacks, with average precision, recall, and F1 scores of 0.96, 0.93, and 0.94, respectively, and maintains over 80% accuracy under noise levels up to 40 dB. This research provides a solid foundation for proactive security defense mechanisms in industrial control systems.

Finite-state Markov modeling of Gamma-Gamma fading FSO links with pointing errors

The knowledge of atmospheric channel is quite imperative for free-space optical (FSO) communications. To capture the time-varying essence of intensity fluctuations in an FSO connection, we propose a finite-state Markov (FSM) model for FSO links affected by Gamma-Gamma distributed atmospheric turbulence and pointing errors. First, the level crossing rate (LCR) of the unified intensity fluctuation is derived considering the non-generic channel autocorrelation characteristics, based on which the state transition probability (STP) matrix of the FSM model is calculated. Then, numerical simulations are presented for different strength parameters and frequency parameters of the channel. The results show that the misalignment strength does not affect the ratio of the level where the LCR reaches its maximum to the mean level. Finally, a series of well-designed experiments are carried out to verify the theory. As a result, the theoretical predictions conform to the experimental data, with averaged STP relative errors below 6.7%.

APPLICATION OF EXPECTED CREDIT LOSS MODEL AND MARKOV CHAIN TO CALCULATE NET SINGLE PREMIUM OF UNSECURED CREDIT INSURANCE

Transferring credit risk to an insurance company is a way to mitigate risk. Premiums should be calculated accurately to attain economic value for both the lender and the guarantor. The aim of this study was to determine the net single premium (NSP) values for an unsecured credit insurance product using the expected credit loss (ECL) method from IFRS 9. This study used data generated through simulation of insurance policies issued in 2015 or 2016. Their state classifications were monthly observed from 2016 to 2020. The probability of disbursed claim (PDC) parameter replaced the probability of default parameter on the ECL model, whereas the PDC model was constructed based on the components of a state-transition probability matrix, obtained with the Markov chain approach using the cohort method: = 0.999181, = 0.000130, and = 0.000689. The PDC model validation showed relatively decent results, whereas MSE = 2.457% and zs = 0.608 with a = 5%. These results indicated that the PDC model was a good fit to calculate ECL. 5,000 iterations were done as part of the cash flow simulation process, whereas debtors’ loan amounts were randomly generated during each iteration, and the average NPV of these iterations was -Rp564.419.305. Based on model sensitivity analysis, cash flow values were most sensitive to the variable used to construct the PDC model (). Thus, the 5,000-iteration process was repeated with the newly adjusted PDC value, which were = 0.998924 and = 0.000946. The new average NPV of these iterations was Rp409,877,840, indicating that the constructed ECL model was a good fit to calculate NSP values for unsecured credit insurance products.

Optimal online energy management strategy of a fuel cell hybrid bus via reinforcement learning

An energy management strategy (EMS) based on reinforcement learning is proposed in this study to enhance the fuel economy and durability of a fuel cell hybrid bus (FCHB). Firstly, a comprehensive powertrain system model for the FCHB is established, mainly including the FCHB’s power balance, fuel cell system (FCS) efficiency, and aging models. Secondly, the state–action space, state transition probability matrix (TPM), and multi-objective reward function of Q-learning algorithm are designed to improve the fuel economy and the durability of power sources. The FCHB’s demand power and battery state of charge (SOC) serve as the state variables and the FCS output power is used as the action variable. Using the demonstration FCHB data, a state TPM is created to represent the overall operation. Finally, an EMS employing Q-learning is formulated to optimize the fuel economy of FCHB, maintain battery SOC, suppress FCS power fluctuations, and enhance FCS lifetime. The proposed EMS is tested and verified through hardware-in-the-loop (HIL) tests. The simulation results demonstrate the effectiveness of the proposed strategy. Compared to a rule-based EMS, the Q-learning-based EMS can improve the energy economy by 7.8%. Furthermore, it is only a 3.7% difference to the best energy economy under dynamic optimization, while effectively reducing the decline and enhancing the durability of the FCS.

A Quantitative Framework for Propagation Paths of Natech Domino Effects in Chemical Industrial Parks: Part I—Failure Analysis

Along with global climate change and industrialization, domino effects caused by Natech events occurred frequently in chemical industrial parks over the past decades. Previous research has not yet proposed a reliable method to obtain all possible paths of Natech domino effects, and moreover, a risk assessment and mitigation system has not been established. The present work aims to develop a quantitative framework for propagation paths of Natech domino effects, which can effectively safeguard the sustainable development of chemical industrial parks. The presentation of this work is divided into two parts: Part I (current paper) proposes a path probability calculation method that can simultaneously consider multiple primary accident scenarios and multi-level domino effects triggered by natural disasters. The proposed method transforms the propagation paths of domino effects into the paths of directed graph by constructing the equipment failure state transition matrix and the equipment failure state transition probability matrix. The depth-first traversal algorithm is used to obtain all possible propagation paths and their propagation probabilities, providing data support for the quantitative risk assessment and prevention and control measures presented in the accompanying paper (Part II). The case study shows that the probability of equipment failure caused by multi-level domino effects triggered by Natech accidents is higher than that of conventional accidents. However, the present work only considers the spatial propagation of domino effects, while their spatio-temporal propagation remains as a further direction for this area of inquiry.

Driving cycle prediction based on Markov chain combined with driving information mining

Driving cycle prediction (DCP) is of great importance for vehicle’s awareness of surrounding environment and optimization of control strategy. It has won much attention around the world. However, the used driving database could not reflecting the great diversity in multi driving cycles in real world. The characteristic parameters could not represent the complex and diverse driving conditions. Therefore, DCP models’ adaptability and generalization ability are limited. Moreover, prediction methods in existing researches always take large computation burden, difficult to apply in practice. To tackle these issues, the present research focus on DCP based on Markov chain combined with driving information mining (DIM). Firstly, many standard test cycles and real cycles are collected to construct an database of great diversity. Secondly, DIM technologies are studied to extract the optimal parameter set reflecting driving characteristics and determine the driving cycle categories. Thirdly, prediction method based on recursive self-learning Markov chain is proposed. Recursive equation of Markov state transition probability matrix (TPM) is deduced to save computation time. The key parameter of prediction method is adjusted to achieve optimal prediction performance. Synthesized standard test driving cycles (SSTDC) and real driving cycles (RDC) in database, as well as a new real driving cycle (NRDC) are tested. Driving pattern prediction accuracy of the proposed method is 99.25% for SSTDC, 100.00% for RDC, and 99.84% for NRDC, respectively. Compared with the benchmark method, the prediction accuracy is improved by 18.87% for SSTDC, 10.07% for RDC and 23.51% for NRDC, respectively. In addition, processing time of the prediction program is significantly reduced by 2.23 s (36.14%) for SSTDC, 12.30 s (51.25%) for RDC and 3.99 s (52.16%) for NRDC, respectively. It can be verified that the proposed method has high precision, short response time, as well as good adaptability and charming generalization ability for new cycles.

Assessment of Vigilance Level during Work: Fitting a Hidden Markov Model to Heart Rate Variability

This study aimed to enhance the real-time performance and accuracy of vigilance assessment by developing a hidden Markov model (HMM). Electrocardiogram (ECG) signals were collected and processed to remove noise and baseline drift. A group of 20 volunteers participated in the study. Their heart rate variability (HRV) was measured to train parameters of the modified hidden Markov model for a vigilance assessment. The data were collected to train the model using the Baum-Welch algorithm and to obtain the state transition probability matrix A^ and the observation probability matrix B^. Finally, the data of three volunteers with different transition patterns of mental state were selected randomly and the Viterbi algorithm was used to find the optimal state, which was compared with the actual state. The constructed vigilance assessment model had a high accuracy rate, and the accuracy rate of data prediction for these three volunteers exceeded 80%. Our approach can be used in wearable products to improve their vigilance level assessment functionality or in other fields that have key positions with high concentration requirements and monotonous repetitive work.

A Loosely Coupled Model for Simulating and Predicting Land Use Changes

The analysis and modeling of spatial and temporal changes in land use can reveal changing urban spatial patterns and trends. In this paper, we introduce a linear transformation optimization Markov (LTOM) model that can be exploited to estimate the state transition probability matrix of land use, building a loosely coupled ANN-CA-LTOM model for simulating and predicting land use changes. The advantages of this model are that it is flexible and high expansibility; it can maintain semantic coupling between the Artificial Neural Networks (ANN), Cellular Automata (CA), and LTOM model and enhance their functions; and it can break the limitation of requiring two periods of land use data when calculating the transition probability matrix. We also construct a suitability atlas of land use as the transition rules into the CA-LTOM model, taking into account the regional natural and socioeconomic driver factors, by exploiting the ANN model. The ANN-CA-LTOM model is employed to simulate the distribution of the three major types of land use, i.e., construction land, agricultural land, and unused land, in the Nansha District, China, in 2018 and 2020. The results show that the model performs well and the overall accuracy of the land use simulation was 97.72%, with a kappa coefficient of 0.962761. Furthermore, the simulated and predicted results of land use changes from 2021 to 2023 in Nansha District show changing trends in construction, agricultural, and unused land use. This study provides an approach for estimating a Markov transition probability matrix and a coupled mode of the models for simulating and predicting land use changes.

Research on the co-evolution of competitive public opinion and intervention strategy based on Markov process

Under Omni-media environment, Online Social Networks (OSN) have gradually become the most momentous platform for information propagation. Considering the interaction and coexistence of both positive and negative public opinion information (referred to as public opinion), it is of great significance for social development and economic stability to understand the co-evolution process of competitive public opinion and compress the spreading space of negative public opinion. Allowing for this point, this paper constructed a two-stage spreading model of competitive public opinion combing with the actual case of public opinion propagation, analysed the main factors influencing the co-evolution process, such as netizens’ intimacy, network literacy, and so on, and redefined netizens’ state transition probability matrix with the help of Markov process. Then, the effectiveness of the spreading model was verified and the propagation rule of public opinion was discussed in open and closed OSN through simulation experiments. Finally, the intervention strategies were proposed and optimised with the limitation of cost. The results show that the propagation of public opinion mainly depends on netizens’ behaviour with low literacy and presents difference characteristics in two types of OSN. During the intervention process of public opinion propagation, there exists an effective intervention interval and the best intervention strategy varies with the change of network topology. Our research provided a cornerstone for further understanding of the co-evolution process of competitive public opinion and the research conclusions also provided a certain decision-making reference for enterprises, governments and other regulators to reasonably respond to the propagation of public opinion.

The impact of customer clubs on lifetime value of banking customers

Today, with the abundance of private banks, state-owned banks and financial and credit institutions, banks are trying to satisfy customers more than ever in order to be preferred by them. In this regard, they use customer clubs as a communication network to improve their up-selling, cross-selling and word of mouth marketing. Therefore, this study aims to investigate the short-term impact of customer clubs on the lifetime value of customers as one of the most important indicators that reflects customer satisfaction, loyalty and thereby financial gain. To this end, the RFM model was used for clustering customers based on their behaviours. Then customers' lifetime value for each cluster was calculated in four periods. The state transition probability matrix of a Markov chain was also used to analyse the customers' transitions and change in their values through the seasons. The results indicated that customers' lifetime value and the number of profitable customers increased in the last period in which the customer club was used.

Reliability evaluation method of PMFSM system based on hidden Markov

To solve the complex calculation in the reliability evaluation of the motor system with the Markov model, this study developed a reliability evaluation method based on the hidden Markov model (HMM). Next, the developed method was employed in a type of permanent magnet hybrid segmented flux-switching machine (PMFSM). By analyzing and simplifying the components of the PMFSM system, the reliability measurement standard and the failure modes of the respective component were examined, and the relationships between the failure modes of each component and the system parameters were built. The optimal parameter observation sequence was determined by calculating the Minkowski distance, and the HMM was iteratively calculated by using the Baum–Welch algorithm. Moreover, the state transition probability matrix of the system was built, and the system reliability was evaluated. As revealed from the calculation results, the reliability of the PMFSM system is significantly higher than that of the conventional structure at the same time, and the mean time to failure (MTTF) of the PMFSM system is 14.8% higher than that of the conventional structure. Compared with the conventional evaluation method based on the Markov model, the two methods have similar reliability results for conventional and novel PMFSM systems, and the variation trend is the same. It shows that the proposed reliability evaluation method is feasible and effective, and the calculation is simple. The work in this study provides a basis for the design of the novel PMFSM system, and a practical engineering method to efficiently evaluate the reliability of other motor systems.

Optimal Path Planning of Autonomous Marine Vehicles in Stochastic Dynamic Ocean Flows Using a GPU-Accelerated Algorithm

Autonomous marine vehicles play an essential role in many ocean science and engineering applications. Planning time and energy optimal paths for these vehicles to navigate in stochastic dynamic ocean environments are essential to reduce operational costs. In some missions, they must also harvest solar, wind or wave energy (modeled as a stochastic scalar field) and move in optimal paths that minimize net energy consumption. Markov decision processes (MDPs) provide a natural framework for sequential decision making for robotic agents in such environments. However, building a realistic model and solving the modeled MDP becomes computationally expensive in large-scale real-time applications, warranting the need of parallel algorithms and efficient implementation. In this article, we introduce an efficient end-to-end graphical processing unit (GPU)-accelerated algorithm that 1) builds the MDP model (computing transition probabilities and expected one-step rewards) and 2) solves the MDP to compute an optimal policy. We develop methodical and algorithmic solutions to overcome the limited global memory of GPUs by 1) using a dynamic reduced-order representation of the ocean flows; 2) leveraging the sparse nature of the state transition probability matrix; 3) introducing a neighboring subgrid concept; and 4) proving that it is sufficient to use only the stochastic scalar field’s mean to compute the expected one-step rewards for missions involving energy harvesting from the environment, thereby saving memory and reducing the computational effort. We demonstrate the algorithm on a simulated stochastic dynamic environment and highlight that it builds the MDP model and computes the optimal policy 600–1000 times faster than conventional CPU implementations, making it suitable for real-time use.

Research on future parallel power grid architecture

Abstract This paper introduces the six distribution levels of physical object layer, object perception layer, storage layer, operation layer, analysis layer and parallel control layer of the future parallel power grid architecture. On this basis, a more targeted analysis model is proposed. Starting from the resource operation level of parallel power grid, the abstract model method is used to analyse the changes of the architecture of resource nodes in different virtual agent sets. The detailed changes of the resource system are described through multi-layer nodes, and the random matrix is used to systematically analyse the operation data of the parallel power grid in the future so as to obtain the architecture mode with the highest adaptability when the actual space and virtual space are associated. Considering that the virtual community in the future parallel power grid is the core part of the architecture, the guidance information received by the virtual community in the power grid is regarded as a three-dimensional state transition probability matrix, which captures the real-time changes of the virtual community. On this basis, taking the future parallel power grid macro-economy as the development prospect, the supply and demand balance conditions of the future power grid are obtained to help the power grid achieve strong development. The experimental results show that the parallel power grid architecture optimised by this method is more in line with the actual development needs; the resource utilisation rate and operation income have a positive growth trend and finally achieve an impressive result of approximately 100%, which is of great help to the development of parallel power grid in the future.

An M-Estimation-Based Improved Interacting Multiple Model for INS/DVL Navigation Method

This paper presents an M-estimation-based Improved Interacting Multiple Model (IIMM) algorithm for inertial navigation system (INS)/Doppler velocity logs (D VL) integrated method to directly apply in complex underwater environment without empirical parameters. Considering the time-varying measurement noise covariance, a variable model structure based on Bayesian estimation law is proposed to adapt to the environment and an adaptive state transition probability matrix is generated to promote the model adaption. Meanwhile, M-estimation-based filter is employed as primary model to save IIMM from the effect of outliers. The performance of the IIMM algorithm is evaluated on simulation, land vehicle, and Yangtze River test, where the state-of-the-art is compared adequately. The robustness simulation test is also performed. It is highlighted that IIMM approach can adapt to actual model rapidly and can resist outliers efficiently. The proposed IIMM method can improve the accuracy and robustness of the navigation system in a severe and unknown environment without increasing the computational load.

Steady-State Availability Evaluation for Heterogeneous Edge Computing-Enabled WSNs with Malware Infections

To evaluate the steady-state availability of heterogeneous edge computing-enabled wireless sensor networks (HECWSNs) with malware infections, we first propose a Stackelberg attack-defence game to predict the optimal strategies of malware and intrusion detection systems (IDSs) deployed in heterogeneous sensor nodes (HSNs). Next, we present a new malware infection model—heterogeneous susceptible-threatened-active-recovered-dead (HSTARD) based on epidemic theory. Then, considering the heterogeneity of sink sensor nodes and common sensor nodes and the malware attack correlation, we derive the state transition probability matrix of an HSN based on a semi-Markov process (SMP), as well as the steady-state availability of an HSN. Furthermore, based on a data flow analysis of HSNs, we deduce the steady-state availability of HECWSNs with various topologies, including the star topology, cluster topology, and mesh topology. Finally, numerical analyses illustrate the influence of the IDS parameters on the optimal infection probability of malware and reveal the effect of multiple factors on the steady-state availability of HSNs, including the initial infection rate, the infection change rate, and the malware attack correlation. In addition, we present data analyses of the steady-state availability of HECWSNs with various topologies, including the star topology, cluster topology, and mesh topology, which provide a theoretical basis for the design, deployment, and maintenance of high-availability HECWSNs.