Small Probability Events Research Articles

An Accelerated Simulation Approach for Multistate System Mission Reliability and Success Probability under Complex Mission

The mission reliability and success probability estimation of multistate systems under complex mission conditions are studied. The reliability and success probability of multistate phased mission systems (MS-PMS) is difficult to use analytic modeling and solving. An estimation approach for mission reliability and success probability based on Monte Carlo simulation is established. By introducing accelerated sampling methods such as forced transition and failure biasing, the sampling efficiency of small-probability events is improved while ensuring unbiasedness. The ship’s propulsion and power systems are used as applications, and the effectiveness of the method is verified by a numerical example. Under complex missions, such as missions with different mission time and their combinations, and phased-missions, the proposed method is superior in small-probability event sampling than the crude simulation method. The calculation example also studies the influence of mission factors or system reliability and maintainability factors on system availability and mission success probability, and analyzes the relationship between different mission types and system availability and success probability.

Identification of multi-element geochemical anomalies using unsupervised machine learning algorithms: A case study from Ag–Pb–Zn deposits in north-western Zhejiang, China

Identifying significant geochemical anomalies is a fundamental task in mineral exploration. In terms of stochastic process, a specific ore-forming process can be regarded as a small probability event since the deposit type sought rarely occurs in the crust of the earth. Therefore, machine learning algorithms for anomaly detection are suitable for identification of mineralization-related geochemical anomalies. However, identification of significant geochemical anomaly signatures is often hindered by irrelevant and/or redundant variables in a geochemical dataset. It is then necessary to extract significant features by means of dimensional reduction before applying unsupervised machine learning algorithms for anomaly detection. In this study, six unsupervised machine learning methods including isolation forest (IF) and five hybrid models that combined IF with dimensional reduction techniques were implemented and evaluated in identifying Ag–Pb–Zn polymetallic mineralization-related geochemical anomalies using stream sediment geochemical data from north-western Zhejiang Province, China. The five hybrid models involved combining IF with hierarchical clustering (HC-IF), principal component analysis (PCA-IF), stacked denoising autoencoder (SDAE-IF), both HC and PCA (HC-PCA-IF) and both HC and SDAE (HC-SDAE-IF). Anomalies identified by these six methods were evaluated by the receiver operating characteristic (ROC) curve method. The spatial overlay of known mineral deposits and the identified geochemical anomaly maps showed that a strong spatial association existed between the locations of discovered mineral deposits and high geochemical anomaly areas. Furthermore, the area under the curve (AUC) values suggested that the HC-SDAE-IF hybrid model performed best and the HC-IF and HC-PCA-IF models performed better than the IF, PCA-IF and SDAE-IF models. This study demonstrated that: (i) IF is a useful and efficient unsupervised machine learning algorithm for detection of Ag-Pb-Zn polymetallic mineralization-related geochemical anomalies in the study area; and, (ii) the proper manipulation of dimensional reduction and extraction of mineralization-related geochemical variables can not only significantly improve the performance of unsupervised machine learning algorithms including deep learning methods in geochemical anomaly detection, but also be instrumental in clarifying the geological meaning of the identified geochemical anomaly. In this regard, reducing dimensionality and extracting mineralization-related variables (i.e. elements) from raw geochemical data is strongly recommended before applying unsupervised machine learning methods, particularly the sophisticated deep learning algorithms, in geochemical anomaly detection.

Making up the shortages of the Bayes classifier by the maximum mutual information classifier

The Bayes classifier is often used because it is simple, and the maximum posterior probability (MPP) criterion it uses is equivalent to the least error rate criterion. However, it has issues in the following circumstances: (i) if information instead of correctness is more important, we should use the maximum likelihood criterion or maximum information criterion, which can reduce the rate of failure to report small probability events. (ii) For unseen instance classifications, the previously optimised classifier cannot be properly used when the probability distribution of true classes is changed. (iii) When classes’ feature distributions instead of transition probability functions (TPFs) are stable, it is improper to train the TPF, such as the logistic function, with parameters. (iv) For multi-label classifications, it is difficult to optimise a group of TPFs with parameters that the Bayes classifier needs. This study addresses these issues by comparing the MPP criterion with the maximum likelihood criterion and maximum mutual information (MMI) criterion. It suggests using the MMI criterion for most unseen instance classifications. It presents a new iterative algorithm, the channel matching (CM) algorithm, for the MMI classification. It uses two examples to show the advantages of the CM algorithm: fast and reliable.

Storage Space Allocation Strategy for Digital Data with Message Importance.

This paper mainly focuses on the problem of lossy compression storage based on the data value that represents the subjective assessment of users when the storage size is still not enough after the conventional lossless data compression. To this end, we transform this problem to an optimization, which pursues the least importance-weighted reconstruction error in data reconstruction within limited total storage size, where the importance is adopted to characterize the data value from the viewpoint of users. Based on it, this paper puts forward an optimal allocation strategy in the storage of digital data by the exponential distortion measurement, which can make rational use of all the storage space. In fact, the theoretical results show that it is a kind of restrictive water-filling. It also characterizes the trade-off between the relative weighted reconstruction error and the available storage size. Consequently, if a relatively small part of total data value is allowed to lose, this strategy will improve the performance of data compression. Furthermore, this paper also presents that both the users’ preferences and the special characteristics of data distribution can trigger the small-probability event scenarios where only a fraction of data can cover the vast majority of users’ interests. Whether it is for one of the reasons above, the data with highly clustered message importance is beneficial to compression storage. In contrast, from the perspective of optimal storage space allocation based on data value, the data with a uniform information distribution is incompressible, which is consistent with that in the information theory.

Reliability and sensitivity analysis of more‐electric aircraft based on cross‐entropy

The traditional Monte Carlo sampling method has several deficiencies in evaluating the reliability of more-electric aircraft (MEA), such as more sampling times and longer simulation time, due to the complex topology and low probability of failure of the aircraft. In this Letter, the optimal probability distribution of multi-component systems is introduced by cross-entropy, which theoretically makes the variance of reliability evaluation zero. Based on this, an evaluation method for the power-supply reliability of MEA is presented. For further analysis of the system vulnerabilities, the sensitivity of MEA is modelled based on reliability calculation. Finally, the hybrid MEA and high-voltage direct current MEA power systems are used as application scenarios to calculate the reliability and sensitivity. The results show that the proposed method has advantages in evaluating small probability events, and it is useful for the optimisation of MEA's power-supply system structure.

Genetic algorithm-driven discovery of unexpected thermal conductivity enhancement by disorder

Discovering exceptions has been a major route for advancing sciences but a challenging and risky process. Machine learning has shown effectiveness in high throughput search of materials and nanostructures, but using it to discover exceptions has been out of the norm. Here we demonstrate the use of genetic algorithm to discover unexpected thermal conductivity enhancement in disordered nanoporous graphene as compared to periodic nanoporous graphene. Recent studies have concluded that random pores in nanoporous graphene lead to reduced thermal conductivity than periodic pores, due to phonon Anderson localization. This work, however, aims to challenge this accepted knowledge by searching for exceptions. A manual search was first shown to be expensive and unsuccessful. An efficient “two-step” search process coupled with genetic algorithm was then designed, and unexpected thermal conductivity enhancement was successfully discovered in certain structures with random pores, at a fraction of the computational cost of the manual search. Through structural analysis, we proposed that such unusual enhancement is due to the effect of shape factor and channel factor dominating over that of the phonon localization. Our work not only provides insights in thermal transport in disordered materials but also demonstrates the effectiveness of machine learning to discover small probability events and the intriguing physics behind.

Screening and analysis of the serum thyroid stimulating hormone (TSH) of rural pregestational women In Yunnan province

Objective Through screening the serum TSH in rural pregestational women, fo find out the high incidence area and high risk population of thyroid disease in Yunnan province, to reduce the probability of infertility and abnormal pregnancy and provide important reference basis for prenatal aristogenesis. Methods The serum TSH level of 30 163 rural pregestational women(20-49 yeas old) which from 16 different areas and 16 different ethnicities of Yunnan province was detected by using TSH3-Ultra (TSH3-UL) reagent on a Siemens centar-XP automatic chemiluminescence apparatus. Results 1 797 women were found with abnormal serum TSH level, the abnormal rate was 5.96%. Among them, the proportion of women with decreased TSH was 3.09% and increased TSH was 2.87%, compared with the normal reference range. And there was a significant difference of abnormal serum TSH rate between pregestational women from different areas and ethnicities. Moreover, in the 16 tested areas, the highest abnormal serum TSH rate was 8.90% from Ximeng county, the lowest was 3.60% from Diqing state. However in the 16 tested ethnicities, the highest abnormal serum TSH rate was 9.73% from Lisu ethnic minority and the lowest was 3.60% from Tibetan ethnic minority. Conclusions The results inferred that ethnic and regional factors both to some extent affect the prevalence of thyroid disease in pregestational women and it is a small probability event that high occurrence of TSH decreased or increased situation simultaneously happened in one area or one ethnicity. Key words: Thyroid stimulating hormone; Pregestational Women; Rural areas; Minority of Yunnan Province

Noise detection and image denoising based on fractional calculus

Abstract Fractional-order integral can weaken high-frequency signal and greatly preserve low-frequency signal nonlinearly, that is, the high-frequency noise can be removed while retaining the information of low-frequency image itself, thus the fractional calculus can achieve good a denoising effect and the application of fractional calculus theory in the digital image processing field has also been favored by more and more scholars. On the basis of summarizing and analyzing previous research works, this paper proposes a new method for detecting noise points in images using fractional differential gradient and an improved image denoising algorithm based on fractional integration. The noise detection method determines the noise position by the fractional differential gradient, and achieves to detect the noise, snowflake and stripe anomaly though utilizing the neighborhood information feature of the image and the contour and direction distribution of various noise anomalies in spatial domain; the image denoising algorithm firstly regards the appearance of noise points in image as a small probability event and divides it, then applies the fractional calculus to subtly transform and adjust signal filter and meanwhile utilizes iterative idea to control image denoising effect for accurately distinguishing between noise and high-frequency signals, so that the original image feature information is more preserved while the image is denoised. The simulation results show that this algorithm model can effectively remove the noise while maintaining the details of image edge and texture, and have the characteristics of simple algorithm and good stability. The model can effectively remove the noise and meanwhile preserve the details of image edges and textures, and has the characteristics of simple algorithm and high stability. The study results of this paper provide a reference for further research on the noise detection and image denoising based on fractional calculus.

Research on the influence of equity pledge on stock price crash risk: based on financial shock of 2015 stock market crisis

The 2015 Chinese stock market crisis triggered liquidation because of equity pledge so that the leverage effect of the small probability event with severe results got intensive attention from investors. It is found that the effects of equity pledge on stock price crash risk reversed significantly before and after the 2015 stock market crisis. In the mechanism analysis, we further find that the equity pledge influenced the stock price crash risk by longer suspension and greater price fluctuation. The shareholding ratio of institutional investors and information environment also had a significant moderating effect on the influence of equity pledge on stock price crash risk. Alternative interpretation tests excluded the tunnel effect and pressure effect by shareholders and incentive effect by management. This study by analysing empirical data provides evidence on the change of investors’ risk recognition, which is caused by financial shock, in the Chinese capital market.

Complexity analysis of time series based on generalized fractional order cumulative residual distribution entropy

Based on the theory of cumulative residual entropy and distribution entropy, this paper proposes a new model—cumulative residual distribution entropy (CRDE), which is more suitable for the complexity and risk analysis of time series The new model makes full use of the known information, including not only the information of probability, but also the information about the value of random variables. Additionally, the CRDE considers the potential inherent information of vector-to-vector distance in the state space. By combining theoretical analysis with empirical research, this paper verifies that the new model has more advantages in measuring extreme events and small probability events, with higher consistency, stability and practicability. In this paper, the cumulative residual distribution entropy model is extended to the fractional order. The generalized fractional cumulative residual distribution entropy (GCRDE) can better capture the tiny evolution of time series, which is more advantageous for studying the dynamic characteristics of complex systems. The new model makes up for some shortcomings of traditional models and it can play a guiding role in the study of complex systems in the real world.

Research on Ecological Efficiency for the Remanufacturing Process Considering Optimization and Evaluation

The ecological efficiency (EE) of the remanufacturing process occupies an important position in the whole index system of remanufacturing because it will directly affect the economic and environmental benefits of remanufacturing. Therefore, in order to study the EE of the remanufacturing process, a method is proposed to optimize and evaluate the EE of the remanufacturing process. In this method, firstly, the original remanufacturing sub-schemes of used components are designed according to the extracted fault characteristics; secondly, a set of optional process schemes are integrated by using directed graph (DG) to reduce the process schemes; thirdly, the objective function of EE is established, and then an ant colony algorithm with elite strategy (ES-ACO) is proposed to optimize the process schemes. After obtaining the optimal value of EE, the quality coefficient of used components can be calculated, and then numerical simulations (NS) are used to analyze the correlation between the quality coefficient and the optimized EE, after that, polynomial function fitting (PFF) is applied to construct the evaluation model of EE oriented to the quality coefficient, then, the evaluation model is utilized to analyze the range of quality coefficient of used components suitable for remanufacturing under cost constraints. Finally, the feasibility of this method is verified by the example of the used lathe spindle remanufacturing; and the case study shows that in the optimization phase, ES-ACO can not only optimize the process schemes but also has better performance than ACO; in the evaluation stage, the probability of deviation of the evaluation function established by using PFF is 5%, meeting the small probability event. (i.e., the occurrence of very low-frequency events), that is, the accuracy of the evaluation meets the requirements.

STUDY ON THE FEATURE OF ELECTROMAGNETIC RADIATION UNDER COAL OXIDATION AND TEMPERATURE RISE BASED ON MULTIFRACTAL THEORY

During coal oxidation and temperature rise, a significantly large amount of electromagnetic radiation (EMR) signals are generated as a result of thermal deformation and thermal cracking. The generation of EMR signal is the comprehensive embodiment of the physical and chemical changes in coal during its oxidation and subsequent heating. Therefore, the generated signals contain complex and rich messages that can reflect the changes in the internal structure of coal. In this work, the characteristics of EMR signal were analyzed by multifractal theory. Multifractal analysis was used to deconstruct EMR signals. Our preliminary study indicated that the multifractal spectrum of the EMR signal had a feature of small probability event. Further analysis demonstrated that the characteristic parameters of EMR signal are quite different at the later heating stage from those at the early heating stage. At the initial stage of coal spontaneous combustion, the multifractal spectrum of the signal is wider, but when coal combustion is reached, the scale range of EMR signal increases significantly, and its structure changes at different temperature ranges. The research results presented provide a basis for monitoring and presenting an early warning of coal spontaneous combustion risk.

An efficient acceleration Monte Carlo simulation for pricing Asian option under variance gamma process by splitting

ABSTRACTThis paper proposes a hybrid acceleration method for pricing Asian options with arithmetic average under variance gamma process. We split the payoff of Asian option into two parts, the first part is a small probability event, so the importance sampling method is naturally used to reduce variance of simulation efficiently. The second part can be simplified as a semi-closed form, which also be considered as a conditional expectation formula, so it may reduce variance of simulations. Then the importance sampling method is also adapted to reduce variance of simulations further. To save computation cost for determining the optimal parameters in the importance sampling, a moment matching-based technique is proposed. Numerical results are given to show the high efficiency of our method. The idea used in this paper may also be applicable to price Asian option and Basket option under other time-changed Brownian motion and Heston models.

Single Stock Call Options as Lottery Tickets: Overpricing and Investor Sentiment

The authors investigate whether the overpricing of out-of-the money single stock calls can be explained by Tversky and Kahneman's [1992] cumulative prospect theory (CPT). They hypothesize that these options are expensive because investors overweight small probability events and overpay for positively skewed securities (i.e., lottery tickets). The authors find that overweighting of small probabilities embedded in the CPT explains the richness of out-of-the money single stock calls better than other utility functions. Nevertheless, overweighting of small probabilities events is less pronounced than suggested by the CPT, is strongly time varying, and most frequent in options of short maturity. The authors find that fluctuations in overweighting of small probabilities are largely explained by the sentiment factor.

Probability weighting functions obtained from Hong Kong index option market

In this paper we estimate the pricing kernel from the Hong Kong index option market and obtain the empirical probability weighting functions based on the rank-dependent expected utility. The empirical pricing kernel is estimated semi-parametrically as the ratio of the risk-neutral and objective densities. We employ a two-step estimation procedure to estimate the objective and risk-neutral densities under a consistent parametric framework of the non-affine generalised autoregressive conditional heteroskedasticity (G.A.R.C.H.) diffusion model. In the first step, we develop a continuous particle filters-based maximum likelihood estimation method to estimate the objective parameters of the G.A.R.C.H. diffusion model using the Hang Seng Index (H.S.I.) returns. In the second step of our estimation, we depart from the usual pure calibration approach and use the H.S.I. option prices to estimate the risk-neutral parameters of the G.A.R.C.H. diffusion model by constraining certain parameters to be consistent with the time-series behaviour of H.S.I. returns. Based on the estimated objective and risk-neutral parameters, the objective and risk-neutral densities are obtained by inverting the corresponding characteristic functions. Empirical results indicate that the empirical pricing kernel estimated from the Hong Kong index option market is non-monotonic and the estimated probability weighting functions are S-shaped, which implies that investors underweight small probability events and overweight large ones.

Importance of Small Probability Events in Big Data: Information Measures, Applications, and Challenges

In many applications (e.g., anomaly detection and security systems) of smart cities, rare events dominate the importance of the total information of big data collected by Internet of Things (IoTs). That is, it is pretty crucial to explore the valuable information associated with the rare events involved in minority subsets of the voluminous amounts of data. To do so, how to effectively measure the information with importance of the small probability events from the perspective of information theory is a fundamental question. This paper first makes a survey of some theories and models with respect to importance measures and investigates the relationship between subjective or semantic importance and rare events in big data. Moreover, some applications for message processing and data analysis are discussed in the viewpoint of information measures. In addition, based on rare events detection, some open challenges related to information measures, such as smart cities, autonomous driving, and anomaly detection in IoTs, are introduced which can be considered as future research directions.

Generalized Belief Operator and the Impact of Small Probability Events on Higher Order Beliefs

This paper studies the robustness of an equilibrium to incomplete information in binary-action supermodular games. Using a generalized version of belief operator, we explore the restrictions that prior beliefs impose on higher order beliefs. In particular, we obtain a non-trivial lower bound on the probability of a common belief event, uniform over type spaces, when the underlying game has a monotone potential. Conversely, when the game has no monotone potential, we construct a type space with an arbitrarily high probability event in which players never have common belief about that event. As an implication of these results, we show for generic binary-action supermodular games that an action profile is robust to incomplete information if and only if it is a monotone potential maximizer. Our study offers new methodology and insight to the analysis of global game equilibrium selection.

Estimation of population size in entropic perspective

Nonparametric estimation of population size is a long standing and difficult problem. It is difficult because, particularly from a likelihood perspective, the underlying distribution could vary greatly and many small probability events may not be observed in a sample. However if approached from an entropic standpoint, certain trends can be exploited. This article proposes several estimators based on an entropic representation of population size, and establishes their consistency. Simulation results of the proposed estimators are also reported in comparison with a well-known estimator, and the advantages are noted. Two examples with real data are also included.

Multiobjective Optimization Approach for Coordinating Different DG from Distribution Network Operator

Integrating with analysis of uncertainties, this paper presented a multiobjective optimization approach for coordinating different DG from the perspective of Distribution Network Operator (DISOPER). Aiming to three uncertain factors including fuzzy variable, random variable, and interval variable, the information entropy and interval analysis methods are adopted to construct multistate models of multisource uncertainty. The information entropy method is to convert fuzzy variable into equivalent random variable. Interval analysis method is to transform random variables into interval variables by setting a confidence level. Then plenty of simulation analysis based on the small probability event and expectation are investigated to reduce the computational burden and eliminate invalid computation. Subsequently, multiobjective formulations based on multistate are built by analyzing systematical power loss, voltage quality, reliability, and environment change provide some reference for DISOPER in dealing with access of privately owned DG units. Furthermore, based on network topology analysis and modified nondominated sorting genetic algorithm (NSGA), a combinatorial optimization method is proposed to reduce search space and solve the constructed formulations efficiently. Simulations are carried out on IEEE 37-bus systems and results are presented and discussed.

Time-variant reliability analysis using the parallel subset simulation

Time-variant reliability problems commonly occur in practical engineering applications due to deterioration in material properties, dynamic load and other causes. Since this kind of problem is usually a small probability event, subset simulation is more efficient than Monte Carlo simulation (MCS). However, subset simulation can only focus on a single limit function when propagating the conditional samples. Parallel subset simulation is applied to deal with time-dependent reliability analysis in this paper. A new method is proposed to construct a function called “principal variable”. The “principal variable” can represent limit state at each time instant to generate conditional samples. In addition, the update procedure of “principal variable” should be set at each simulation stage to keep the correlations between “principal variable” and nt limit states strong. Two numerical examples are used to demonstrate the effectiveness and accuracy of the developed parallel subset simulation for time-variant reliability analysis.