Accurate Reliability Estimation Research Articles

Estimating confidence interval of software reliability with adaptive testing strategy

Software reliability assessment is a critical problem in safety-critical and mission-critical systems. In the reliability assessment of such a system, both an accurate reliability estimate and a tight confidence interval are required. Adaptive testing (AT) is an on-line testing framework, which dynamically selects test cases from different subdomains to achieve some optimization object. Although AT has been proved effective in minimizing reliability estimator variance, its performance on providing the corresponding confidence interval has not been investigated. In order to address this issue, an AT strategy combined with Bayesian inference (AT-BI) is proposed in this study. The novel AT-BI strategy is expected to be effective in providing both a low-variance estimator and a tight confidence interval. Experiments are set up to validate the effectiveness of the AT-BI strategy.

A Dynamical Reliability Prediction Algorithm for Composite Service

Dynamic selection and dynamic binding and rebinding at runtime are new characters of composite services. The traditional static reliability prediction models are unsuitable to dynamic composite services. A new reliability predicting algorithm for composite services is proposed in this paper. Firstly, a composite service is decomposed into composition unites (executing path, composite module and atomic service) according to their constituents. Consequently, a hierarchical graph of all composite units is constructed. Lastly, a new dynamic reliability prediction algorithm is presented. Comparing with the traditional reliability model, the new dynamic reliability approach is more flexible, which does not recompute reliability for all composite units and only computes the reliability of the effected composite units. In addition, an example to show how to measure the reliability based on our algorithm is designed. The experimental results show our proposed methods can give an accurate estimation of reliability. Furthermore, a more flexible sensitivity analysis is performed to determine which service component has the most significant impact on the improvement of composite service reliability.

Measurement of composite reliability in research using partial least squares

The accurate estimation of reliability is of great importance to the conduct and interpretation of empirical research as it is used to judge the quality of reported research, often plays a role in publication decisions, and is a key element of meta-analytic reviews. When employing partial least squares (PLS) as the method of analysis, the reliability of the composites involved in the model is typically the parameter examined. In this research, we describe the existence of three important issues concerning the accuracy of composite reliability estimation in PLS analysis: the assumption of equal indicator weights, the bias in loading estimates, and the lack of independence between indicator loadings and weights. We subsequently present an alternative approach to correct these issues. Using a Monte Carlo simulation we provide a demonstration of both the effects of these issues on research decisions and the improved accuracy of the alternative method.

Comparing Alternate Approaches to Calculating Reliability for Dichotomous Data: The Sample Case of Adolescent Selection, Optimization, and Compensation

Estimating reliability for scales or factors is an essential data analysis step in much of the research in developmental science. In this article, we demonstrate the importance of using the appropriate statistical method and underlying correlation matrix to estimate reliability for dichotomous data that represent a normally-distributed latent factor. We used an example case of three waves of adolescent data collected from responses to the Selection, Optimization, and Compensation questionnaire (SOC; Freund & Baltes, 2002) of intentional self-regulation to illustrate how calculating composite reliability (or ω) using tetrachoric correlations provides a more accurate estimate of reliability when compared to both raw covariance-based ω, as well as raw covariance-based and tetrachoric correlation-based Cronbach's α approaches. In addition, we describe methods for calculating each of these approaches to reliability estimation, and we offer suggestions for future researchers for estimating reliability for such dichotomous data.

Proportional Odds Nonparametric Accelerated Life Test for Reliability Prediction: An overview

One way of obtaining information about reliability of units is to accelerate their life by testing at higher levels of stress (such as increasing elevated temperatures or voltages). Predicting the lifetime of a unit at normal operating conditions based on data collected at accelerated conditions is a common objective of these tests. Different models of accelerated life testing are used for such extrapolations. Two statistical based models are widely used: parametric models which require a prior specified lifetime distribution, and nonparametric models that relax of the assumption of the life time distribution. The proportional odds model is a nonparametric model in accelerated life testing based on the odds function and show that it gives a more accurate reliability estimates than proportional hazard model. This paper will concentrate on the models of proportional odds nonparametric accelerated life test for reliability prediction.

Reliability Analysis of a Composite Wind Turbine Blade Section Using the Model Correction Factor Method: Numerical Study and Validation

Reliability analysis of fiber-reinforced composite structures is a relatively unexplored field, and it is therefore expected that engineers and researchers trying to apply such an approach will meet certain challenges until more knowledge is accumulated. While doing the analyses included in the present paper, the authors have experienced some of the possible pitfalls on the way to complete a precise and robust reliability analysis for layered composites. Results showed that in order to obtain accurate reliability estimates it is necessary to account for the various failure modes described by the composite failure criteria. Each failure mode has been considered in a separate component reliability analysis, followed by a system analysis which gives the total probability of failure of the structure. The Model Correction Factor method used in connection with FORM (First-Order Reliability Method) proved to be a fast and efficient way to calculate the reliability index of a complex composite structure.

Sample size planning for composite reliability coefficients: Accuracy in parameter estimation via narrow confidence intervals

Composite measures play an important role in psychology and related disciplines. Composite measures almost always have error. Correspondingly, it is important to understand the reliability of the scores from any particular composite measure. However, the point estimates of the reliability of composite measures are fallible and thus all such point estimates should be accompanied by a confidence interval. When confidence intervals are wide, there is much uncertainty in the population value of the reliability coefficient. Given the importance of reporting confidence intervals for estimates of reliability, coupled with the undesirability of wide confidence intervals, we develop methods that allow researchers to plan sample size in order to obtain narrow confidence intervals for population reliability coefficients. We first discuss composite reliability coefficients and then provide a discussion on confidence interval formation for the corresponding population value. Using the accuracy in parameter estimation approach, we develop two methods to obtain accurate estimates of reliability by planning sample size. The first method provides a way to plan sample size so that the expected confidence interval width for the population reliability coefficient is sufficiently narrow. The second method ensures that the confidence interval width will be sufficiently narrow with some desired degree of assurance (e.g., 99% assurance that the 95% confidence interval for the population reliability coefficient will be less than W units wide). The effectiveness of our methods was verified with Monte Carlo simulation studies. We demonstrate how to easily implement the methods with easy-to-use and freely available software.

미소한 신뢰도 감소율을 가지는 원샷 시스템의 가부반응 데이터를 이용한 저장 신뢰도 추정방법 개발

This paper proposes a new reliability estimation method for one-shot systems using quantal response data, which is based on a parametric estimation method. The proposed method considers the time-variant failure ratio of the quantal response data and it can overcome the problems in parametric estimation methods. Seven reliability estimation methods in the literature were compared with the proposed method in terms of the accuracy of reliability estimation in order to verify the proposed method. To compare the accuracy of reliability estimation, the SSEs (Sum of Squared Error) of the reliability estimation results for the different estimation methods were evaluated according to the various numbers of samples tested. The proposed method provided more accurate reliability estimation results than any of the other methods from the results of the accuracy comparison.

Effect of stress concentration factors on the structural integrity assessment of multi-planar offshore tubular DKT-joints based on the fracture mechanics fatigue reliability approach

A reasonably accurate estimate of fatigue reliability is considered to be essential for offshore structures due to their high construction cost and failure consequences. In the present paper, firstly, a fracture mechanics based fatigue limit-state function is derived and effects of assigning different values to deterministic variables involved in this limit-state function on the results of reliability analysis are investigated. These deterministic variables are the material parameter, number of stress cycles, and the mean value of critical crack size. Afterwards, effects of stress concentration factors (SCFs) on the reliability of the tubular joint are studied. The SCF is one of the most important parameters in the fatigue reliability analysis, which exhibits considerable scatter. Results showed that change of the mean value of SCF, from the value typically assumed in the previous research works (=2.5) to a realistic value obtained in the present research for a widely used multi-planar joint (∼15), leads to considerable decrease in the reliability index and consequently significant increase in the probability of failure. However, change of the coefficient of variation (CoV) of SCFs, from the typically assumed value (=0.15) to a realistic value obtained in the present research (∼1.048), does not considerably affect the results.

Fatigue Assessment of Slender Long-Span Bridges: Reliability Approach

It is well known that wind loads result in wind-induced buffeting of slender long-span bridges. Traffic also frequently exists on a slender long-span bridge simultaneously throughout its service life, which can affect the dynamic behavior of the slender structural systems. The nature of the vibration of a slender long-span bridge is further complicated by the dynamic coupling effects among the bridge, wind, and traffic. Fatigue damage is an important consideration in the serviceability performance of slender long-span bridges that typically experience cyclic or repetitive loads, such as those induced by wind and traffic. Stochastic wind and traffic loads, dynamic interaction, and fatigue accumulation for a slender long-span bridge essentially vary over time, with uncertainty contributions from each. Therefore, a reliability-based fatigue assessment model, which can rationally consider the combined load effects from wind and traffic as well as the associated uncertainties, is needed to provide an accurate estimate of reliability for fatigue. Initially, a scenario-based deterministic fatigue analysis model is developed. A typical year is categorized into several representative scenarios of traffic and wind conditions. After identifying the duration for each representative scenario in a typical year, a cumulative yearly fatigue damage factor can be predicted by superposing the cumulative hourly damage factors for all representative scenarios. On the basis of the scenario-based deterministic analytical model, the framework of the reliability-based fatigue damage assessment for a slender long-span bridge is further developed. Finally, an illustrative example for a prototype bridge is conducted.

Guided tail modelling for efficient and accurate reliability estimation of highly safe mechanical systems

Classical tail modelling is based on performing a relatively small number of limit-state calculations through Monte Carlo sampling, and then fitting a generalized Pareto distribution to the tail part of the data. The limit-state calculations that do not belong to the tail part are discarded. To reduce the amount of discarded data, this article proposes an efficient tail modelling procedure based on guiding the limit-state evaluations towards the sampling points that have high chances of yielding limit-state values falling into the tail region. The guidance of the limit-state evaluations is achieved through a procedure that utilizes limit-state approximation and distribution fitting. The accuracy of the proposed method is tested through a mathematical problem and four structural mechanics problems, and it is found that the accuracy of reliability estimations can be significantly increased compared to classical tail modelling techniques for the same number of limit-state function evaluations. In addition, it is also found that the improvement in accuracy can be traded off for reducing the number of limit-state evaluations.

Varying coefficient meta-analytic methods for alpha reliability.

The conventional fixed-effects (FE) and random-effects (RE) confidence intervals that are used to assess the average alpha reliability across multiple studies have serious limitations. The FE method, which is based on a constant coefficient model, assumes equal reliability coefficients across studies and breaks down under minor violations of this assumption. The RE method, which is based on a random coefficient model, assumes that the selected studies are a random sample from a normally distributed superpopulation. The RE method performs poorly in typical meta-analytic applications where the studies have not been randomly sampled from a normally distributed superpopulation or have been randomly sampled from a nonnormal superpopulation. A new confidence interval for the average reliability coefficient of a specific measurement scale is based on a varying coefficient statistical model and is shown to perform well under realistic conditions of reliability heterogeneity and nonrandom sampling of studies. New methods are proposed for assessing reliability moderator effects. The proposed methods are especially useful in meta-analyses that involve a small number of carefully selected studies for the purpose of obtaining a more accurate reliability estimate or to detect factors that moderate the reliability of a scale.

Fast reliability analysis of combinatorial logic circuits using conditional probabilities

Probabilistic reliability analysis is a common approach in logic circuit reliability analysis. Existing methods suffer from accuracy or scalability problems for large circuits because of combinatorial explosion. In this work we show how the use of conditional probabilities can overcome scalability problems while maintaining accurate reliability estimation. The source of accuracy and scalability problems in these approaches is the presence of reconverging signals. An efficient use of conditional probabilities used to decorrelate signals allows for fast and accurate reliability analysis.

Assessment of the Maximal Split-Half Coefficient to Estimate Reliability

The maximal split-half coefficient is computed by calculating all possible split-half reliability estimates for a scale and then choosing the maximal value as the reliability estimate. Osburn compared the maximal split-half coefficient with 10 other internal consistency estimates of reliability and concluded that it yielded the most consistently accurate estimate of reliability across the conditions explored. In this simulation study, the authors demonstrate that the maximal split-half coefficient can, under some conditions, badly overestimate reliability because of capitalization on chance and under other conditions appreciably underestimate reliability. Cross-validation can correct the former problem. A difficulty with the cross-validated coefficient is that it is a relatively inefficient estimator of reliability.

A Note on Structural Equation Modeling Estimates of Reliability

Reliability can be estimated using structural equation modeling (SEM). Two potential problems with this approach are that estimates may be unstable with small sample sizes and biased with misspecified models. A Monte Carlo study was conducted to investigate the quality of SEM estimates of reliability by themselves and relative to coefficient alpha. The SEM approach showed minimal bias when the model was correctly specified if items were relatively well defined by their underlying factor(s). They tended to demonstrate somewhat greater bias when the model was misspecified, particularly underspecified. Overall, SEM estimates were more stable than anticipated. Researchers are more likely to obtain accurate estimates of reliability using SEM by conducting large-sample studies with well-constructed scales and critically assessing model fit.

Application of bootstrap method in conservative estimation of reliability with limited samples

Accurate estimation of reliability of a system is a challenging task when only limited samples are available. This paper presents the use of the bootstrap method to safely estimate the reliability with the objective of obtaining a conservative but not overly conservative estimate. The performance of the bootstrap method is compared with alternative conservative estimation methods, based on biasing the distribution of system response. The relationship between accuracy and conservativeness of the estimates is explored for normal and lognormal distributions. In particular, detailed results are presented for the case when the goal has a 95% likelihood to be conservative. The bootstrap approach is found to be more accurate for this level of conservativeness. We explore the influence of sample size and target probability of failure on the quality of estimates, and show that for a given level of conservativeness, small sample sizes and low probabilities of failure can lead to a high likelihood of large overestimation. However, this likelihood can be reduced by increasing the sample size. Finally, the conservative approach is applied to the reliability-based optimization of a composite panel under thermal loading.

Accurate Conjunction of Yield Models for Fault-Tolerant Memory Integrated Circuits

Critical defects, i.e., faults, inevitably occur during semiconductor fabrication, and they significantly reduce both manufacturing yield and product reliability. To decrease the effects of the defects, several fault-tolerance methods, such as the redundancy technique and the error correcting code (ECC), have been successfully applied to memory integrated circuits. In the semiconductor business, accurate estimation of yield and reliability is very important for determining the chip architecture as well as the production plan. However, a simple conjunction of previous fault-tolerant yield models tends to underestimate the manufacturing yield if several fault-tolerance techniques are employed simultaneously. This paper concentrates on developing and verifying an accurate yield model which can be applied successfully in such situations. The proposed conjunction model has been derived from the probability of remaining redundancies and the average number of defects after repairing the defects with the remaining redundancies. The validity of the conjunction yield model is verified by a Monte Carlo simulation.

On the Integrity of Reliability Estimation in Classical Test Theory: The Case for an Additive Coefficient of Stability

This article addresses deficiencies in the most widely used estimators of reliability and draws attention to the reason that this issue is important. Accurate calibration of relationships between constructs is critical to theory development. Unless workers have accurate estimates of scale reliability, accurate estimates of those relationships will not be forthcoming because the classical disattenuation formula requires them. This article shows that classical test theory can easily accommodate the delineation of its error component E in test scores into two sources, inconsistency across content (E1) and inconsistency across time (E2). Viewed from this extended model, the alternate forms approach to reliability estimation is complete in that it gauges simultaneously both sources of error. Because that approach is rarely used today for that purpose, the integrity of estimation has been lost. In its place arose estimators of partial reliability--those for estimating generalizability over one medium or the other, but not both, thereby precluding the additivity of error components. Recent developments promise to restore the integrity of the alternate forms approach without the need for alternate forms and suggest an additive alternative to the current nonadditive coefficient of stability.

Early inference on reliability of upgraded automotive components by using past data and technical information

When a new product is the result of design and/or process improvements introduced in its predecessors, then the past failure data and the expert technical knowledge constitute a valuable source of information that can lead to a more accurate reliability estimate of the upgraded product. This paper proposes a Bayesian procedure to formalize the prior information available about the failure probability of an upgraded automotive component. The elicitation process makes use of the failure data of the past product, the designer information on the effectiveness of planned design/process modifications, information on actual working conditions of the upgraded component and, for outsourced components, technical knowledge on the effect of possible cost reductions. By using the proposed procedure, more accurate estimates of the failure probability can arise. The number of failed items in a future population of vehicles is also predicted to measure the effect of a possible extension of the warranty period. Finally, the proposed procedure was applied to a case study and its feasibility in supporting reliability estimation is illustrated.

Architectural-Level Soft-Error Modeling for Estimating Reliability of Computer Systems

This paper proposes a soft-error model for accurately estimating reliability of a computer system at the architectural level within reasonable computation time. The architectural-level soft-error model identifies which part of memory modules are utilized temporally and spatially and which single event upsets (SEUs) are critical to the program execution of the computer system at the cycle accurate instruction set simulation (ISS) level. The soft-error model is capable of estimating reliability of a computer system that has several memory hierarchies with it and finding which memory module is vulnerable in the computer system. Reliability estimation helps system designers apply reliable design techniques to vulnerable part of their design. The experimental results have shown that the usage of the soft-error model achieved more accurate reliability estimation than conventional approaches. The experimental results demonstrate that reliability of computer systems depends on not only soft error rates (SERs) of memories but also the behavior of software running in computer systems.