Confidence In Correctness Research Articles

A Hardware Virtualization Based Component Sandboxing Architecture

Modern applications comprise multiple components, such as browser plug-ins, often of unknown provenance and quality. Statistics show that failure of such components accounts for a high percentage of software faults. Enabling isolation of such fine-grained components is therefore necessary to increase the robustness and resilience of security-critical and safety-critical computer systems. In this paper, we evaluate whether such fine-grained components can be sandboxed through the use of the hardware virtualization support available in modern Intel and AMD processors. We compare the performance and functionality of such an approach to two previous software based approaches. The results demonstrate that hardware isolation minimizes the difficulties encountered with software based approaches, while also reducing the size of the trusted computing base, thus increasing confidence in the solution's correctness. We also show that our relatively simple implementation has equivalent run-time performance, with overheads of less than 34%, does not require custom tool chains and provides enhanced functionality over software-only approaches, confirming that hardware virtualization technology is a viable mechanism for fine-grained component isolation.

Tool Support for Change-Centric Test Development

Testing increases confidence in software's correctness, completeness, and quality. By executing a test on a program, developers can check the outcome against the program's specification to identify faults. Various testing levels can serve different purposes during development-for example, unit and integration testing let developers test an implementation and its effects on existing functionality. In test-driven development, a unit test acts as a functionality specification before implementation, letting developers apply only the code necessary to pass the test. Applying change impact analysis to test-driven development provides software designers quantitative feedback they can use to meet a coverage goal and avoid unanticipated change effects.

A new tool-kit for designing complex material handling systems using IEC61499 function blocks

The model-driven approach to software development has been increasingly applied to manage the growing complexity in modern manufacturing systems. With this approach, a model of a system is first created at a high-level of abstraction. The model is then converted into an executable program by means of automatic code generation tools. Models can be simulated and visualized throughout the entire design flow of the system, thereby increasing confidence of the design's correctness and its adherence to user requirements. Moreover, since the design is specified at a higher level of abstraction, the description itself is easier to understand, and consequently, less prone to errors. However, automated code generation from high-level models have typically been inefficient compared to hand-crafted code. In this paper, we demonstrate an approach for generating efficient code for systems modelled using IEC 61499 function blocks. We have also developed a tool chain that allows complex material handling systems to be modelled, visualized, and synthesized in a seamless fashion. Experimental evaluations show that our approach produces significantly faster and smaller code compared to an existing tool.

The Impact of an Online Evidence System on Confidence in Decision Making in a Controlled Setting

To examine the impact of online evidence retrieval on clinicians' decision-making confidence and to determine if this differs for experienced doctors and nurses. A sample of 44 doctors and 31 clinical nurse consultants (CNCs) answered 8 clinical scenarios (600 scenario answers) before and after the use of online evidence resources. Clinicians rated their confidence in scenario answers and in the evidence they found using the information system. Prior to using online evidence, 37% of doctors and 18% of CNCs answered the scenarios correctly. These clinicians were more confident (56% very confident or confident) in their answers than those with incorrect (34%) answers. Doctors with incorrect answers prior to searching rated their confidence significantly higher than did nurses who were incorrect. After searching, both groups answered 50% of scenarios correctly. Clinicians with correct answers had greater confidence in the evidence found compared to those with incorrect answers. Doctors were more confident in evidence found confirming an initially correct answer than were nurses. More than 50% of clinicians who persisted with an incorrect answer after searching reported that they were confident or very confident in the evidence found. Clinicians who did not know scenario answers before searching placed equal confidence in evidence that led them to a correct or incorrect answer. The information obtained from an online evidence system influenced clinicians' confidence in their answers to the clinical scenarios. The relationship between confidence in answers and correctness is complex. Both existing knowledge and professional role were mediating factors. The finding that many clinicians placed confidence in information that led them to incorrect answers warrants further investigation.

Automatic test program generation: a case study

Design validation is a critical step in the development of present-day microprocessors, and some authors suggest that up to 60% of the design cost is attributable to this activity. Of the numerous activities performed in different stages of the design flow and at different levels of abstraction, we focus on simulation-based design validation performed at the behavioral register-transfer level. Designers typically write assertions inside hardware description language (HDL) models and run extensive simulations to increase confidence in device correctness. Simulation results can also be useful in comparing the HDL model against higher-level references or instruction set simulators. Microprocessor validation has become more difficult since the adoption of pipelined architectures, mainly because you can't evaluate the behavior of a pipelined microprocessor by considering one instruction at a time; a pipeline's behavior depends on a sequence of instructions and all their operands.

Presentation of information for spatial decision support A survey on the use of maps by participants in quantitative water management in the IJsselmeer region, The Netherlands

Spatial decision support systems generate a diversity of information presented in tables, graphs, text and maps. Which form is used is dictated partially by the nature of the information but also by those who prepare information to be used in a decision-making process. The users of the information do not necessarily have the same preference for the way the information is presented. A survey has been conducted amongst stakeholders in a decision-making process to see what their preferences are. The purpose of the survey was to gain insight into the ability of stakeholders to use information presented at different levels of detail when comparing alternatives. This paper reports on the survey’s results. Possible relations between level of detail, the ease of use of information, confidence in correctness and value of the information are presented. The results show that maps and graphs are preferred above tables and text. A high preference for maps is, however, not a measure of the ability to use maps. In addition, the results show that even though the ability to use the information increased with decreasing level of detail, detailed information is found more valuable.

Can fault‐exposure‐potential estimates improve the fault detection abilities of test suites?

AbstractCode‐coverage‐based test data adequacy criteria typically treat all coverable code elements (such as statements, basic blocks or outcomes of decisions) as equal. In practice, however, the probability that a test case can expose a fault in a code element varies: some faults are more easily revealed than others. Thus, several researchers have suggested that if one could estimate the probability that a fault in a code element will cause a failure, one could use this estimate to determine the number of executions of a code element that are required to achieve a certain level of confidence in that element's correctness. This estimate, in turn, could be used to improve the fault‐detection effectiveness of test suites and help testers distribute testing resources more effectively. This conjecture is intriguing; however, like many such conjectures it has never been directly examined empirically. If empirical evidence were to support this conjecture, it would motivate further research into methodologies for obtaining fault‐exposure‐potential estimates and incorporating them into test data adequacy criteria. This paper reports the results of experiments conducted to investigate the effects of incorporating an estimate of fault‐exposure probability into the statement coverage test data adequacy criterion. The results of these experiments, however, ran contrary to the conjectures of previous researchers. Although incorporation of the estimates did produce statistically significant increases in the fault‐detection effectiveness of test suites, these increases were quite small, suggesting that the approach might not be able to produce the gains hoped for and might not be worth the cost of its employment. Copyright © 2002 John Wiley & Sons, Ltd.

A fifth generation numerical modelling system in coastal zone

Nowadays, artificial intelligence (AI) technology is gradually integrated into the numerical modelling system to make the system more intelligent and more user-friendly. The characteristics of the fifth generation numerical modelling are connected with AI applications. The expert system technology as a widely applied AI technology is integrated into our modelling system for coastal water processes with traditional numerical computational tools and the data and graphical pre-processing and post-processing techniques. Five kinds of knowledge bases are built in the system to describe the existing expertise knowledge about model parameters, relations between parameters and physical conditions, various possible selections for parameters and rules of inference. The inference engine is designed to be driven by the confidence of correctness, and the rule base is built with the factor of confidence to link the various relations. The decision tree is designed to drive the inference engine to explore the route of selection procedure of modeling. The decision tree depends on the real problem specifications and can be modified during the dialogue between the system and the user. The forward chaining and backward chaining inference techniques are mixed together in the system to help matching the parameters in the model and the possible selections with sufficiently high confidence. The expert system technology is successfully integrated into the system to provide help for model parameter selection or model selection, and to make the numerical model system more accessible for non-expert users.

Confidence in performance on science tests and student preparation strategies

This exploratory study was designed to investigate the accuracy of high school students' confidence in their answers on classroom tests, how that accuracy varied between different types of questions, and whether having to judge their confidence in test items resulted in an improvement in accuracy over the school year. The study was conducted in a rural high school in a South Eastern state of the US involving 54 students enrolled in three sections of a human anatomy class. For an entire school year the teacher asked students to indicate their confidence in a response's correctness when they answered questions on tests and examinations. Further data included interviews with 25 students and a career goals questionnaire. Having students gauge the correctness of their responses to test questions on class tests over the school year did not result in quantifiable improvements in their accuracy. However, students indicated that reflective use of the language of human anatomy and physiology through reading out loud to themselves, practising writing words and phrases, and verbal questioning and discussion with others helped them to achieve higher academic outcomes. Modelling or discussing effective study strategies with family members were also identified as important factors on the ways students prepared for tests.

ATM switch design by high-level modeling, formal verification and high-level synthesi

Asynchronous Transfer Mode (ATM) has emerged as a backbone for high-speed broadband telecommunication networks. In this paper, we present ATM switch design, starting from a parametric high-level model and debugging the model using a combination of formal verification and simulation. The model has been used to synthesize ATM switches according to customers' choices, by choosing concrete values for each of the generic parameters. We provide a pragmatic combination of simulation, model checking, and theorem proving to gain confidence in the ATM switch design correctness.

Verifying Timing Constraints of Real-Time Systems by Means of Evolutionary Testing

Many industrial products are based on the use of embedded computer systems. Usually, these systems have to fulfil real-time requirements, and correct system functionality depends on their logical correctness as well as on their temporal correctness. In order to verify the temporal behavior of real-time systems, previous scientific work has, to a large extent, concentrated on static analysis techniques. Although these techniques offer the possibilty of providing safe estimates of temporal behavior for certain cases, there are a number of cases in practice for which static analysis can not be easily applied. Furthermore, no commercial tools for timing analysis of real-world programs are available. Therefore, the developed systems have to be thoroughly tested in order to detect existing deficiencies in temporal behavior, as well as to strengthen the confidence in temporal correctness. An investigation of existing test methods shows that they mostly concentrate on testing for logical correctness. They are not specialised in the examination of temporal correctness which is also essential to real-time systems. For this reason, existing test procedures must be supplemented by new methods which concentrate on determining whether the system violates its specified timing constraints. Normally, a violation means that outputs are produced too early, or their computation takes too long. The task of the tester therefore is to find the input situations with the longest or shortest execution times, in order to check whether they produce a temporal error. If the search for such inputs is interpreted as a problem of optimization, evolutionary computation can be used to automatically find the inputs with the longest or shortest execution times. This automatic search for accurate test data by means of evolutionary computation is called evolutionary testing. Experiments using evolutionary testing on a number of programs with up to 1511 LOC and 5000 input parameters have successfully identified new longer and shorter execution times than had been found using other testing techniques. Evolutionary testing, therefore, seems to be a promising approach for the verification of timing constraints. A combination of evolutionary testing and systematic testing offers further opportunities to improve the test quality, and could lead to an effective test strategy for real-time systems.

Grc4f v1.1: a four-fermion event generator for e +e − collisions

grc4f is a Monte Carlo package for generating e + e −→4-fermion processes in the standard model. All of the 76 LEP-2 allowed fermionic final state processes evaluated at tree level are included in version 1.1 grc4f addresses event simulation requirements at e + e − colliders such as LEP and upcoming linear colliders. Most of the attractive aspects of grc4f come from its link to the GRACE system: a Feynman diagram automatic computation system. The GRACE system has been used to produce the computational code for all final states, giving a higher level of confidence in the calculation correctness. Based on the helicity amplitude calculation technique, all fermion masses can be kept finite and helicity information can be propagated down to the final state particles. The phase space integration of the matrix element gives the total and differential cross sections, then unweighted events are generated. Initial state radiation (ISR) corrections are implemented in two ways, one is based on the electron structure function formalism and the second uses the parton shower algorithm called QEDPS. The latter can also be applied for final state radiation (FSR) though the interference with the ISR is not yet taken into account. Parton shower and hadronization of the final quarks are performed through an interface to JETSET. Coulomb correction between two intermediate W's, anomalous coupling as well as gluon contributions in the hadronic processes are also included.

Systematic testing and formal verification to validate reactive programs

The use of systematic testing and formal verification in the validation of reactive systems implemented in synchronous languages is illustrated. Systematic testing and formal verification are two techniques for checking the consistency between a program and its specification. The approach to validation is through specification: two system views are developed in addition to the program, a behavioural specification for systematic testing and a logical specification for formal verification. Pursuing both activities, reactive programs can be validated both more efficiently (in terms of costs) and more effectively (in terms of confidence in correctness). This principle is demonstrated here using the well known lift example.

The development of anti-hepatitis C virus agents

We have built a model of the specificity pocket of the protease of hepatitis C virus on the basis of the known structures of trypsin-like serine proteases and of the conservation pattern of the protease sequences among various hepatitis C strains. The model allowed us to predict that the substrate of this protease should have a cysteine residue in position P1. This hypothesis was subsequently proved by N-terminal sequencing of two products of the protease. The success of this “blind” test increases our confidence in the overall correctness of our proposed alignment of the enzyme sequence with those of other proteases of known structure and constitutes a first step in the construction of a complete model of the viral protease domain.

Molecular model of the specificity pocket of the hepatitis C virus protease: implications for substrate recognition.

We have built a model of the specificity pocket of the protease of hepatitis C virus on the basis of the known structures of trypsin-like serine proteases and of the conservation pattern of the protease sequences among various hepatitis C strains. The model allowed us to predict that the substrate of this protease should have a cysteine residue in position P1. This hypothesis was subsequently proved by N-terminal sequencing of two products of the protease. The success of this "blind" test increases our confidence in the overall correctness of our proposed alignment of the enzyme sequence with those of other proteases of known structure and constitutes a first step in the construction of a complete model of the viral protease domain.

Structure, multiple site binding, and segmental accommodation in thymidylate synthase on binding dUMP and an anti-folate

The structure of Escherichia coli thymidylate synthase (TS) complexed with the substrate dUMP and an analogue of the cofactor methylenetetrahydrofolate was solved by multiple isomorphous replacement and refined at 1.97-A resolution to a residual of 18% for all data (16% for data greater than 2 sigma) for a highly constrained structure. All residues in the structure are clearly resolved and give a very high confidence in total correctness of the structure. The ternary complex directly suggests how methylation of dUMP takes place. C-6 of dUMP is covalently bound to gamma S of Cys-198(146) during catalysis, and the reactants are surrounded by specific hydrogen bonds and hydrophobic interactions from conserved residues. Comparison with the independently solved structure of unliganded TS reveals a large conformation change in the enzyme, which closes down to sequester the reactants and several highly ordered water molecules within a cavernous active center, away from bulk solvent. A second binding site for the quinazoline ring of the cofactor analogue was discovered by withholding addition of reducing agent during crystal storage. The chemical change in the protein is slight, and from difference density maps modification of sulfhydryls is not directly responsible for blockade of the primary site. The site, only partially overlapping with the primary site, is also surrounded by conserved residues and thus may play a functional role. The ligand-induced conformational change is not a domain shift but involves the segmental accommodation of several helices, beta-strands, and loops that move as units against the beta-sheet interface between monomers.

No IFs, ANDs, or ORs: A study of database querying

The difficulty of expressing database queries was examined as a function of the language used. Two distinctly different query methods were investigated. One used a standard database query language, SQL, requiring users to express an English query using a formal syntax and appropriate combinations of boolean operators. The second used a newly designed Truth-table Exemplar-Based Interface (TEBI), which only required subjects to be able to choose examplars from a system-generated table representing a sample database. Through users' choices of critical exemplars, the system could distinguish between interpretations of an otherwise ambiguous English query. Performance was measured by number correct, time to complete queries, and confidence in query correctness. Individual difference analyses were done to examine the relationship between subjects' characteristics and ability to express database queries. Subjects' performance was observed to be both better, and more resistant to variability in age and levels of cognitive skills, when using TEBI than when using SQL to specify queries. Possible reasons for these differences are discussed.

A Mathematical Framework for the Investigation of Testing

Testing has long been in need of mathematical underpinnings to explain its value as well as its limitations. This paper develops and applies a mathematical framework that 1) unifies previous work on the subject, 2) provides a mechanism for comparing the power of methods of testing programs based on the degree to which the methods approximate program verification, and 3) provides a reasonable and useful interpretation of the notion that successful tests increase one's confidence in the program's correctness.

Recognition Memory for Visually Presented Ambiguous Sentences

Summary Three experiments examined characteristics of recognition memory for visually presented ambiguous sentences. Ss were male and female college students (Experiment I, N = 118; Experiment II, N = 48; Experiment III, N = 180). In each experiment, Ss saw stimulus sentences that were either ambiguous or nonambiguous. They then judged the meaning of comparison sentences that were either correct or incorrect paraphrases of the stimulus sentences. Both immediate (Experiments II and III) and delayed (Experiments I and III) retention intervals were used. Both judgments of confidence in paraphrase correctness (Experiment I) and dichotomous choice response of paraphrase correctness (Experiment II and III) were obtained. The results extend understanding of the single-meaning model and suggest the importance of surface characteristics in recognition memory.

Use of subjective probability in decision making

After discussing general problems concerning theoretical models of decision making, the author divided the experimental situations (on the basis of the quantity of information or experience) into 3 categories: (a) exactly known objective probability of individual events, (b) objective probability approximately estimated, (c) completely uncertain situation without the possibility of determining objective probability. The main research problems are: (a) extrapolation of verified findings (from the viewpoints of experimental methods used, experimental subjects, utility of alternatives), (b) criteria of correct decisions (adequacy of the decision made, latency time, subjective confidence of correctness, etc.), (c) the role of personality characteristics in decision making and (d) comparative research of various experimental conditions.