Conformance Testing Research Articles

Assessing the coverage of W-based conformance testing methods over code faults

We present novel empirical assessments of prominent finite state machine (FSM) conformance test derivation methods against their coverage of code faults. We consider a number of realistic extended FSM examples with their related Java implementations and derive for these examples complete test suites using the W method and its HSI and H derivatives considering the case when the implementation under test (IUT) has the same number of states as the specification FSM. We also consider W++, HSI++, and H++ test suites derived considering the case when the IUT can have one more extra state. For each pair of considered test suites, we determine if there is a difference between the pair in covering the implementations faults. If the difference is significant, we determine which test suite outperforms the other. We run two other assessments which show that the obtained results are not due to the size or length of the test suites. In addition, we conduct assessments to determine whether each of the methods has better coverage of certain classes of faults than others and whether the W outperforms the HSI and H methods over only certain classes of faults. The results and outcomes of conducted experiments are summarized. Major artifacts used in the assessments are provided as benchmarks for further studies.

TestShark: A Passive Conformance Testing System for ISO 15118 Using Wireshark

ISO 15118 is an international standard for charging communication between electric vehicles and charging infrastructure. Among the series of ISO 15118, ISO 15118-4, ISO 15118-5, ISO 15118-9, and ISO 15118-21 (under development) define active conformance testing methodologies where the test system engages with the system under test as a primary actor and tests whether the counterparty is complying with the standard. However, such a full-fledged test system is not appropriate during system development and field testing. In this paper, a passive conformance testing approach that monitors the packets between the electric vehicle and the charger and performs conformance testing against the requirements of ISO 15118 is proposed. This novel approach exhibits practicality yet comprehensive testing capability for field testing because of its passive access to the packets with deep analysis for the conformance. TestShark, a modified packet analysis tool, showed that the passive conformance testing system can help developers and operators analyze problems in charging communication and assess the quality of implementation with respect to ISO 15118.

Exploring the Use of AI for Enhanced Accessibility Testing of Web Solutions.

Artificial Intelligence (AI) holds significant potential for enhancing accessibility and user experience across digital products and services. However, mainstream web solutions commonly used by the general population still face accessibility barriers, hindering equal participation in the information society for people with disabilities. This article explores several promising applications of AI that can be used to create accessible solutions for people with disabilities. We also present our research, which aims to explore and demonstrate how various AI-based techniques can enhance and streamline accessibility testing for web solutions. We selected four success criteria from the Web Content Accessibility Guidelines (WCAG) that currently require extensive manual work and developed four prototypes using open-source machine-learning models to enhance conformance testing. While the prototypes need further optimization and evaluation, the results suggest that AI-based techniques can significantly reduce the need for manual work in accessibility testing.

A Coq Mechanization of JavaScript Regular Expression Semantics

We present an executable, proven-safe, faithful, and future-proof Coq mechanization of JavaScript regular expression (regex) matching, as specified by the latest published edition of ECMA-262 section 22.2. This is, to our knowledge, the first time that an industrial-strength regex language has been faithfully mechanized in an interactive theorem prover. We highlight interesting challenges that arose in the process (including issues of encoding, corner cases, and executability), and we document the steps that we took to ensure that the result is straightforwardly auditable and that our understanding of the specification aligns with existing implementations. We demonstrate the usability and versatility of the mechanization through a broad collection of analyses, case studies, and experiments: we prove that JavaScript regex matching always terminates and is safe (no assertion failures); we identify subtle corner cases that led to mistakes in previous publications; we verify an optimization extracted from a state-of-the-art regex engine; we show that some classic properties described in automata textbooks and used in derivatives-based matchers do not hold in JavaScript regexes; and we demonstrate that the cost of updating the mechanization to account for changes in the original specification is reasonably low. Our mechanization can be extracted to OCaml and JavaScript and linked with Unicode libraries to produce an executable regex engine that passes the relevant parts of the official Test262 conformance test suite.

The Relationship between the Level of Satisfaction of Blood Delivery Workers with Five Dimensions of Service Quality at UDD PMI Bekasi Regency in 2023

(1) Hospital Blood Bank (BDRS) is one of the health service facilities to meet the needs of blood transfusions of patients but does not rule out the possibility that blood stocks in BDRS are not available so they require assistance from the Indonesian Red Cross Blood Donor Unit (UDD PMI) in meeting the blood needs of patients. UDD PMI Bekasi Regency is carried out almost every time, regardless of the time and weather at that time. Blood collection is usually done by special officers from the hospital. This officer is what connects the chain of blood requests BDRS to UDD PMI Bekasi Regency, where later the officer will carry out blood retention and memory. (2) This research aims to determine the level of ability to use five dimensions quality services to have reliability, response, empathy, and blood transmission in UDD PMI District Bekasii. (3) Methodology This study by distributing questionnaires to 52 blood delivery officer respondents, data processing using data analysis of satisfaction index and ordinal logistic regression, namely simultaneous tests, partial tests, and model conformity tests (Hosmer-Lemeshow Goodness-of-fit Test). (4) The results of this study show that the level of satisfaction of blood delivery officers is based on five dimensions of satisfaction with UDD PMI district services. Bekasi has very satisfactory criteria.

Conformal test martingales for hypergraphical models

Conformal test martingales for hypergraphical models

Validation and Conformity Testing of Sentinel-3 Green Instantaneous FAPAR and Canopy Chlorophyll Content Products

This article presents validation and conformity testing of the Sentinel-3 Ocean Land Colour Instrument (OLCI) green instantaneous fraction of absorbed photosynthetically active radiation (FAPAR) and OLCI terrestrial chlorophyll index (OTCI) canopy chlorophyll content (CCC) products with fiducial reference measurements (FRM) collected in 2018 and 2021 over two sites (Las Tiesas—Barrax, Spain, and Wytham Woods, UK) in the context of the European Space Agency (ESA) Fiducial Reference Measurement for Vegetation (FRM4Veg) initiative. Following metrological principles, an end-to-end uncertainty evaluation framework developed in the project is used to account for the uncertainty of reference data based on a two-stage validation approach. The process involves quantifying uncertainties at the elementary sampling unit (ESU) level and incorporating these uncertainties in the upscaling procedures using orthogonal distance regression (ODR) between FRM and vegetation indices derived from Sentinel-2 data. Uncertainties in the Sentinel-2 data are also accounted for. FRM-based high spatial resolution reference maps and their uncertainties were aggregated to OLCI’s native spatial resolution using its apparent point spread function (PSF). The Sentinel-3 mission requirements, which give an uncertainty of 5% (goal) and 10% (threshold), were considered for conformity testing. GIFAPAR validation results revealed correlations > 0.95, RMSD ~0.1, and a slight negative bias (~−0.06) for both sites. This bias could be partly explained by the differences in the FAPAR definitions between the satellite product and the FRM-based reference. For the OTCI-based CCC, leave-one-out cross-validation demonstrated correlations > 0.8 and RMSDcv ~0.28 g·m−2. Despite the encouraging validation results, conclusive conformity with the strict mission requirements was low, with most cases providing inconclusive results (driven by large uncertainties in the satellite products as well as by the uncertainties in the upscaling approach). It is recommended that mission requirements for bio-geophysical products are reviewed, at least at the threshold level. It is also suggested that the large uncertainties associated with the two-stage validation approach may be avoided by directly comparing with spatially representative FRM.

Frequency selective surface with quad-band absorption/transmission and angular stability for TE and TM polarizations

The proposed FSS structure employs two square annular rings for absorption functionalities and two square aperture slots for transmission properties. Through comprehensive simulations and analysis, the FSS structure demonstrates remarkable performance in selectively absorbing and transmitting electromagnetic waves across four distinct frequency bands. Moreover, the design exhibits excellent TE and TM polarizations angular stability, consistently maintaining constant absorption and transmission characteristics across varying incidence angles. A comprehensive analysis of the equivalent circuit model of the FSS structure has also been performed to understand the quad-band transmission and absorption properties. The measurement results underscore the effectiveness of the design of a 250*250 mm array, with absorption peaks notably occurring at 4.2 GHz and 7.23 GHz, boasting peak absorption rates of 92% and 85%, respectively. Moreover, the insertion loss within the pass band at 8.8 GHz and 12.2 GHz stands impressively low at 0.2 dB and 0.32 dB, respectively. Notably, the proposed quad-band FSS showcases polarization-insensitive behavior across a range from 0° to 60° and successfully undergoes conformal testing with a curvature radius up to 50 mm. The results of experimental testing on a quad-band FSS prototype demonstrate superior agreement with those obtained through simulations and analytically derived equivalent circuit model (ECM) results, validating its practical applicability in real-world scenarios.

Test of conformance or non-conformance with geometrical specifications

Abstract Systematic and random measurement errors are the cause of uncertain conformance and non-conformance statements. A wrong conformance statement occurs if the workpiece is accepted and is a reject part (type II error or false-negative) and a wrong non-conformance statement if the workpiece is rejected and is an in-spec part (type I error or false-positive). In order to avoid type I and type II errors, measurement uncertainty must be taken into account in the conformance and non-conformance testing. In practice, some procedures are used to consider measurement errors or the uncertainty that deviate from the state-of-the-art in research and technology. As these methods have become established over many years, they are still widely used despite better theoretical knowledge. The standard ISO 14253-1:2017 specifies a procedure based on probability and measurement uncertainty that is in accordance to the internationally accepted “Guide to the expression of uncertainty in measurement” and its supplements but is often not used due to the complexity of the evaluation of measurement uncertainty. In this contribution we give an overview and comparison of the different existing methods and provide an suggestion for supplementing the standard ISO 14253-1:2017, as Monte Carlo simulations enable a direct probability-based conformance and non-conformance testing even for complex measurement processes.

Utilising LANDHYPERNET data products over a deciduous broadleaf forest to validate Sentinel-2 and Landsat surface reflectance products

This paper demonstrates the use of a novel, autonomous hyperspectral surface reflectance data collected at Wytham Woods, United Kingdom (WWUK) as part of the LANDHYPERNET network for the validation of multispectral data from Sentinel-2, Landsat 8 and Landsat 9. The deployment of the HYPSTAR instrument at the site and the corresponding quality control of the data products is described. In addition, a methodology based upon metrological principles is outlined showing the propagation of uncertainties from the LANDHYPERNET and satellite data products to enable conformity testing of the satellite products using the satellite mission requirements. A total of 9 matchups are found for satellite validation at the site, where there is a cloud-free satellite scene and a corresponding LANDHYPERNET sequence, which has passed all quality checks, within two hours of the overpass. An analysis of the impact of the spatial variability of the site is presented and can account for up to 40% of the uncertainty associated with the in-situ surface reflectance data. There is no systematic bias in the Bottom-Of-Atmosphere reflectance data obtained from the LANDHYPERNET data in comparison to the satellite data. In the best case, differences of less than 2% are found for certain spectral bands. However, in the worst cases, relatively large differences are found which exceed 100%, this is affected by the relatively low reflectance values found in the visible bands. These differences could be caused by the spatial and temporal mismatch between the in-situ and satellite measurement, or due to shadowing caused by the flux tower. Further data quality control and assurance is needed to best choose data sets suitable for satellite validation. Incorporating a Bidirectional Reflectance Distribution Function model into the processing chain for the forest canopy is recommended. Overall, although there are areas to further characterise, the site provides a useful benchmark for which to develop techniques for validation of satellite surface reflectance products over a challenging environment.

Conformal Coating Testing in Various Test Environments

Conformal coatings have traditionally been tested by determining the mean time to failure of conformally coated hardware exposed to corrosive test environments. This test approach has serious shortcomings: The test temperatures are most often too high. At these high temperatures, the conformal coating properties may be quite different from those at the application temperatures. In addition, the times to failure are unacceptably long extending into many months. Overcoming these shortcomings is an iNEMI championed test that involves exposing conformally coated thin films of copper and silver to sulfur vapors at 40-50 oC in flowers of sulfur (FoS) chamber and using the corrosion rates of the coated metal thin films as a measure of the corrosion protection capabilities of the conformal coatings. The test temperatures are similar to the application temperatures, the test durations are no more than a week and can be conducted under various temperature and humidity conditions. The purpose of this paper was to determine if testing in the industry-standard mixed-flowing gas corrosion chamber would give similar results as those using the FoS chamber. Acrylic, fluorinated acrylate, and atomic layer deposition conformal coatings were tested in three environments: (a) flowers of sulfur (FoS), (b) mixed-flowing gas (MFG), and (c) iodine vapor. The performance of the coatings tested in the FoS and the MFG corrosion chambers were quantitatively similar. The iodine vapor test results were in qualitative agreement with the FoS and MFG test results. In addition, we present early results pointing to the utility of terahertz-frequency imaging as a technique for measuring conformal-coating thickness nondestructively.

Comparative Analysis of the Combination of MOORA and GRA with PIPRECIA Weighting in the Selection of Warehouse Heads

The Head of Warehouse has the main responsibility to manage and supervise the daily operations of the warehouse, including receiving, storing, and dispensing goods. In addition, the warehouse head is responsible for leading the warehouse staff team, providing the necessary direction, training, and supervision to ensure efficiency and safety in warehouse operations. This study aims to select warehouse heads by applying a combination of MOORA and PIPRECIA methods, as well as GRA and PIPRECIA. The combination of MOORA and PIPRECIA as well as GRA and PIPRECIA will be compared based on the final ranking of the two methods used. Based on the ranking results on the MOORA Method, the highest value with a value of 0.40017 was obtained by FHY with rank 1, a value of 0.36637 was obtained by AGL with rank 2, and a value of 0.35721 was obtained by YLS with rank 3 Based on the rating results on the GRA Method, the highest value with a value of 0.12164 was obtained by AGL with rank 1, a value of 0.12017 was obtained by FHY with rank 2, and a value of 0.1054 obtained by TAN with rank 3. Ranking differences between Multi-Method Objective Optimization by Ratio Analysis (MOORA) and Gray Relational Analysis (GRA) can arise due to differences in approaches in evaluating and comparing alternatives. The results of the conformity test can be concluded in the selection of warehouse heads who recommend the GRA method compared to the MOORA method, because the results of the conformity level of the GRA method get a value of 99.99919% higher than the MOORA method.

Benchmarking Combinations of Learning and Testing Algorithms for Automata Learning

Automata learning enables model-based analysis of black-box systems by automatically constructing models from system observations, which are often collected via testing. The required testing budget to learn adequate models heavily depends on the applied learning and testing techniques. Test cases executed for learning (1) collect behavioural information and (2) falsify learned hypothesis automata. Falsification test-cases are commonly selected through conformance testing. Active learning algorithms additionally implement test-case selection strategies to gain information, whereas passive algorithms derive models solely from given data. In an active setting, such algorithms require external test-case selection, like repeated conformance testing to extend the available data. There exist various approaches to learning and conformance testing, where interdependencies among them affect performance. We investigate the performance of combinations of six learning algorithms, including a passive algorithm, and seven testing algorithms by performing experiments using 153 benchmark models. We discuss insights regarding the performance of different configurations for various types of systems. Our findings may provide guidance for future users of automata learning. For example, counterexample processing during learning strongly impacts efficiency, which is further affected by testing approach and system type. Testing with the random Wp-method performs best overall, while mutation-based testing performs well on smaller models.

Characterization of Ti-6Al-4V Bar for Aerospace Fastener Pin Axial Forging

Ti-6Al-4V warm forged fasteners are a critical part of the aerospace industry, as they are used in vast quantities for mechanical joining of components for the fuselage, wing-skin and aero-engine. These components are produced in vast quantities at rapid production rates through multi-blow axial forging However the rate that they are manufactured means that manufacturers rely upon periodic part conformance testing to understand if the part is within tolerance or if any undesirable manufacturing defects such as cracks or underfilling are present. Thus, a right-first-time manufacturing approach is essential to minimize non-conformant scrap. An analysis of the Ti-6Al-4V supplied raw material for axial forging, in a variety of different bar diameter sizes and from different industrial suppliers, was conducted. This was to attempt to understand whether material property variation or operator variation was the root cause for some material behaving differently during the manufacture route. Experimental testing was performed through microstructure characterization and mechanical testing methods. The volume fraction of the β-phase was noted to be marginally higher in material with good forgeability. The hardness of the inner core of the bar appears to be a critical material property for the Ti-6Al4V bar, with an overly hard bar-core hindering forgeability of the bar. This is believed to be due to the hotter central region malleability being key for forgeability. Micro-void porosity was also noted which could lead to stress concentration locations, or crack initiation, and as such is a deleterious property for forgeability. The experienced forgeability of the Ti-6Al-4V bars have been demonstrated to be sensitive to rather small variation in measured microstructure and mechanical property. It is believed that cumulative impacts of small differences, 1% variation in α-phase volume fraction, small variations in elongation to failure, 1% variation in elastic modulus and microhardness profile variation at the center of the bar of less than 10 HV0.3, can combine to significantly impact the forgeability of Ti-6Al-4V bar.

Grid‐forming converters in interconnected power systems: Requirements, testing aspects, and system impact

AbstractIn this study, the integration of grid‐forming (GFM) converters in power systems is discussed in terms of both the fundamental aspects of system stability and the technical possibilities of converter‐based resources. The paper provides a survey and comparison of various GFM control concepts with respect to their transient and stationary behavior. A method for black box characterization and conformity testing of GFM converters is presented and relevant results of lab and field tests with GFM converters are discussed. Finally, important system aspects are addressed. This includes an approach to consider the behavior of GFM converters in distribution grid models and investigations on the minimum share of GFM units required to ensure system stability. The findings of this paper provide valuable insights for the design and implementation of GFM converters in modern converter‐based power systems.

Multiple conformity tests to assess deviations from the Newcomb-Benford Law (NBL): A replication of Koch and Okamura (2020)

In this paper, we critically reevaluate Koch and Okamura’s (2020) conclusions on the conformity of Chinese COVID-19 data with Benford’s Law. Building on Figueiredo et al. (2022), we adopt a framework that combines multiple tests, including Chi-square, Kolmogorov-Smirnov, Euclidean Distance, Mean Absolute Deviation, Distortion Factor, and Mantissa Distribution. The primary rationale behind employing multiple tests is to enhance the robustness of our inference. The main finding of the study indicates that COVID-19 infections in China do not adhere to the distribution expected under Benford’s Law, nor does it align with the figures observed in the U.S. and Italy. The usefulness of deviations from Benford’s Law in detecting misreported or fraudulent data remains controversial. However, addressing this question requires a more careful statistical analysis than what is presented in the Koch and Okamura (2020) paper. By employing a combination of several tests using fully transparent procedures, we establish a more reliable approach to evaluating conformity to the Newcomb-Benford Law in applied research.

Empirical Equation between Predominant Frequency and Sediment Thickness in Plampang, West Nusa Tenggara

Soil characteristics mapping is one of the essential aspects in the development of a region. Soil characteristics such as the predominant frequency and thickness of the sediment layer are needed to ensure the capacity of soil against the load of an infrastructure construction. The existence of tectonic activity causes West Nusa Tenggara Province to be an area with high seismic activity. Along with the plan to accelerate the development of the Samota Area, seismic studies are one of the aspects that determine project feasibility. The predominant frequency value from the results of microtremor measurements is used in the development of an empirical formula to predict soil thickness in the Plampang area. The empirical formula from previous research is used as a conformity test of the empirical formula obtained. Empirical equations are then used to map variations in sediment thickness in the Plampang area, which are justified based on geological data. The results of the sediment thickness map based on the empirical formula for the Plampang area showed a good correlation with the predominant frequency and also showed a good correlation with the results of the formula that has been carried out by previous studies.

Persepsi dan Keputusan Wisatawan dalam Membeli Produk Berlabel Green Product di Desa Wisata Panglipuran, Bali

The purpose of this study is to analyze the influence of tourist perceptions and decisions on the purchase of green products labeled products in Panglipuran Bali Tourism Village and to determine the influence of tourist decisions on and purchases of green products labeled products in Penglipuran Bali Tourism Village, in this study using quantitative research using data analysis from software from smartPLS 4.0, then in testing conducted by researchers among others as follows, the number of latent variables in this study is 4 variables, namely Perception (PR), Tourist Decision (TD) is an endogenous variable, while exogenous variables are Purchase (PMB) and Green Products. (GP). Furthermore, the results of respondents' answers will be processed and analyzed with smartPLS-SEM 4.0 software analysis techniques. The formulas are tested for convergent and discriminant validity, reliability, construct multicollinearity, R Square Test, F-Square Model Conformity Test (F2) and Path Coefficient Test. Based on the results of the hypothesis test, it shows that the hypothesis test in this study was carried out by looking at the t-statistical value and p-value. A research hypothesis is considered acceptable if the t-statistic value is greater than 1.65 and the p value < 0.05. Furthermore, it is known to test the hypothesis that this study shows the results for the results of H1 analysis, namely the perception of a significant effect on the purchase of green product-based products in Panglipuran Tourism Village Bali, by looking at the statistical value of T of 2.3150, then at the P value of 0.0113 smaller than 0.05, in other words that H1 is accepted. Then on H2 about the influence of tourist decisions that have a significant effect on the purchase of products labeled green products in Panglipuran Tourism Village Bali can be seen with a statistical value of T of 2.4835, then for the P veleus value of 0.1201, with the results of this hypothesis test that for H2 is acceptable.

Computer-aided designed 3D-printed polymeric scaffolds for personalized reconstruction of maxillary and mandibular defects: a proof-of-concept study

PurposeTo investigate the potential reconstruction of complex maxillofacial defects using computer-aided design 3D-printed polymeric scaffolds by defining the production process, simulating the surgical procedure, and explore the feasibility and reproducibility of the whole algorithm.MethodsThis a preclinical study to investigate feasibility, reproducibility and efficacy of the reconstruction algorithm proposed. It encompassed 3 phases: (1) scaffold production (CAD and 3D-printing in polylactic acid); (2) surgical simulation on cadaver heads (navigation-guided osteotomies and scaffold fixation); (3) assessment of reconstruction (bone and occlusal morphological conformance, symmetry, and mechanical stress tests).ResultsSix cadaver heads were dissected. Six types of defects (3 mandibular and 3 maxillary) with different degree of complexity were tested. In all case the reconstruction algorithm could be successfully completed. Bone morphological conformance was optimal while the occlusal one was slightly higher. Mechanical stress tests were good (mean value, 318.6 and 286.4 N for maxillary and mandibular defects, respectively).ConclusionsOur reconstructive algorithm was feasible and reproducible in a preclinical setting. Functional and aesthetic outcomes were satisfactory independently of the complexity of the defect.

An application of hybrid Taguchi-ANN to predict tool wear for turning EN24 material

This work is an attempt to predict tool wear for turning EN24 material by the hybrid Taguchi-ANN (Taguchi-Artificial Neural Network) method. The objective is to minimize the tool wear. The independent factors are cutting environment, feed rate, depth of cut, nose radius, and tool type. A Spinner numerical control lathe is used to assess performance. As per the Taguchi orthogonal array, 27 experiments are conducted for each value of the uncontrollable factor (spindle vibration). Optimal setting is structured by Taguchi analysis and the response table. The additive model is used to predict the response. Conformity test is carried out to check whether the predicted and experimental values of response are within the range given by the confidence interval. Furthermore, the ANN is used to predict and analyze the tool wear. The result showed that the supremely important parameter is depth of cut and the least important parameter is tool type. The ideal set found is A3, B3, C3, D1, and E3. Through ANN analysis, it is observed that the experimental values are very close to the predicted values of tool wear. The predicted value at optimal setting is 0.0401 mm. The experimental values at optimal setting is 0.0422 mm. In addition, the study showed that when the feed rate and nose radius are both set to high levels and the depth of cut is medium, using an uncoated tungsten carbide tool with minimal lubrication results in the least amount of tool wear.