Model-based Evaluation Method Research Articles

Adapted Galvanostatic Charge Method for cell-individual in-situ characterization of PEM fuel cell stacks and systems

Increasingly challenging lifetime requirements for automotive Polymer-Electrolyte-Membrane (PEM) fuel cells call for onboard characterization methods to enable state-of-health (SOH) adaptive operation strategies. In this work, an adapted Galvanostatic Charge Method (GCM) is presented, that allows for cell-individual electrochemical characterization without nitrogen flush, making it system- and potentially onboard-compatible. Key innovation is to utilize the system’s standard shutdown procedure, i.e. oxygen depletion with closed air valves, to create an inert atmosphere in the cathode volume in which the characterization takes place. The electrochemical processes triggered during the characterization with GCM are now embedded in a closed-volume-problem instead of a steady-flow-environment. This causes the hydrogen concentration difference between anode and cathode to be dynamic instead of constant during the characterization. In this work, a 0D-simulation model of the closed-volume-problem and complementary measurements on a system testbench are presented. Additionally, a novel model-based evaluation method is introduced which extracts the SOH-indicators electrochemical active surface area (ECSA), double layer capacity, hydrogen permeability of the membrane and short circuit resistance from the measurements by fitting simulated to measured voltage curves. The simulation model is thoroughly analyzed and a systematic error, evoked by the fluidic coupling of the individual cells, is found to influence the results obtained for membrane permeability. The influence of different initial conditions (among them initial hydrogen concentration) on the evaluation results is experimentally investigated and found to have no significant influence. For further studies, improvement potential was found in modeling, measurement procedure and evaluation method.

Maximum Exposure Evaluation for MIMO Based on Optimization Algorithm

IEC/IEEE FDIS 63195-2 requires the maximum exposure evaluation to determine if the product meets the radiation limit requirements specified by ICNIRP. The miniaturization and integration of millimeter wave antennas make it possible to deploy antenna arrays supporting Multiple-Input Multiple-Output (MIMO) technology in mobile terminals, which brings challenges to traditional exposure compliance testing. This is caused by the complex electromagnetic environment generated around the equipment by beam focusing technology and the difficulty in determining the maximum exposure of electromagnetic exposure dose by traditional measurement methods. However, mathematical model-based evaluation methods may produce overly conservative results, limiting the maximum allowable transmit power of 5G devices and resulting in wasted system performance. In this study, we take advantage of the optimization algorithm in searching for globally optimal solutions under multivariate constraints, and its search strategy is improved to evaluate the largest exposure of millimeter wave MIMO antenna arrays in the target region.

FFCI: A Framework for Interpretable Automatic Evaluation of Summarization

In this paper, we propose FFCI, a framework for fine-grained summarization evaluation that comprises four elements: faithfulness (degree of factual consistency with the source), focus (precision of summary content relative to the reference), coverage (recall of summary content relative to the reference), and inter-sentential coherence (document fluency between adjacent sentences). We construct a novel dataset for focus, coverage, and inter-sentential coherence, and develop automatic methods for evaluating each of the four dimensions of FFCI based on cross-comparison of evaluation metrics and model-based evaluation methods, including question answering (QA) approaches, semantic textual similarity (STS), next-sentence prediction (NSP), and scores derived from 19 pre-trained language models. We then apply the developed metrics in evaluating a broad range of summarization models across two datasets, with some surprising findings.

Data-driven and physical model-based evaluation method for the achievable demand response potential of residential consumers' air conditioning loads

Demand response (DR) of thermostatically controlled load (TCL) represented by residential consumers' air conditioning (AC) is playing an increasingly important role in enhancing the operational flexibility and sustainability of the power grid. The evaluation of the DR potential of AC is of great significance in the power grid dispatching and the selection of DR target consumer groups. However, due to the large number of AC units, the large difference in parameters, and the randomness and uncertainty of consumers' electricity consumption behavior and DR willingness, it is difficult to accurately evaluate the achievable potential of the AC. In order for power system operator (PSO) or load aggregator (LA) to better mine and manage the AC resources, this paper proposes a systematic method suitable for evaluating the achievable DR potential of a single residential AC and aggregate residential ACs the next day. Firstly, based on the parameter identification of equivalent thermal parameter (ETP) model, a method for estimating the theoretical time-varying potential of the AC is proposed. Then considering the influence of the randomness of residents' electricity consumption behavior on the DR potential, an AC state prediction model is constructed based on the deep learning network. On this basis, social psychology and fuzzy systems are used to quantify the impact of residential consumers' DR willingness on the DR potential, and an achievable potential evaluation model is constructed. Finally, based on the actual load data of the Muller project in Austin, the effectiveness and accuracy of the proposed DR potential evaluation method are further demonstrated.

Rapid evaluation of the mechanical fault severity in induction motors using the model‐based diagnosis technique

The motor current signal analysis (MCSA) technique is widely used as a non-invasive method for detecting mechanical faults in induction motors by capturing characteristic components in the stator line current. However, the threshold of the characteristic component is not clear now, which makes it difficult for MCSA to judge whether the fault occurs or evaluates the mechanical fault severity. The existing model-based evaluation methods cannot meet the requirement of online condition monitoring because of their slow calculation. To solve these problems, a simplified dynamic motor model under any type of mechanical fault is established, and a formula for the amplitude relationship between the radial vibration of the rotor and fault-related component in the stator line current is derived. The radial vibration amplitude is related to mechanical fault severity. Using this formula, the MCSA technique can rapidly evaluate the mechanical fault severity according to the amplitude of the characteristic component in the collected stator current. The simulation study results demonstrate the accuracy of the simplified model and formula. The experimental results presented for condition monitoring in a real induction motor clearly validate the evaluation approach.

Reverse osmosis membrane element integrity evaluation using imperfection model

The operational efficacy of membrane separation processes such as reverse osmosis (RO) and nanofiltration (NF) can be compromised by the lack of mechanical or chemical integrity of the membrane element. Currently, there are several methods applied to analyse membrane and membrane element integrity. However, given that most of them are somewhat inaccurate, expensive or time consuming, the motivation of this study was to implement a simple model-based evaluation method for a single element. The approach was built based on the solution-diffusion-imperfection model and its subsequent revised versions of this model. The models were fitted to experimental data obtained for three types of elements: new, damaged under controlled laboratory conditions and damaged during full scale operation. Based on the experimental results and the fitting of two model parameters, which are the leakage factor and the solute permeability, a criterion for diagnosis of the damage type was formulated. The results confirm the applicability of the modeling approaches. Finally, a decision tree is presented with the aim to help the plant operator to diagnose the type of damage the membrane element suffers.

Evaluating Computational Language Models with Scaling Properties of Natural Language

In this article, we evaluate computational models of natural language with respect to the universal statistical behaviors of natural language. Statistical mechanical analyses have revealed that natural language text is characterized by scaling properties, which quantify the global structure in the vocabulary population and the long memory of a text. We study whether five scaling properties (given by Zipf’s law, Heaps’ law, Ebeling’s method, Taylor’s law, and long-range correlation analysis) can serve for evaluation of computational models. Specifically, we test n-gram language models, a probabilistic context-free grammar, language models based on Simon/Pitman-Yor processes, neural language models, and generative adversarial networks for text generation. Our analysis reveals that language models based on recurrent neural networks with a gating mechanism (i.e., long short-term memory; a gated recurrent unit; and quasi-recurrent neural networks) are the only computational models that can reproduce the long memory behavior of natural language. Furthermore, through comparison with recently proposed model-based evaluation methods, we find that the exponent of Taylor’s law is a good indicator of model quality.

Simple two-step assessment of novel adsorbents for drying: The trade-off between adsorber size and drying time

Adsorptive drying systems aim at short drying times and small packaging. Thus, fast heat and mass transfer and high uptakes of the adsorbent are required. For this purpose, novel adsorbents are developed. To evaluate novel adsorbents for drying applications, we propose a simple two-step model-based method using equilibrium data and a dynamic process model. Based on a case study for an adsorption dishwasher, we determine the trade-off between adsorber unit size and drying times for Zeolite 13X, Silica Gel 125, SWS-1L and SBA-15 E120. For the adsorption dishwasher, SWS-1L is found to be a very promising adsorbent. Employing SWS-1L instead of Zeolite 13X in an adsorption dishwasher reduces the amount of adsorbent by 50%. Even though the adsorbent mass is lower, the drying speed is still the fastest among the studied adsorbents. The proposed two-step model-based evaluation method enables the identification of promising adsorbents for adsorptive drying systems.

Mathematical model-based evaluation methodology for environmental burden of chemical mechanical planarization process

Chemical mechanical planarization (CMP) is an essential manufacturing process in semiconductor fabrication. Chipmakers continue to adopt CMP for device planarization or surface finishing of substrate materials. Evaluating the environmental impact of the CMP process may contribute to the greening of the semiconductor process. In this paper, we propose a mathematical model-based evaluation method to determine the environmental burden of the CMP process. We adopted our previously reported material removal rate (MRR) model for CMP and modified it to incorporate the effect of the slurry flow rate and process temperature. The established model was compared with the experimental results. The environmental burden of the CMP process was evaluated by converting the electric energy consumption, slurry consumption, and ultrapure water (UPW) consumption into their carbon dioxide equivalents (CDEs). The results showed that the slurry consumption strongly impacted the CDE of the CMP process. The results of this study may help optimize the process parameters for a sustainable CMP process.

SimGine: A simulation engine for stochastic discrete-event systems based on SDES description

Discrete-event systems have gained a lot of interest due to their wide range of applications, and discrete-event simulation is a useful method for the performance evaluation of such systems. In this domain, model-based evaluation methods play an important role and there are many formalisms and realistic experiments using these methods. In this paper, we introduce SimGine, a multi-formalism simulation engine for stochastic discrete-event systems based on SDES, which is a unified abstract description for stochastic discrete-event systems. The engine is also capable of rare-event simulation of models using the importance sampling technique, which makes it the first multi-formalism simulation tool with rare-event simulation capability. The XML-based input language of SimGine allows for definition of the required methods. The body of each method is expressed by codes in a high-level programming language and this provides a powerful and flexible approach for defining events with complex behavior. For the simulation of an existing model, a tool for translating models into the SimGine input language should be prepared. SimGine can be used as a stand-alone simulation tool or as a simulation engine in other tools.

Evaluation of Viscoelastic Property of Articular Cartilage Based on Mechanical Model Considering Tissue Microstructure

The mechanical behavior of articular cartilage is strongly dependent on its microstructure characterized by three-dimensional, depth-dependent densities and anisotropic arrangements of cartilage cells and collagen fibres in extracellular matrix. The inclusion of cartilage microstructure to model analysis is essential for the accurate evaluation of the mechanical property. This study proposes a model-based evaluation method, which could predict tissue microstructure (collagen fibres, proteoglycans, and cartilage cells) based on a time history of reaction force obtained from indentation test, thereby estimating the depth-dependent cartilage mechanical property that is difficult to be estimated by experiment in conventional methods. The model was used for evaluating the viscoelasticities of cartilage at medial (load-supporting) and lateral (non load-supporting) regions on bovine femoral head. The present simulation allows fitting the time-history of reaction force measured from indentation test (R2 = 0.993 ± 0.0012 for lateral region and R2 = 0.990 ± 0.0032 for medial region). The differences of the elastic modulus of collagen fibre and the permeability between medial and lateral regions were consistent to those observed from the chemical composition analysis.

Evaluation of plant alarm systems by behavior simulation using a virtual subject

To design an effective plant alarm system, it is essential to evaluate various performances of fault detection and identification (FDI) in emergencies. In this study, an operator model, which mimics the FDI behavior of a human operator with primary cognitive and executive capabilities, is developed as a virtual subject for supervising a chemical plant system. Analyzing the FDI tracks generated by the behavior simulation using the virtual subject under an alarm system makes it possible to evaluate the performance of the alarm system. In a case study, the evaluation method using the virtual subject is applied for designing an appropriate alarm system in a boiler-plant simulator. FDI performances of the original alarm system for all assumed malfunctions are evaluated and alarm signal selection and limit settings are reconsidered according to the evaluation results. The results of the case study demonstrate the usefulness of the model-based evaluation method.

Performability: a retrospective and some pointers to the future

As computing and communication systems become physically and logically more complex, their evaluation calls for continued innovation with regard to measure definition, model construction/solution, and tool development. In particular, the performance of such systems is often degradable, i.e., internal or external faults can reduce the quality of a delivered service even though that service, according to its specification, remains proper (failure-free). The need to accommodate this property, using model-based evaluation methods, was the raison d'être for the concept of performability. To set the stage for additional progress in its development, we present a retrospective of associated theory, techniques, and applications resulting from work in this area over the past decade and a half. Based on what has been learned, some pointers are made to future directions which might further enhance the effectiveness of these methods and broaden their scope of applicability.

New Naval Hospital Facilities

To design an effective plant alarm system, it is essential to evaluate various performances of fault detection and identification (FDI) in emergencies. In this study, an operator model, which mimics the FDI behavior of a human operator with primary cognitive and executive capabilities, is developed as a virtual subject for supervising a chemical plant system. Analyzing the FDI tracks generated by the behavior simulation using the virtual subject under an alarm system makes it possible to evaluate the performance of the alarm system. In a case study, the evaluation method using the virtual subject is applied for designing an appropriate alarm system in a boiler-plant simulator. FDI performances of the original alarm system for all assumed malfunctions are evaluated and alarm signal selection and limit settings are reconsidered according to the evaluation results. The results of the case study demonstrate the usefulness of the model-based evaluation method.