Generation Of Performance Models Research Articles

Deep active-learning based model-synchronization of digital manufacturing stations using human-in-the-loop simulation

The effective and accurate modeling of human performance is one of the key technologies in virtual/smart manufacturing systems. However, a significant challenge lies in acquiring sufficient data for such modeling. Virtual Reality (VR) emerges as a promising solution, making human manufacturing experiments more practical and accessible. In this paper, we present a novel framework that efficiently models human assembly duration by leveraging VR to prototype data-acquisition systems for assembly tasks. Central to the framework is an active learning model, which intelligently selects experimental conditions to yield the most informative results, effectively reducing the number of experiments required. As a result, the system demands fewer experimental trials and operates on an automated basis. In VR experiments involving throughput rate, the active model significantly reduces the data requirement, thereby expediting the experiment and modeling process. While this framework demonstrates remarkable efficiency, it does exhibit sensitivity to non-constant noise and may necessitate prior data from similar assembly tasks to identify high-noise. Notably, this proposed method extends beyond manufacturing, allowing the quick generation of human performance models in virtual systems and enhancing experiment scalability across various fields. With its potential to revolutionize human performance modeling, our framework represents a promising avenue for advancing virtual/smart manufacturing systems and other related applications.

Design-time performance modeling of compositional parallel programs

Performance models are powerful instruments for understanding the performance of parallel systems and uncovering their bottlenecks. Already during system design, performance models can help ponder alternative development options. However, creating a performance model – whether theoretically or empirically – for an entire application that does not exist yet is challenging. In this paper, we propose to generate performance models of full programs from performance models of their components using formal composition operators derived from parallel design patterns. As long as the design of the overall system follows such a pattern, its performance model can be predicted with reasonable accuracy without an actual implementation. We demonstrate our approach with design patterns of varying complexity, including pipeline, task pool, and eventually MapReduce, which is representative of a broad class of data-analytics applications.

189 Fundamentals, Common Mistakes, and Graduate Education in Statistics

Abstract Experimental design and statistical data analyses are fundamental components of animal science research. Proper design of experiments and adequate sampling permits testing hypotheses raised by researchers and sets the stage for collecting required data and subsequent statistical analysis. When designing experiments, researchers should respect rules of randomization of treatments to avoid statistical bias and permit proper inference to be drawn. Use of sample sizes that result in adequate statistical power to identify the hypothesized differences among factor levels of interest is key and should be driven by formal processes determining such. Best practices for data collection should be performed to obtain high quality data by reducing collection (e.g., mislabeling, improper technique) and measurement errors. With sound data, appropriate and optimal statistical methods should be used to generate valid results. The statistical method deployed should be chosen based on assumptions about residuals (e.g., normality, correlation, and homogeneity) and on the type of data (e.g., quantitative continuous or categorical). The appropriate statistical model used should also be consistent with the experimental design to validate the respective test statistics. The science of statistics is changing rapidly. With the development of high-throughput technologies, the generation of large datasets, high performance and sophisticated models and the interest in Big Data, the training of animal science graduate students in data management and rigorous statistical analyses is more important than ever. In order to meet the demands of current trends, animal science graduate students must be trained in several complex statistical and computational skills to meet the challenges imposed by these complicated, sophisticated and nuanced analytical methods. The livestock production sector will benefit from improved training, use of advanced and appropriate experimental designs, and collection and analysis of quality data in research.

The comparative effect of HMX content on the detonation performance characterization of PBX 9012 and PBX 9501 high explosives

A series of detonation performance experiments are described for the plastic-bonded high explosive (HE) PBX 9012, nominally composed of 90% octahydro-1,3,5,7-tetranitro-1,3,5,7-tetrazine (HMX) and 10% polymeric binder vinylidene-hexafluoropropylene copolymer (Viton VTR-5883), by weight. The experiments provide information on the PBX 9012’s detonation propagation properties and metal pushing characteristics. Although this HE formulation has been in use for 50 years as a booster (compositionally similar to LX-07), our new data represent the first full detonation performance characterization for the HE (along with recently reported shock initiation experiments). Here, we compare the new PBX 9012 data with the well-studied PBX 9501 (95% HMX and 5% polymeric binder), establishing how the 5% difference in HMX content between the explosives is manifested in the performance measurements. A scaling argument based on the known energetic material content in a large selection of high HMX content formulations is also given to help explain the observed results. Finally, the accumulated PBX 9012 data is used to generate performance models using both programmed and reactive burn techniques, providing an additional avenue to compare PBX 9012 to the more energetic PBX 9501.

Automated model-based performance analysis of software product lines under uncertainty

Context: A Software Product Line (SPL) can express the variability of a system through the specification of configuration options. Evaluating performance characteristics, such as the system response time and resource utilization, of a software product is challenging, even more so in the presence of uncertain values of the attributes. Objective: The goal of this paper is to automate the generation of performance models for software products derived from the feature model by selection heuristics. We aim at obtaining model-based predictive results to quantify the correlation between the features, along with their uncertainties, and the system performance. This way, software engineers can be informed on the performance characteristics before implementing the system. Method: We propose a tool-supported framework that, starting from a feature model annotated with performance-related characteristics, derives Queueing Network (QN) performance models for all the products of the SPL. Model-based performance analysis is carried out on the models obtained by selecting the products that show the maximum and minimum performance-based costs. Results: We applied our approach to almost seven thousand feature models including more than one hundred and seventy features. The generation of QN models is automatically performed in much less than one second, whereas their model-based performance analysis embeds simulation delays and requires about six minutes on average. Conclusion: The experimental results confirm that our approach can be effective on a variety of systems for which software engineers may be provided with early insights on the system performance in reasonably short times. Software engineers are supported in the task of understanding the performance bounds that may encounter when (de)selecting different configuration options, along with their uncertainties.

Model-based prediction of automatic memory management and garbage collection behavior

Performance models focus on resource consumption and the effects of CPU, network, or hard-disk utilization. These resources usually have the largest effect on the response times and throughput of an application. However, deficient memory management can have severe effects on an application and its runtime, such as overlong response times or even crashes. As memory management has been disregarded in performance simulations, we address this gap with an approach based on memory measurements and derived metrics to predict the behavior of this resource and the effects on the CPU. Although numerous works exist that analyze memory management and especially garbage collections, accurate prediction models are rare. We demonstrate the automatic extraction of memory behavior using a performance model generator. Furthermore, the approach is evaluated using the SPECjEnterprise2010 and the SPECjEnterpriseNEXT industry benchmark, using different resource environments, garbage collection algorithms, and workloads. This work demonstrates that a certain set of probabilities allows one to create a memory profile for an architecture and predict the behavior of the memory management. The results of such predictions can be used for better capacity planning (on-premise), cost-prediction (cloud), architecture evaluation and optimization, or memory profiling. This approach allows for a continuous model-based evaluation of an enterprise architecture regarding its memory footprint.

Multi-Objective Optimization of Deployment Topologies for Distributed Applications

Modern applications are typically implemented as distributed systems comprising several components. Deciding where to deploy which component is a difficult task that today is usually assisted by logical topology recommendations. Choosing inefficient topologies allocates the wrong amount of resources, leads to unnecessary operation costs, or results in poor performance. Testing different topologies to find good solutions takes a lot of time and might delay productive operations. Therefore, this work introduces a software-based deployment topology optimization approach for distributed applications. We use an enhanced performance model generator that extracts models from operational monitoring data of running applications. The extracted model is used to simulate performance metrics (e.g., resource utilization, response times, throughput) and runtime costs of distributed applications. Subsequently, we introduce a deployment topology optimizer, which selects an optimized topology for a specified workload and considers on-premise, cloud, and hybrid topologies. The following three optimization goals are presented in this work: (i) minimum response time for an optimized user experience, (ii) approximate resource utilization around certain peaks, and (iii) minimum cost for running the application. To evaluate the approach, we use the SPECjEnterpriseNEXT industry benchmark as distributed application in an on-premise and in a cloud/on-premise hybrid environment. The evaluation demonstrates the accuracy of the simulation compared to the actual deployment by deploying an optimized topology and comparing measurements with simulation results.

Performance prediction of adaptive filters for EEG signal

A methodology for predicting the performance of algorithm in terms of fidelity parameters is introduced. Algorithm performance is measured in terms of space and time complexities required to solve a particular problem. The proposed approach correlates the performance with input data and output results. Initial theory of performance model generation is based on polynomial regressing and curve fitting of least mean square (LMS) algorithm for adaptive filtering. Testing and validation of the generated model has been done through the coefficient of determination parameters calculated over interpolation and extrapolation results. Other measures like adjusted R 2, sum of square error and mean square error are also reported. Overall study is summarised in form of algorithm, and is tested with LMS and recursive LMS algorithm of adaptive filtering applied over electroencephalography/event related potentials (EEG/ERP) noise removal to build up relation between input signal-to-noise ratio (SNR) and output fidelity parameters (like output SNR and correlation (r)).

The COMPLEX methodology for UML/MARTE Modeling and design space exploration of embedded systems

The design of embedded systems is being challenged by their growing complexity and tight performance requirements. This paper presents the COMPLEX UML/MARTE Design Space Exploration methodology, an approach based on a novel combination of Model Driven Engineering (MDE), Electronic System Level (ESL) and design exploration technologies. The proposed framework enables capturing the set of possible design solutions, that is, the design space, in an abstract, standard and graphical way by relying on UML and the standard MARTE profile. From that UML/MARTE based model, the automated generation framework proposed produces an executable, configurable and fast performance model which includes functional code of the application components. This generated model integrates an XML-based interface for communication with the tool which steers the exploration. This way, the DSE loop iterations are efficiently performed, without user intervention, avoiding slow manual editions, or regeneration of the performance model. The novel DSE suited modelling features of the methodology are shown in detail. The paper also presents the performance model generation framework, including the enhancements with regard the previous simulation and estimation technology, and the exploration technology. The paper uses an EFR vocoder system example for showing the methodology and for demonstrative results.

Automated Generation of Performance and Dependability Models for the Assessment of Wireless Sensor Networks

Wireless Sensor Networks (WSNs) are widely recognized as a promising solution to build next-generation monitoring systems. Their industrial uptake is however still compromised by the low level of trust on their performance and dependability. Whereas analytical models represent a valid mean to assess nonfunctional properties via simulation, their wide use is still limited by the complexity and dynamicity of WSNs, which lead to unaffordable modeling costs. To reduce this gap between research achievements and industrial development, this paper presents a framework for the assessment of WSNs based on the automated generation of analytical models. The framework hides modeling details, and it allows designers to focus on simulation results to drive their design choices. Models are generated starting from a high-level specification of the system and by a preliminary characterization of its fault-free behavior, using behavioral simulators. The benefits of the framework are shown in the context of two case studies, based on the wireless monitoring of civil structures.

A Methodology for Generation of Performance Models for the Sizing of Analog High-Level Topologies

This paper presents a systematic methodology for the generation of high-level performance models for analog component blocks. The transistor sizes of the circuit-level implementations of the component blocks along with a set of geometry constraints applied over them define the sample space. A Halton sequence generator is used as a sampling algorithm. Performance data are generated by simulating each sampled circuit configuration through SPICE. Least squares support vector machine (LS-SVM) is used as a regression function. Optimal values of the model hyper parameters are determined through a grid search-based technique and a genetic algorithm- (GA-) based technique. The high-level models of the individual component blocks are combined analytically to construct the high-level model of a complete system. The constructed performance models have been used to implement a GA-based high-level topology sizing process. The advantages of the present methodology are that the constructed models are accurate with respect to real circuit-level simulation results, fast to evaluate, and have a good generalization ability. In addition, the model construction time is low and the construction process does not require any detailed knowledge of circuit design. The entire methodology has been demonstrated with a set of numerical results.

Parametric performance completions for model-driven performance prediction

Performance prediction methods can help software architects to identify potential performance problems, such as bottlenecks, in their software systems during the design phase. In such early stages of the software life-cycle, only a little information is available about the system’s implementation and execution environment. However, these details are crucial for accurate performance predictions. Performance completions close the gap between available high-level models and required low-level details. Using model-driven technologies, transformations can include details of the implementation and execution environment into abstract performance models. However, existing approaches do not consider the relation of actual implementations and performance models used for prediction. Furthermore, they neglect the broad variety of possible implementations and middleware platforms, possible configurations, and possible usage scenarios. In this paper, we (i) establish a formal relation between generated performance models and generated code, (ii) introduce a design and application process for parametric performance completions, and (iii) develop a parametric performance completion for Message-oriented Middleware according to our method. Parametric performance completions are independent of a specific platform, reflect performance-relevant software configurations, and capture the influence of different usage scenarios. To evaluate the prediction accuracy of the completion for Message-oriented Middleware, we conducted a real-world case study with the SPECjms2007 Benchmark [ http://www.spec.org/jms2007/]. The observed deviation of measurements and predictions was below 10% to 15%.

Order‐to‐delivery process performance in delivery scheduling environments

Purpose – The purpose of this paper is to generate a performance model for an order-to-delivery (OTD) process in delivery scheduling environments. It aims to do this with a triadic approach, encompassing a customer, a supplier and a logistics service provider. Design/methodology/approach – The paper takes the form of a conceptual analysis and a triadic case study on performance measurement requirements in an OTD process characterized by delivery scheduling, and generating performance models. Findings – Two OTD process performance models, one for the supplier’s delivery sub-process and one for the customer’s delivery scheduling, the logistics service provider’s transportation and the customer’s good receipt sub-process, in delivery scheduling environments are generated. Research limitations/implications – A single case study limits the levels of external validity and reliability to analytical generalization. Practical implications – The generated performance models include definitions of four sub-processes and outline ten performance dimensions that should be of relevance for several companies to apply. Originality/value – This is the first approach that generates performance models for a triadic OTD process for use in delivery scheduling environments.

AUTOMATIC MAPPING OF ASSIST APPLICATIONS USING PROCESS ALGEBRA

Grid technologies aim to harness the computational capabilities of widely distributed collections of computers. Due to the heterogeneous and dynamic nature of the set of grid resources, the programming and optimisation burden of a low level approach to grid computing is clearly unacceptable for large scale, complex applications. The development of grid applications can be simplified by using high-level programming environments. In the present work, we address the problem of the mapping of a high-level grid application onto the computational resources. In order to optimise the mapping of the application, we propose to automatically generate performance models from the application using the process algebra PEPA. We target applications written with the high-level environment ASSIST, since the use of such a structured environment allows us to automate the study of the application more effectively.

An intermediate metamodel with scenarios and resources for generating performance models from UML designs

Performance analysis of a software specification in a language such as UML can assist a design team in evaluating performance-sensitive design decisions and in making design trade-offs that involve performance. Annotations to the design based on the UML Profile for Schedulability, Performance and Time provide necessary information such as workload parameters for a performance model, and many different kinds of performance techniques can be applied. The Core Scenario Model (CSM) described here provides a metamodel for an intermediate form which correlates multiple UML diagrams, extracts the behaviour elements with the performance annotations, attaches important resource information that is obtained from the UML, and supports the creation of many different kinds of performance models. Models can be made using queueing networks, layered queues, timed Petri nets, and it is proposed to develop the CSM as an intermediate language for all performance formalisms. This paper defines the CSM and describes how it resolves questions that arise in performance model-building.

Incorporating SPE into MDA

A typical feature of a distributed system is the heterogeneity of its components (their geographical spreading, using different programming languages and platform architectures, etc.). To solve some of the problems related to this heterogeneity, many distributed systems use communication middleware.This paper presents an MDA model transformation algorithm and tool for transforming a high-level Platform Specific Model (high-level PSM) to a low-level PSM by including the structural changes and the overhead of using CORBA as a middleware. The resulting PSM (a UML model annotated with performance information, using the UML performance profile) can be used for generating performance models of the system, using existing methods and tools.

Simulation-based generation of posynomial performance models for the sizing of analog integrated circuits

This paper presents an overview of methods to automatically generate posynomial response surface models for the performance characteristics of analog integrated circuits based on numerical simulation data. The methods are capable of generating posynomial performance expressions for both linear and nonlinear circuits and circuit characteristics, at SPICE-level accuracy. This approach allows for automatic generation of an accurate sizing model for a circuit that composes a geometric program that fully describes the analog circuit sizing problem. The automatic generation avoids the time-consuming and approximate nature of handcrafted analytic model generation. The methods are based on techniques from design of experiments and response surface modeling. Attention is paid to estimating the relative goodness-of-fit of the generated models. Experimental results illustrate the capabilities and effectiveness of the presented methods.

Automatic derivation of software performance models from CASE documents

Lifecycle validation of software performance (or prediction of the product ability to satisfy the user performance-requirements) is based on the automatic derivation of software performance models from CASE documents or rapid prototypes. This paper deals with the CASE document alternative. After a brief overview of existing automatic derivation methods, it introduces a method that unifies existing techniques that use CASE documents. The method is step-wise clear, can be used from the early phases of the software lifecycle, is distributed-software oriented, and can be easily incorporated into modern (e.g., UML-based) CASE tools. The method enables the software designer with no specific knowledge of performance theory to predict at design time the performance of various final product alternatives. The designer does only need to feed the CASE documents into the performance model generator. The paper carries on an application case study that deals with the development of distributed software, where the method is used to predict the performance of different distributed architectures the designer could select at preliminary design time to obtain the best performing final product. The method can be easily incorporated into modern object-oriented software development environments to encourage software designers to introduce lifecycle performance validation into their development best practices.

Early generation of performance models for object-oriented systems

Exploiting features of the OMT-based object-oriented approach, the paper presents a method for the performance assessment early in the lifecycle, and before the design phase. To this end the actor-event graphs (AEG) new formalism is presented that provides an intermediate model between specification models and software performance models. The AEG formalism unifies performance relevant information contained in various OMT models (class diagrams, interaction diagrams, state transition diagrams) into a single representation. Two algorithms deriving AEGs from different analysis results are described and the effectiveness of the approach is illustrated by an application case study.

Artificial neural network-based performance assessments

We have explored the ability of artificial neural network technologies to generate performance models of complex problem-solving tasks without the detailed a priori knowledge of the nature of the task. To test the generalizibility of this approach we applied this analysis to two diverse content domains—high school genetics and clinical patient management. In both domains, the artificial neural networks, using only the sequence of actions taken while performing the task, generated multiple classification groups defining different levels of competence. The validity of these neural network performance groupings was further established by the good concordance of these classifications with independently derived expert ratings.