System Evaluation Criteria Research Articles

Evaluation criteria for health information systems using human, organization and technology-Fit (HOT-Fit) Framework: Comprehensive review

Evaluation criteria for health information systems using human, organization and technology-Fit (HOT-Fit) Framework: Comprehensive review

Problems of energy resources allocation at decision-making stages in architecture and urban planning activities

All the levels of architecture and urban planning activities (management, designing, execution of construction works) bring up various questions of the optimal distribution of energy resources. This paper shows that the energy resources allocation is connected with different classes of tasks in optimal decisions making and belongs to mathematical programming problems. Optimization tasks at the stages of scientific research, substantiation and version design should be performed on the basis of dynamic system models of the type «population ↔ environment». The article justifies the need for deductive approach and system methods and evaluation criteria of energy resources distribution at the stages of design and quality evaluation of architectural objects. It is demonstrated that in architectural theory and practice, for the purpose of quantitative measurement of the quality of architectural and construction activity products, the scientific research of the new area – architectural metrology – is needed. The energy interpretation of the general optimality criterion of integral architectural objects as systems, functioning in accordance with general system principles, is discussed.

Optimizing Course System of Information Engineering—Under the Background of Establishing the Emerging Engineering

Under the background of establishing the Emerging Engineering, combined with the requirements of engineering education certification system for the training of information engineering professionals, the optimization scheme of curriculum system is put forward. On the basis of introducing the background of curriculum system construction, this paper elaborates on the optimization ideas, curriculum system structure design and evaluation criteria. Focusing on the optimization of the curriculum system, the teaching team is formed with the curriculum group as the teaching unit to form a systematic, coordinated and adaptable teaching-group mode.

Parametric Analysis of Solid Oxide Co-Electrolysis Cell Systems: Relationship between System Performance, External Heat and Parasitic Power

In this study, we develop a high-fidelity SOEC system model formulated in C# language, establish potential system layouts, and find an optimal operation condition. Multiple system pressure conditions are examined aiming at commercial-scale carbon dioxide reduction. Based on the CO2 reduction ratio and its efficiency as system evaluation criteria, pressure conditions are ranked in terms of carbon decomposition, parasitic power and the amount of heat required from external heat sources.

Evaluation of Google question-answering quality

PurposeThe purpose of this paper is to evaluate Google question-answering (QA) quality.Design/methodology/approachGiven the large variety and complexity of Google answer boxes in search result pages, existing evaluation criteria for both search engines and QA systems seemed unsuitable. This study developed an evaluation criteria system for the evaluation of Google QA quality by coding and analyzing search results of questions from a representative question set. The study then evaluated Google’s overall QA quality as well as QA quality across four target types and across six question types, using the newly developed criteria system. ANOVA and Tukey tests were used to compare QA quality among different target types and question types.FindingsIt was found that Google provided significantly higher-quality answers to person-related questions than to thing-related, event-related and organization-related questions. Google also provided significantly higher-quality answers to where- questions than to who-, what- and how-questions. The more specific a question is, the higher the QA quality would be.Research limitations/implicationsSuggestions for both search engine users and designers are presented to help enhance user experience and QA quality.Originality/valueParticularly suitable for search engine QA quality analysis, the newly developed evaluation criteria system expanded and enriched assessment metrics of both search engines and QA systems.

Negative Sequence Analysis

Negative sequential patterns (NSPs) produced by negative sequence analysis (NSA) capture more informative and actionable knowledge than classic positive sequential patterns (PSPs) due to involving both occurring and nonoccurring items, which appear in many applications. However, the research on NSA is still at an early stage, and NSP mining involves very high computational complexity and a very large search space, there is no widely accepted problem statement on NSP mining, and different settings on constraints and negative containment have been proposed in existing work. Among existing NSP mining algorithms, there are no general and systemic evaluation criteria available to assess them comprehensively. This article conducts a comprehensive technical review of existing NSA research. We explore and formalize a generic problem statement of NSA; investigate, compare, and consolidate the definitions of constraints and negative containment; and compare the working mechanisms and efficiency of existing NSP mining algorithms. The review is concluded by discussing new research opportunities in NSA.

Big Data Quality Assurance Through Data Traceability: A Case Study of the National Standard Reference Data Program of Korea

In the era of big data, the scientific and social demand for quality data is aggressive and urgent. This paper sheds light on the expanded role of metrology of verifying validated procedures of data production and developing adequate uncertainty evaluation methods to ensure the trustworthiness of data and information. In this regard, I explore the mechanism of the national standard reference data (SRD) program of Korea, which connects various scientific and social sectors to metrology by applying useful metrological concepts and methods to produce reliable data and convert such data into national standards. In particular, the changing interpretation of metrological key concepts, such as “measurement,” “traceability,” and “uncertainty,” will be explored and reconsidered from the perspective of data quality assurance. As a result, I suggest the concept of “data traceability” with “the matrix of data quality evaluation” according to the elements of a data production system and related evaluation criteria. To conclude, I suggest social and policy implications for the new role of metrology and standards for producing and disseminating reliable knowledge sources from big data.

Public Health Emergency Preparedness System Evaluation Criteria and Performance Metrics: A Review of Contributions of the CDC-Funded Preparedness and Emergency Response Research Centers.

The US Centers for Disease Control and Prevention (CDC)-funded Preparedness and Emergency Response Research Centers (PERRCs) conducted research from 2008 to 2015 aimed to improve the complex public health emergency preparedness and response (PHEPR) system. This paper summarizes PERRC studies that addressed the development and assessment of criteria for evaluating PHEPR and metrics for measuring their efficiency and effectiveness. We reviewed 171 PERRC publications indexed in PubMed between 2009 and 2016. These publications derived from 34 PERRC research projects. We identified publications that addressed the development or assessment of criteria and metrics pertaining to PHEPR systems and describe the evaluation methods used and tools developed, the system domains evaluated, and the metrics developed or assessed. We identified 29 publications from 12 of the 34 PERRC projects that addressed PHEPR system evaluation criteria and metrics. We grouped each study into 1 of 3 system domains, based on the metrics developed or assessed: (1) organizational characteristics (n = 9), (2) emergency response performance (n = 12), and (3) workforce capacity or capability (n = 8). These studies addressed PHEPR system activities including responses to the 2009 H1N1 pandemic and the 2011 tsunami, as well as emergency exercise performance, situational awareness, and workforce willingness to respond. Both PHEPR system process and outcome metrics were developed or assessed by PERRC studies. PERRC researchers developed and evaluated a range of PHEPR system evaluation criteria and metrics that should be considered by system partners interested in assessing the efficiency and effectiveness of their activities. Nonetheless, the monitoring and measurement problem in PHEPR is far from solved. Lack of standard measures that are readily obtained or computed at local levels remains a challenge for the public health preparedness field. (Disaster Med Public Health Preparedness. 2019;13:626-638).

DUAL APPROACH TO ESTIMATION OF INNOVATIVE TELNOLOGIES IN TELECOMMUNICATION SYSTEMS VIA QUANTITATIVE CRITERIA

The main criteria for quantitative evaluation of telecommunication systems are considered. It is shown that the existing quantitative criteria reflect the technical capabilities of the system and do not take into account the attendant factors that reflect the essence of innovative activity in telecommunication systems. It is proposed to assess the quantitative parameters of telecommunication systems taking into account innovative technologies in two stages. The quantitative criteria are divided into two groups, depending on the stage of development or the stage of operation of the system. Each group of criteria is divided into two subgroups: absolute and relative. The notion of intermediate estimates and weighting coefficients is introduced. Expressions were obtained for determining intermediate and total quantitative assessments of the quality of telecommunication systems at different stages. The feasibility of a dual approach to the evaluation criteria for telecommunication systems.

System principles and evaluation criteria of reliability for optimization of technological parameters of coal deposits

The article deals with solving the scientific problem of select mining equipment for selection longwall faces of mining venture. The main goal of the paper is to study technology of coal extraction of mining venture. The paper proposes a new approach to solve a problem of mining equipment selection for longwall faces of mining venture. The article describes new method for selection of mining equipment based on theory graph. Special attention is given to technological aspects; they are lenght of longwall faces, depth of coal stratum. Predictions obtained for daily production of mining equipment are compared with design outputs. Conclusions regarding the main reason of instability of longwall faces workings are made. It is found that to be able to use standard algorithms find the shortest path between vertices, you must perform matrix description of the constructed graphs that illustrate the structure of the interaction of different types of control equipment. Using classical optimization of method of discrete mathematics and algorithms for finding the shortest path between two vertices of network models obtained from the formalization of graphs with maximum results of specific types of equipment production chains, solved the problem of rational choice cleaning equipment for the new site with the cost parameters of the mining equipment and cost of mining coal. The authors developed effective and appropriate variants for the mine development for different coal deposits of mining venture.

The functional aspect of assessing the language for specific purposes at Law Departments

The paper analyzes an evaluation system of law students LSP level that is developed and introduced by the author in Petrozavodsk State University. The development of such an evaluation system is determined by the necessity of making the assessment of law students foreign language competence more systematic, objective, communicative, individualized, covering all language skills, less stressful and including self-evaluation. The paper draws attention to the functional aspect of the evaluation system as the functions of the assessment are responsible for the correct arrangement of testing, evaluation impact on the learning process, students motivation, grading and skills development by means of testing procedures. The author visually presents the application of the assessment functions at different stages of the learning process and their role in entry, formative, progress achievement and summative assessment. The paper also examines other aspects of the evaluation system aims, developed skills and evaluation criteria.

The Degree of Guilt as the Special Criteria of Individualization of Administrative Responsibility of Legal Entities and Individual Entrepreneurs

The article reveals the conflicts and uncertainties of the legislative techniques of assessment of guilt and the individualization of administrative responsibility of legal entities and individual entrepreneurs who have made administrative offenses. Proposed system evaluation criteria when making decisions and other decisions of the courts and supervisory bodies of executive power about the choice of alternative administrative penalties, mitigation of the administrative penalty, the exemption from administrative responsibility to determine the degree of guilt of a legal entity and individual entrepreneur, brought to administrative responsibility in the implementation of the state control and supervision.

Usefulness, localizability, humanness, and language-benefit: additional evaluation criteria for natural language dialogue systems

Human---computer dialogue systems interact with human users using natural language. We used the ALICE/AIML chatbot architecture as a platform to develop a range of chatbots covering different languages, genres, text-types, and user-groups, to illustrate qualitative aspects of natural dialogue system evaluation. We present some of the different evaluation techniques used in natural dialogue systems, including black box and glass box, comparative, quantitative, and qualitative evaluation. Four aspects of NLP dialogue system evaluation are often overlooked: usefulness in terms of a user's qualitative needs, localizability to new genres and languages, humanness or naturalness compared to human---human dialogues, and language benefit compared to alternative interfaces. We illustrated these aspects with respect to our work on machine-learnt chatbot dialogue systems; we believe these aspects are worthwhile in impressing potential new users and customers.

Identifying and Prioritizing Evaluation Criteria for User-Centric Digital Identity Management Systems

Identity Management systems are used for securing digital identity of users in reliable, automated and compatible way. Service providers employ identity management systems which are cost effective and scalable but cause poor usability for users. Identity management systems are user-centric applications which should be designed by considering users’ perspective. User centricity is a remarkable concept in identity management systems as it provides more powerful user control and privacy. This approach has been evolved from amending past paradigms. Thus, evaluation of digital identity management systems based on users’ point of view, is really important. The main objective of this paper is to identify the appropriateness of the criteria used in evaluation of user-centric digital identity management systems. These criteria are gathered from the literature and then categorized into four groups for the first time in this work to examine the importance of each parameter. In this approach, several interviews were performed as a qualitative research method and two questionnaires have been filled out by forty six users who were involved with identity management systems. Since the answers are perception based data the most important criteria in each category are assessed by using fuzzy method. This research found that the most important criteria are related to security category. The results of this research can provide valuable information for managers and decision makers of hosting companies as well as system designers to adapt and develop appropriate user-centric digital identity management systems.

Multicriteria optimization of natural and recycled aggregate concrete for structural use

Recycling of concrete waste to produce recycled concrete aggregates (RCA) and its use in the production of structural concrete is the way towards reducing the depletion of natural mineral resources and the amount of construction and demolition waste being land-filled. The goal of this study was to determine the optimal choice of aggregate type and transport scenario in concrete production, employing a multicriteria optimization method taking into account technical, economical and environmental limits and constraints. Several concrete types with different type of used aggregate (river aggregate, crushed stone and recycled concrete aggregate) and different transport scenarios were analyzed. The environmental system evaluation criteria were chosen according to the Life Cycle Assessment methodology and the economical system evaluation criteria were determined in accordance with the current state of the ready-mixed concrete market in Serbia. The normative multicriteria optimization method (VIKOR method) was employed to determine the optimal solution. Analysis was performed for different decision making scenarios that gave emphasis to different criteria. Results have shown that concrete with a 50% replacement ratio of coarse aggregate with RCA can be presented as an optimal solution. Since natural aggregate concrete made with river aggregate was shown to have the lowest price, a further analysis was conducted to determine what economical measures should be undertaken in order to achieve cost equality of recycled and natural aggregate concrete. The analysis identified taxes on river aggregate, taxes on land-filling and subsidies for using RCA as viable measures.

Evaluation and optimization of dynamic stiffness values of the PKMs: Collinear stiffness value approach

Dynamic stiffness of the parallel-kinematics machines is considered and a new engineering index – dynamic collinear stiffness value (DynCSV) – is proposed. The DynCSV presents an extension of a concept derived by Portman for evaluation of the static stiffness values. The DynCSV depends on robot-configuration and inertia parameters and presents the function of the vibration frequency. Mathematically, the DynCSV is a quadratic form associated with the real symmetric matrix of the Newton–Euler equation. The minimal and maximal values of the DynCSV give natural criteria for evaluation, limitation, and optimization of dynamic systems of machines and robots. In the frequency range, where the dynamic stiffness matrix has a semipositive definition, the minimum DynCSV is applied to construct a PKM workspace satisfying preliminary given stiffness limits and compute a virtual protective barrier, keeping the PKM mechanism from approaching both singular configurations and resonance frequencies. As an application example, the DynCSV-based local and global stiffness features of the Gough–Stewart platform mechanism are simulated, investigated, and visualized.

MRI findings at baseline and after neoadjuvant chemotherapy in orbital retinoblastoma (IRSS stage III)

BackgroundPublished findings on MRI results in retinoblastoma patients treated with neoadjuvant chemotherapy (NACT) are lacking. The present study evaluates the role of MRI in International Retinoblastoma Staging System (IRSS) stage...

Hierarchical multi-criteria risk-benefit analysis in fuzzy environment

The objectives of this paper are: (1) to utilize hierarchical fuzzy technique for order preference by similarity to ideal solution (TOPSIS) to evaluate the most suitable RFID-based systems decision and (2) to list key risks and benefits of radio frequency identification technology in library services. Researcher explains the importance of selection criteria for evaluation of RFID-based systems. It provides key elements on radio frequency identification, fuzzy hierarchical TOPSIS methodology and an algorithm that can be followed to solve the problem. A sample problem using the algorithm is solved and results are explained. Due to the fact that a better management of a library system is related to the full understanding of the technologies implemented and the system under consideration, some background on the radio frequency identification technology is provided and the kind of systems that most likely managements would face with and select are provided.

Evaluating recommender systems from the user’s perspective: survey of the state of the art

A recommender system is a Web technology that proactively suggests items of interest to users based on their objective behavior or explicitly stated preferences. Evaluations of recommender systems (RS) have traditionally focused on the performance of algorithms. However, many researchers have recently started investigating system effectiveness and evaluation criteria from users’ perspectives. In this paper, we survey the state of the art of user experience research in RS by examining how researchers have evaluated design methods that augment RS’s ability to help users find the information or product that they truly prefer, interact with ease with the system, and form trust with RS through system transparency, control and privacy preserving mechanisms finally, we examine how these system design features influence users’ adoption of the technology. We summarize existing work concerning three crucial interaction activities between the user and the system: the initial preference elicitation process, the preference refinement process, and the presentation of the system’s recommendation results. Additionally, we will also cover recent evaluation frameworks that measure a recommender system’s overall perceptive qualities and how these qualities influence users’ behavioral intentions. The key results are summarized in a set of design guidelines that can provide useful suggestions to scholars and practitioners concerning the design and development of effective recommender systems. The survey also lays groundwork for researchers to pursue future topics that have not been covered by existing methods.

Study of Concrete Pump Truck Structural Health Monitoring

Concrete pump truck is a kind of mobile construction machinery, with the characteristic of complex structure, poor working condition and difficult to maintenance. Therefore, adopting appropriate health monitoring methods, and accurate grasping the running status information of the pump truck is significant to the pump truck’s safety use and pre-judgment maintenance plan arrangement. In this paper, the traditional structural health monitoring methods was studied. And the structure, load and work environment characteristics of concrete pump truck were analyzed. Taking into account the economy and reliability of the structural health monitoring system, the technical route of the concrete pump truck structural health monitoring system and health evaluation criteria were proposed. The evaluation criteria takes into account both the cumulative health effect and timely health status of concrete pump truck structure.