Systematic Performance Evaluation Research Articles

Benchmarking quantum computer simulation software packages: State vector simulators

Rapid advances in quantum computing technology lead to an increasing need for software simulators that enable both algorithm design and the validation of results obtained from quantum hardware. This includes calculations that aim at probing regimes of quantum advantage, where a quantum computer outperforms a classical computer in the same task. High performance computing (HPC) platforms play a crucial role as today’s quantum devices already reach beyond the limits of what powerful workstations can model, but a systematic evaluation of the individual performance of the many offered simulation packages is lacking so far. In this Technical Review, we benchmark several software packages capable of simulating quantum dynamics with a special focus on HPC capabilities. We develop a containerized toolchain for benchmarking a large set of simulation packages on a local HPC cluster using different parallelisation capabilities, and compare the performance and system size-scaling for three paradigmatic quantum computing tasks. Our results can help finding the right package for a given simulation task and lay the foundation for a systematic community effort to benchmark and validate upcoming versions of existing and also newly developed simulation packages.

Development of lightweight bricks using agricultural and industrial waste

Abstract The construction industry is undergoing a notable shift towards environmental sustainability, driving the innovation of construction materials that are robust, lightweight, and eco-friendly. this study delves into the formation of lightweight bricks, with a focus on their mechanical properties, thermal insulation capabilities, and environment-imperative agricultural and industrial waste material. In response to the environmental imperative to reduce reliance on clay and other raw materials in construction, this study investigates the utilization of flower waste, iron ore, chaff, and wheat husk, in the production of lightweight bricks. This alternative material offers a sustainable solution to mitigate the environmental impact of traditional brick manufacturing processes. Through a systematic evaluation of physical properties, materialistic properties, and thermal performance the study provides insights into the feasibility and efficacy of incorporating chaff and wheat husk into brick production. The findings offer practical recommendations for the development of environment-friendly construction materials contribute to the conservation of natural resources and the promotion of sustainable building practices.

High-Efficiency Enzyme Assay and Screening of Enzyme-Inhibiting Nanomaterials Using Capillary Electrophoresis with Hierarchically Porous Metal-Organic Framework-Based Immobilized Enzyme Microreactor.

Enzyme-inhibiting nanomaterials have significant potential for regulating enzyme activity. However, a universal and efficient method for systematically screening and evaluating the inhibitory effects of various nanomaterials on drug target enzymes has not been established. While the integrated technique of immobilized enzyme microreactor (IMER) with capillary electrophoresis (CE) serves as an effective tool for enzyme analysis, it still faces challenges such as low enzyme loadability, unsatisfactory stability, and limited applicability. Herein, hierarchical porous metal-organic frameworks (HP-MOFs) were explored as high-performance enzyme immobilization carriers and stationary phases to develop a novel HP-MOFs-based IMER-CE microanalysis system for efficient online enzyme assay and systematic screening of enzyme-inhibiting nanomaterials. As a proof-of-concept demonstration, the model enzyme xanthine oxidase (XOD) was immobilized on a HP-UiO-66-NH2 coated capillary, serving as an efficient and durable IMER for screening potential XOD-inhibiting nanomaterials. The hierarchically micro- and mesoporous structure and superior enzyme loadability of as-prepared HP-UiO-66-NH2-IMER was intensively characterized, followed by systematic evaluation of the separation performance of HP-UiO-66-NH2 coated column and the enzyme kinetics of the immobilized XOD. Compared to the microporous UiO-66-NH2-IMER, the HP-UiO-66-NH2-IMER-CE system showed significant improvements in enzyme loading, maximum reaction rate, repeatability, and long-term stability. Furthermore, the established method was effectively employed to screen the XOD inhibitory activity of various nanomaterials, revealing that graphene oxide, single wall carbon nanotube and three other nanomaterials exhibited inhibitory potentials. The HP-MOFs-based microanalysis system can be easily expanded by modifying the types of immobilized enzymes and holds the potential to accelerate the identification and rational design of effective enzyme-inhibiting nanomaterials.

Enhancing accuracy and interpretability of multi-steps water demand prediction through prior knowledge integration in neural network architecture

In the field of water supply management, multi-steps water demand forecasting plays a crucial role. While there have been many studies related to multi-steps water demand forecasting based on deep learning, little attention has been paid to the interpretability of forecasting models. Aiming to improve both the forecasting accuracy and interpretability of the model, a novel urban water demand forecasting neural network (UWDFNet) was presented in this paper. Compared with traditional deep learning models, it innovatively considered domain-specific prior knowledge from water supply management and incorporated the correlation relationship between different input variables into the design of the neural network structure, and verified the consistency between the knowledge learned by the model and prior knowledge through interpretability analysis. Additionally, a systematic performance evaluation was conducted and proved that UWDFNet possesses better accuracy and stability compared to other baseline models(e.g., gated recurrent unit network (GRUN), GRUN with a corrected Network (GRUN+CORRNet), GRUN+PID, GRUN+Kmeans).

Agents of Autonomy: A Systematic Study of Robotics on Modern Hardware

As robots increasingly permeate modern society, it is crucial for the system and hardware research community to bridge its long-standing gap with robotics. This divide has persisted due to the lack of (i) a systematic performance evaluation of robotics on different computing platforms and (ii) a comprehensive, open-source, cross-platform benchmark suite. To address these gaps, we present a systematic performance study of robotics on modern hardware and introduce RoWild, an open-source benchmark suite for robotics that is comprehensive and cross-platform. Our workloads encompass a broad range of robots, including driverless vehicles, pilotless drones, and stationary robotic arms, and we evaluate their performance on a spectrum of modern computing platforms, from low-end embedded CPUs to high-end server-grade GPUs. Our findings reveal that current architectures experience significant inefficiencies when executing robotic workloads, highlighting the need for architectural advancements. We discuss approaches for meeting these requirements, offering insights for improving the performance of robotics. The full version of the paper is available in [11], and the source code of the benchmark suite is available in [2].

Systematic evaluation of single-cell RNA-seq analyses performance based on long-read sequencing platforms

IntroductionThe rapid development of next-generation sequencing (NGS)-based single-cell RNA sequencing (scRNA-seq) allows for detecting and quantifying gene expression in a high-throughput manner, providing a powerful tool for comprehensively understanding cellular function in various biological processes. However, the NGS-based scRNA-seq only quantifies gene expression and cannot reveal the exact transcript structures (isoforms) of each gene due to the limited read length. On the other hand, the long read length of third-generation sequencing (TGS) technologies, including Oxford Nanopore Technologies (ONT) and Pacific Biosciences (PacBio), enable direct reading of intact cDNA molecules. ObjectivesBoth ONT and PacBio have been used in conjunction with scRNA-seq, but their performance in single-cell analyses has not been systematically evaluated. MethodsTo address this, we generated ONT and PacBio data from the same single-cell cDNA libraries containing different amount of cells. ResultsUsing NGS as a control, we assessed the performance of each platform in cell type identification. Additionally, the reliability in identifying novel isoforms and allele-specific gene/isoform expression by both platforms was verified, providing a systematic evaluation to design the sequencing strategies in single-cell transcriptome studies. ConclusionBeyond gene expression analysis, which the NGS-based scRNA-seq only affords, TGS-based scRNA-seq achieved gene splicing analyses, identifying novel isoforms. Attribute to higher sequencing quality of PacBio, it outperforms ONT in accuracy of novel transcripts identification and allele-specific gene/isoform expression.

Systematic evaluation of the influence of Taq enzyme choice and amplification conditions on targeted environmental DNA assay performance and detection in field samples

AbstractTargeted environmental DNA (eDNA) studies mainly rely on quantitative real‐time polymerase chain reaction (qPCR) to amplify extremely low concentrations of DNA present in environmental samples. Understanding factors that influence targeted eDNA assay performance and detection in field samples, such as Taq DNA polymerase enzyme type and thermocycle protocol, is critical for the interpretation of qPCR results. We completed a systematic performance evaluation of five distinct targeted eDNA assays (eANFI6, eFISH1, eANBO5, eGLIN1, and eLICA3 targeting sablefish, general fish, Boreal toad, wood turtle, and the American bullfrog) by subjecting the same samples to analysis by five different Taq enzyme reagent mixes (Immolase, Environmental Master Mix (EMM), Amplitaq, QIAcuity, and QuantiNova) and the commonly used 2‐step (95°C, 60°C) and 3‐step (95°C, 64°C, 72°C) thermocycle protocols. We evaluated assay performance using a standardized dilution series of synthetic dsDNA target sequences and calculated limits of detection (LOD) and quantification (LOQ) and 95% confidence intervals for each combination of Taq enzyme and thermocycle protocol. All assays performed within acceptable performance criteria as defined by the Canadian national standard for targeted eDNA assays. Based on data generated by synthetic dsDNA fragments, the eDNA assays performed comparably regardless of the enzyme reagent mix and protocol used, except for eANFI6 and eGLIN1 using EMM and the 3‐step protocol, where no amplification was observed. On freshwater field samples, EMM and Immolase performed best. On marine field samples, Immolase, EMM, Qiacuity, and QuantiNova performed equally well, although EMM failed to amplify some samples. The work reveals that an enzyme reaction mix or thermocycle protocol can affect the result of an eDNA assay, but the appropriate choice also depends on the nature of the field sample. It is therefore imperative that these are considered when selecting appropriate reaction conditions and that they are clearly reported.

IMPACT OF CLASSROOM OBSERVATION TOOL AND RESULTS-BASED PERFORMANCE MANAGEMENT SYSTEM ON ELEMENTARY SCHOOL TEACHER PERFORMANCE

This literature review explores the implications of utilizing the Classroom Observation Tool (COT) and Results-Based Performance Management System (RPMS) on teaching performance among educators in public elementary schools. The COT serves as a structured instrument for assessing classroom activities, while the RPMS provides a framework for evaluating teacher performance based on professional standards. Through a comprehensive analysis of existing research and policy documents, this review examines the impact of these systems on teaching effectiveness and student outcomes. Research indicates that the systematic use of the COT allows educators to identify areas of strength and growth in their teaching practice, leading to targeted professional development interventions. Additionally, the integration of the RPMS ensures alignment between teacher assessments and professional standards, promoting accountability and continuous improvement in teaching practice. Studies have shown that educators who participate in regular classroom observations and performance evaluations experience enhanced job satisfaction and morale, leading to greater retention and commitment to their profession. Moreover, the implementation of these systems has been associated with improvements in student learning outcomes, as teachers are better equipped to meet the diverse needs of their students. However, challenges such as resource constraints and resistance to change may hinder the effective implementation of these systems. Moving forward, further research is needed to explore the long-term impact of the COT and RPMS on teaching performance and student achievement. Overall, this literature review highlights the importance of incorporating systematic observation and performance evaluation mechanisms into teacher professional development initiatives to promote continuous improvement in teaching practice and enhance student learning outcomes in public elementary schools.

Naive Bayes for Diabetes Prediction: Developing a Classification Model for Risk Identification in Specific Populations

Depending on persuasive statistics, the increasing prevalence of diabetes worldwide is a huge challenge for individuals, families, and nations. According to International Diabetes Federation (IDF) projections, the number of adults with diabetes is expected to rise by an astounding 46% by 2045, to reach 783 million, or one in eight. In response to this growing concern, this research explores the implementation of the Naive Bayes algorithm for predicting diabetes, employing comprehensive data cleansing and randomization techniques. A systematic evaluation of the model's performance is conducted using several training and testing split ratios (65:35, 75:25, 85:15). The outcome showed that the model performed best at the 65:35 split ratio, with accuracy reaching its maximum of 88.16%, precision 0.883, recall 0.881, and f1-score 0.882.

How is your loyalty programme going? A framework for a loyalty programme performance indication system

One of the root causes of the failure of membership programmes introduced by many companies is the lack of a systematic performance evaluation framework that can design, monitor and evaluate membership programmes tailored to the company’s own circumstances. This paper proposes a loyalty programme performance evaluation framework called MemPIS to help use the loyalty programme as a strategic platform for digital customer management and to provide a standard for optimising a company's loyalty programme. Using the MemPIS framework, this paper emphasises profitability, stability, growth potential and activeness of membership when evaluating programmes from a financial perspective, and suitability of tier design, reward design, point design and membership operation for the evaluation of programmes from a non-financial perspective. After developing candidate evaluation indicators and measurable matrixes for each evaluated subject, these are applied to a Korean cosmetics company to test the practical validity of the research and to suggest improvements to the membership programme to the company. Weights for all evaluative perspectives, subjects and indicators evaluated were calculated through AHP analysis. This research found that MemPIS can not only reveal the problems of a company's membership programme, but also suggest directions for improvement in the future.

Efficiency Evaluation and Performance Governance of Infrastructure Construction Based on DEA and Improved Neural Networks

Abstract Infrastructure construction is the carrier of regional economic development, and the rational use of urban construction land is an important basis that largely depends on the efficiency of urban infrastructure construction. However, due to the lack of effective integration between urban evaluation and management in infrastructure, the infrastructure performance evaluation system is not yet mature enough. Therefore, this article conducts efficiency evaluation and performance governance of infrastructure construction based on DEA and improved neural networks. Firstly, based on a comprehensive analysis of performance evaluation, the advantages and disadvantages of infrastructure construction were explored. Secondly, in the study of using DEA and improved neural networks to evaluate the efficiency of infrastructure construction, a performance evaluation index system for urban infrastructure construction was constructed from four aspects: input, output, effect, and activity. A systematic performance evaluation of urban infrastructure was conducted, and optimization suggestions were proposed accordingly.

Applying AHP for Assessing Zenbo Division Production Employee Performance

This research proposes an employee assessment system utilizing the Analytical Hierarchy Process (AHP) method to evaluate productivity levels. The absence of such a system poses challenges in gauging employee performance and its impact on company productivity. Through AHP, this study aims to provide decision support for performance evaluation, thereby maintaining high productivity levels. Results indicate significant shifts in employee rankings, highlighting the effectiveness of the proposed methodology in identifying and enhancing productivity. This research contributes to the discourse on employee performance assessment, offering practical implications for decision-makers aiming to optimize workforce productivity. Highlights : Enhanced decision-making: The utilization of AHP provides a structured approach for evaluating employee performance, facilitating informed decision-making processes. Improved productivity rankings: Implementation of the assessment system results in notable shifts in employee rankings, indicating its effectiveness in identifying high-performing individuals. Practical implications: Findings offer valuable insights for organizations seeking to optimize workforce productivity through systematic performance evaluation methodologies. Keywords: Employee productivity, Assessment system, Analytical Hierarchy Process (AHP), Decision support, Workforce optimization

Assessment and characteristics of S-NPP VIIRS Deep Blue and Dark Target aerosol properties under clean, polluted and fire scenarios over the Amazon

The present study carries out the systematic performance evaluation of aerosol optical depth (AOD) products retrieved using Visible Infrared Imaging Radiometer Suite (VIIRS) Deep Blue (DB) and Dark Target (DT) onboard Suomi National Polar-orbiting Partnership (S-NPP) satellite over the Amazon Basin. Characteristics and uncertainty were evaluated under distinct air pollution scenarios such as a clean background in the wet season, polluted conditions in the dry season with biomass burning emissions and peak burning season with higher fire activity. VIIRS retrievals were also analyzed under aerosol loading, particle size and surface vegetation coverage against the Aerosol Robotic Network (AERONET) measurements at 9 sites in 2012–2022. VIIRS DB showed good accuracy, with 78% of AOD matchups falling within the expected error, 84% in the wet season, and 71% in the dry and burning seasons. In contrast, VIIRS DT indicated poor accuracy (64%) and trends overestimate the AOD in all air pollution scenarios. Both algorithms were sensitive to AERONET sites with high elevation and dark vegetated coverage characteristics. VIIRS DB and DT systematically overestimated AERONET AOD as increased aerosol loading. DB trends underestimate aerosol under background conditions and overestimate with coarse and fine particle predominance. Additionally, both algorithms indicated poor accuracy under forest type with a large positive bias. VIIRS DB demonstrated the highest accuracy in the presence of aerosol loading in sites characterized by mixed land cover type. This was observed in both coarse and fine mode scenarios for grassland. VIIRS DT demonstrated satisfactory accuracy under background conditions and dominance of coarse particles within grassland land cover type. For mixed land cover, satisfactory accuracy was found under intermediate aerosol loading conditions. DB algorithm showed greater uncertainty associated with the coarse particle aerosol for the full period and all polluted scenarios. Overall, VIIRS DT accuracy was more sensitive to varying air pollution scenarios.

Pretreatment by a novel photo-electro reactor to control organic and biofouling during reverse osmosis filtration of reclaimed water

This study reports the performance of a novel photo-electro filtration (PEF) reactor consisting of an IrO2/RuO2 coated anodic filter and UV lamp to pretreat biologically treated wastewater effluent prior to reverse osmosis (RO) filtration. Synthetic model solutions and biologically treated wastewater effluent were used for systematic evaluation of the PEF reactor performance in terms of organic degradation, bacterial inactivation, biofilm formation reduction, and ultimately RO fouling mitigation. Experiments with model solutions containing proteins (Bovine Serum Albumin) and bacteria (Pseudomonas aeruginosa) show that UV irradiation and electrochemical treatment with increasing voltage input resulted in superior bacterial inactivation, biofilm formation reduction, and effective organic degradation. When the PEF reactor was used to pretreat biologically treated wastewater effluent, electrochemical treatment at 3 V alone and the combination of electrochemical treatment at 3 V and UV could achieve near complete bacterial inactivation and good fouling reduction during subsequent RO operation. However, the combination of electrochemical treatment at 3 V and UV was superior over electrochemical treatment at 3 V alone in several aspects including fouling reduction and reduced risk of membrane degradation due to chlorine residue. The underlying mechanisms responsible for effective PEF pretreatment performance were elucidated by monitoring the fate of free chlorine, hydroxyl radicals (indirectly monitored by measuring concentration of the quenching chemical N-n-dimethyl-p-nitrosoaniline), H2O2, quorum sensing signal molecules, and organic fractions during UV irradiation and electrochemical treatment. Effective bio and organic fouling mitigation by PEF pretreatment was attributed to the enhanced bacterial inactivation and degradation of organic compounds (primarily protein-like and humic-like substances) by hydroxyl radicals (∙OH) produced from photo-electrochemical treatment.

Cone-beam x-ray luminescence computed tomography (CB-XLCT) prototype development and performance evaluation.

This study developed a prototype for a rotational cone-beam x-ray luminescence computed tomography (CB-XLCT) system, considering its potential application in pre-clinical theranostic imaging. A geometric calibration method applicable to both imaging chains (XL and CT) was also developed to enhance image quality. The results of systematic performance evaluations were presented to assess the feasibility of commercializing XLCT technology. Monte Carlo GATE simulation was performed to determine the optimal imaging conditions for nanophosphor particles (NPs) irradiated by 70 kV x-rays. We acquired a low-dose transmission x-ray tube and designed a prone positioning platform and a rotating gantry, using mice as targets from commercial small animalμ-CT systems. We then employed the image cross-correlation (ICC) automatic geometric calibration method to calibrate XL and CT images. The performance of the system was evaluated through a series of phantom experiments with a linearity of 0.99, and the contrast-to-noise ratio (CNR) between hydroxyl-apatite (HA) and based epoxy resin is 19.5. The XL images of the CB-XLCT prototype achieved a Dice similarity coefficient (DICE) of 0.149 for a distance of 1 mm between the two light sources. Finally, the final XLCT imaging results were demonstrated using the Letter phantoms with NPs. In summary, the CB-XLCT prototype developed in this study showed the potential to achieve high-quality imaging with acceptable radiation doses for small animals. The performance of CT images was comparable to current commercial machines, while the XL images exhibited promising results in phantom imaging, but further efforts are needed for biomedical applications.

The CIPP-TOPSIS Comprehensive Evaluation Method for Computer Major Practice Teaching in Context of Intelligent Service Demand

Practice teaching evaluation is a systematic and comprehensive evaluation of students' performance in practical operations, experiments, internships and other practical activities, aiming to measure students' professional practice ability through reasonable evaluation, and is an important part of practice teaching in colleges and universities. Practical teaching evaluation needs to solve two problems, one is to determine the evaluation model for the evaluation system, and the other is to determine the weight of the evaluation system. Combined with CIPP comprehensive evaluation model, the evaluation system of computer professional practice teaching is built, and the weight of evaluation indicators for computer professional practice is determined by the combinatorial weighting TOPSIS method, so as to comprehensively measure and evaluate the effect of professional teaching practice from multiple perspectives.

A Systematic Review and Performance Evaluation of Open-Source Tools for Smart Contract Vulnerability Detection

A Systematic Review and Performance Evaluation of Open-Source Tools for Smart Contract Vulnerability Detection

Assessment of Organizational Management Capacity of Public Higher Education Institutions: The Case of Sulu State College

This study investigated the organizational management skills of Sulu State College for fiscal year 2023. A purposive sample of 100 people was selected, and several statistical procedures such as weighted mean, standard deviation, t-tests, ANOVA, and Pearson's r were used. According to the findings, the majority of surveyed employees were female, under the age of 30, had less than five years of work experience, had permanent employment, and possessed bachelor's degrees. According to the findings, Sulu State College had a significantly high degree of adaptation, particularly in strategic and operational planning, resource mobilization, communication management, and knowledge development. Interestingly, demographic variables such as gender, age, tenure, job status, and educational background had no significant influence on employees' perceptions of the institution's capacity for management. In addition, those who approved of the organizational management ability for strategic planning had similar perspectives on operational planning, resource mobilization, communication strategy, and knowledge management. This further means that employees from different demographic groups perceived SSC's management capacity similarly. This suggests that the college's practices are viewed as consistent and effective across all employee segments. The study also supported Frederic W. Taylor's Scientific Management Theory, highlighting its importance in improving organizational effectiveness through systematic work processes, task allocation, optimum procedures, performance evaluation, and incentivization. It offers a framework for directing and inspiring employee performance.

Agents of Autonomy: A Systematic Study of Robotics on Modern Hardware

As robots increasingly permeate modern society, it is crucial for the system and hardware research community to bridge its long-standing gap with robotics. This divide has persisted due to the lack of (i) a systematic performance evaluation of robotics on different computing platforms and (ii) a comprehensive, open-source, cross-platform benchmark suite. To address these gaps, we present a systematic performance study of robotics on modern hardware and introduce RoWild, an open-source benchmark suite for robotics that is comprehensive and cross-platform. Our workloads encompass a broad range of robots, including driverless vehicles, pilotless drones, and stationary robotic arms, and we evaluate their performance on a spectrum of modern computing platforms, from low-end embedded CPUs to high-end server-grade GPUs. The source code of the benchmark suite is available in https://cmu-roboarch.github.io/rowild/. Our findings reveal that current architectures experience significant inefficiencies when executing robotic workloads, highlighting the need for architectural advancements that satisfy the primary requirements of robotic tasks. We discuss approaches for meeting these requirements, offering insights for improving the performance of robotics.

Potential Clinical Impact of LAFOV PET/CT: A Systematic Evaluation of Image Quality and Lesion Detection.

We performed a systematic evaluation of the diagnostic performance of LAFOV PET/CT with increasing acquisition time. The first 100 oncologic adult patients referred for 3 MBq/kg 2-[18F]fluoro-2-deoxy-D-glucose PET/CT on the Siemens Biograph Vision Quadra were included. A standard imaging protocol of 10 min was used and scans were reconstructed at 30 s, 60 s, 90 s, 180 s, 300 s, and 600 s. Paired comparisons of quantitative image noise, qualitative image quality, lesion detection, and lesion classification were performed. Image noise (n = 50, 34 women) was acceptable according to the current standard of care (coefficient-of-varianceref < 0.15) after 90 s and improved significantly with increasing acquisition time (PB < 0.001). The same was seen in observer rankings (PB < 0.001). Lesion detection (n = 100, 74 women) improved significantly from 30 s to 90 s (PB < 0.001), 90 s to 180 s (PB = 0.001), and 90 s to 300 s (PB = 0.002), while lesion classification improved from 90 s to 180 s (PB < 0.001), 180 s to 300 s (PB = 0.021), and 90 s to 300 s (PB < 0.001). We observed improved image quality, lesion detection, and lesion classification with increasing acquisition time while maintaining a total scan time of less than 5 min, which demonstrates a potential clinical benefit. Based on these results we recommend a standard imaging acquisition protocol for LAFOV PET/CT of minimum 180 s to maximum 300 s after injection of 3 MBq/kg 2-[18F]fluoro-2-deoxy-D-glucose.