Automatic Evaluation Tools Research Articles

Sustainability analysis of biodiesel production from Cynara Cardunculus crop

The presented work analyses the feasibility of biodiesel production by transesterification of vegetable oil extracted from Cynara Cardunculus. The performance assessment of a plant with a capacity of 5000t/year biodiesel production is carried out based on its rigorous simulation in AspenHysys®. A modular automated evaluation tool programmed in Matlab® retrieves the inventory of the energy and material inputs/outputs, and the environmental releases from the simulation results. The simulation performs a combined acid-catalyzed pretreatment and alkali-catalyzed transesterification based on the cardoon oil characterization and the kinetic data extracted from published experimental studies. The performance of the plant is optimized considering four potential alternatives. The Eco-indicator 99 methodology is used for the impact assessment, achieving similar results to other vegetable sources of biodiesel. With respect to the economic study, a profitability analysis yields better results than the reported in previous works for oils of other agricultural crops, identifying two critical factors: the biodiesel sale price and the plant capacity.

Measuring Walkability: Development of an Automated Sidewalk Quality Assessment Tool

Sidewalks are a critical part of sustainable transportation systems, supporting pedestrian travel and healthy physical activity. Presence and quality of sidewalks is a significant predictor of perceived safety and quality of the pedestrian environment. Lack of adequate pedestrian infrastructure data has been identified as a major barrier to large-scale pedestrian planning. Sidewalk presence, width, and surface condition are identified as important indicators of facility quality and accessibility. Georgia Tech is deploying an Android tablet application to automatically generate spatial sidewalk inventories, automatically assess sidewalk quality, and prioritize sidewalk repairs. The research team has collected field data on sidewalk segments across the City of Atlanta and deployed a quality evaluation survey of selected segments to transportation experts in order to obtain baseline sidewalk quality expert ratings. Researchers then link expert ratings from survey results back to the field data to complete the automated Sidewalk Quality Tool. The sidewalk assessment ratings developed through this project can provide a baseline for municipal and regional sidewalk inventories, influencing transportation decision-making and improving quality of life and public health. The development of automated evaluation tools will help to address current challenges in prioritizing infrastructure improvements, fostering walkability and livability for cities and suburbs.

Automatic evaluation for e-learning using latent semantic analysis: A use case

Assessment in education allows for obtaining, organizing, and presenting information about how much and how well the student is learning. The current paper aims at analysing and discussing some of the most state-of-the-art assessment systems in education. Later, this work presents a specific use case developed for the Universitat Oberta de Catalunya, which is an online university. An automatic evaluation tool is proposed that allows the student to evaluate himself anytime and receive instant feedback. This tool is a web-based platform, and it has been designed for engineering subjects (i.e., with math symbols and formulas) in Catalan and Spanish. Particularly, the technique used for automatic assessment is latent semantic analysis. Although the experimental framework from the use case is quite challenging, results are promising.

RgbF: An Open Source Tool for n-gram Based Automatic Evaluation of Machine Translation Output

rgbF: An Open Source Tool for n-gram Based Automatic Evaluation of Machine Translation Output We describe RGBF, a tool for automatic evaluation of machine translation output based on n-gram precision and recall. The tool calculates the F-score averaged on all n-grams of an arbitrary set of distinct units such as words, morphemes, POS tags, etc. The arithmetic mean is used for n-gram averaging. As input, the tool requires reference translation(s) and hypothesis, both containing the same combination of units. The default output is the document level 4-gram F-score of the desired unit combination. The scores at the sentence level can be obtained on demand, as well as precision and/or recall scores, separate unit scores and separate n-gram scores. In addition, weights can be introduced both for n-grams and for units, as well as the desired n-gram order n.

Enterprise Web Accessibility Levels Amongst the Forbes 250: Where Art Thou O Virtuous Leader?

The research team measured the enterprise web accessibility levels of the Forbes 250 largest enterprises using the fully automatic accessibility evaluation tool Sortsite, and presented the compliance of the evaluated websites to WCAG 1.0, WCAG 2.0 and Section 508 accessibility levels. Given the recent attention to organizational leaders having ethical duties towards their dedicated employees, we propose that ‘societal citizenship behaviour’ concerns ethical duties of organizational leaders towards society in general and in particular to those who have less means to assert their needs. In effect, we found enterprise website accessibility levels to be in need of significant improvement. An interpretation of a positive path forward to better enterprise website accessibility levels is put forth based on a focus-group interaction and using BNML—a novel Business Narrative Modelling Language.

A new Internet tool for automatic evaluation in Control Systems and Programming

In this paper we present a web-based innovative education tool designed for automating the collection, evaluation and error detection in practical exercises assigned to computer programming and control engineering students. By using a student/instructor code-fusion architecture, the conceptual limits of multiple-choice tests are overcome by far. The proposed system is also able to individually parameterize the exercises for each student and allows an instructor to implement innovative self-learning techniques in which student can obtain a measure of their knowledge continuously along the whole course length. The proposed automatic evaluation system has been applied to several control engineering courses at the University of Seville and the results several case studies are presented here. ► A new paradigm for automatic evaluation, via conversion of student answers into executable code. ► The code is executed along with instructor-defined code, in order to compute a grade. ► Solid empirical evidence: over 2000 automatic evaluations carried out in real-life. ► Features: 100% web-based, flexible and multidisciplinary (Control Systems, Mechanics, Science…). ► Strong security features for fraud protection.

Automatic Assessment of 3D Modeling Exams

Computer-based assessment of exams provides teachers and students with two main benefits: fairness and effectiveness in the evaluation process. This paper proposes a fully automatic evaluation tool for the Graphic and Virtual Design (GVD) curriculum at the First School of Architecture of the Politecnico di Torino, Italy. In particular, the tool is designed for the 3D modeling course, taught during the second year, where students have to prove their ability to model static scenes using the open source modeler Blender. During the final exam, students are required to create a 3D model as similar as possible to a reference object proposed by the teacher and shown through a set of 2D views; the similarity of the images is judged according to both model shape and materials. The traditional assessment process is particularly slow and strongly based on teachers subjective evaluation; the proposed solution efficiently implements an objective assessment mechanism that exploits computer vision and image analysis algorithms to automatically extract similarity indices. These indices are related to partial evaluation grades, which are then combined to obtain the final mark. A comparison with the traditional assessment process shows robustness and trustworthiness of the designed approach.

Drugs of abuse screening in urine as part of a metabolite-based LC-MSn screening concept

Today, immunoassays and several chromatographic methods are in use for drug screening in clinical and forensic toxicology and in doping control. For further proof of the authors' new metabolite-based liquid chromatography-mass spectrometry (LC-MS(n)) screening concept, the detectability of drugs of abuse and their metabolites using this screening approach was studied. As previously reported, the corresponding reference library was built up with MS(2) and MS(3) wideband spectra using a LXQ linear ion trap with electrospray ionization in the positive mode and full scan information-dependent acquisition. In addition to the parent drug spectra recorded in methanolic solution, metabolite spectra were identified after protein precipitation of urine from rats after administration of the corresponding drugs and added to the library. This consists now of data of over 900 parent compounds, including 87 drugs of abuse, and of over 2,300 metabolites and artifacts, among them 436 of drugs of abuse. Recovery, process efficiency, matrix effects, and limits of detection for selected drugs of abuse were determined using spiked human urine, and the resulting data have been acceptable. Using two automatic data evaluation tools (ToxID and SmileMS), the intake of 54 of the studied drugs of abuse could be confirmed in urine samples of drug users after protein precipitation and LC separation. The following drugs classes were covered: stimulants, designer drugs, hallucinogens, (synthetic) cannabinoids, opioids, and selected benzodiazepines. The presented LC-MS(n) method complements the well-established gas chromatography-mass spectroscopy procedure in the authors' laboratory.

MAISTAS: a tool for automatic structural evaluation of alternative splicing products

Motivation: Analysis of the human genome revealed that the amount of transcribed sequence is an order of magnitude greater than the number of predicted and well-characterized genes. A sizeable fraction of these transcripts is related to alternatively spliced forms of known protein coding genes. Inspection of the alternatively spliced transcripts identified in the pilot phase of the ENCODE project has clearly shown that often their structure might substantially differ from that of other isoforms of the same gene, and therefore that they might perform unrelated functions, or that they might even not correspond to a functional protein. Identifying these cases is obviously relevant for the functional assignment of gene products and for the interpretation of the effect of variations in the corresponding proteins.Results: Here we describe a publicly available tool that, given a gene or a protein, retrieves and analyses all its annotated isoforms, provides users with three-dimensional models of the isoform(s) of his/her interest whenever possible and automatically assesses whether homology derived structural models correspond to plausible structures. This information is clearly relevant. When the homology model of some isoforms of a gene does not seem structurally plausible, the implications are that either they assume a structure unrelated to that of the other isoforms of the same gene with presumably significant functional differences, or do not correspond to functional products. We provide indications that the second hypothesis is likely to be true for a substantial fraction of the cases.Availability: http://maistas.bioinformatica.crs4.it/.Contact: anna.tramontano@uniromal.it

Assessing the Usability and Accessibility of Malaysia E-Government Website

Problem statement: With the proliferation number existence of Malaysia e-government websites, the usability and accessibility feature may has been overlooked by website developers. Approach: The main objective of this study is to investigate the usability and accessibility of Malaysia e-government websites. The usability measures are being measured by using Nielson usability guideline for the uploading speed and page size of the main page and number of broken links. The accessibility is measured by using Web Content Accessibility Guidelines 1.0 (WCAG). Samples of 155 Malaysia e-government websites were selected from federal government websites and state government websites available at http://www.malaysia.gov.my. The usability and accessibility of the websites was analyzed by using several automatic evaluation tools such as Websiteoptimization, Axandra and EvalAccess 2.0 tools. Results: The evaluation process revealed several issues on usability and accessibility of Malaysia e-government website. There is high number of usability (speed and number of broken links) and accessibility problems for state website upon comparing to federal website. Conclusion/Recommendation: This study provides few recommendations for further improvement of the usability and accessibility features of e-government website based on the highlighted issues and key findings reported in this study.

Web-based Tool for Automatic Assignment Evaluation

The demands for teachers in most educational institutions are continually increasing. Teachers have to teach an increasing number of students every year while simultaneously providing meaningful and constructive feedback and correcting assignments in a minimal amount of time. Consequently, results and assessment of student’s progress can be considerably delayed. We need easy to use technology for grading and assessing test results and communicating information to students. Changes can be brought about to help teachers become more efficient and effective. This paper examines the role of technology and Web-based software in the classroom for evaluating a students’ assignment. The paper has used both primary and secondary sources of data.

A new web–based tool for education and automatic evaluation in Control Systems Engineering

In this paper we present a web–based innovative education tool designed for automating the collection, evaluation and error detection in practical exercises assigned to Control Engineering students. The system is implemented on a Apache–PHP–MySQL database–driven server and uses Matlab to compare, on one hand the solutions provided by students (by executing their code) with the correct solutions in order to automatically give a grade. The proposed system is also able to individually parameterize the exercises for each student. By using a student–code execution architecture, the limitations of multiple-choice tests in practical exercises are overcome. The open platform allows a professor to implement innovative self-learning techniques in which the student can obtain a measure of their knowledge continuously along the whole course length. The proposed automatic evaluation system has been applied to several control engineering courses at the University of Seville and the results of a case study are presented here.

Color‐based tumor tissue segmentation for the automated estimation of oral cancer parameters

This article presents an automatic color-based feature extraction system for parameter estimation of oral cancer from optical microscopic images. The system first reduces image-to-image variations by means of color normalization. We then construct a database which consists of typical cancer images. The color parameters extracted from this database are then used in automated online sampling from oral cancer images. Principal component analysis is subsequently used to divide the color features into four tissue types. Each pixel in the cancer image is then classified into the corresponding tissue types based on the Mahalanobis distance. The aforementioned procedures are all fully automated; in particular, the automated sampling step greatly reduces the need for intensive labor in manual sampling and training. Experiments reveal high levels of consistency among the results achieved using the manual, semiautomatic, and fully automatic methods. Parameter comparisons between the four cancer stages are conducted, and only the mean parameters between early and late cancer stages are statistically different. In summary, the proposed system provides a useful and convenient tool for automatic segmentation and evaluation for stained biopsy samples of oral cancer. This tool can also be modified and applied to other tissue images with similar staining conditions.

A model for online consumer health information quality

AbstractThis article describes a model for online consumer health information consisting of five quality criteria constructs. These constructs are grounded in empirical data from the perspectives of the three main sources in the communication process: health information providers, consumers, and intermediaries, such as Web directory creators and librarians, who assist consumers in finding healthcare information. The article also defines five constructs of Web page structural markers that could be used in information quality evaluation and maps these markers to the quality criteria. Findings from correlation analysis and multinomial logistic tests indicate that use of the structural markers depended significantly on the type of Web page and type of information provider. The findings suggest the need to define genre‐specific templates for quality evaluation and the need to develop models for an automatic genre‐based classification of health information Web pages. In addition, the study showed that consumers may lack the motivation or literacy skills to evaluate the information quality of health Web pages, which suggests the need to develop accessible automatic information quality evaluation tools and ontologies.

Algebra of Systems: A Metalanguage for Model Synthesis and Evaluation

This paper represents system models as algebraic entities and formulates model transformation activities as algebraic operations. We call this modeling framework ldquoalgebra of systemsrdquo (AoS). To show that AoS can automate complex model reasoning tasks in system design projects, we implemented the abstract algebraic specification as an executable metalanguage named object-process network, which serves as a tool for automatic model transformation, enumeration, and evaluation. A case study of the Apollo lunar landing mission design is developed using this algebraic modeling approach.

Good visual aesthetics equals good web accessibility

This paper summarises a doctoral research project titled Empirical Investigation of Visual Aesthetics and Accessibility (EIVAA). The project proposes to investigate the relationship between visual aesthetics and Web accessibility, with the aim of building a framework to guide the development of aesthetic accessible Web content. As a first step, a formative study was conducted to examine the relationship between visual aesthetics and Web accessibility from a technical perspective. Thirty Web pages that had previously been rated on various aspects of their visual appearance in a study by Michailidou et al. [20] were investigated. The visual aesthetic dimensions considered in their study were "clean", "clear", "organised", "beautiful", and "interesting". In our study, all thirty Web pages were tested for accessibility using an automatic evaluation tool, "Cynthia Says". Initial results show that Web pages which were previously agreed by participants to be visually clean, clear, and organised had lesser numbers of Web Content Accessibility Guidelines (WCAG) 1.0 failures. This suggests that visually pleasing pages along these design dimensions may not pose a barrier to accessibility. The implications of our study and future research directions are discussed.

Tool independence for the web accessibility quantitative metric

The Web Accessibility Quantitative Metric (WAQM) aims at accurately measuring the accessibility of web pages. One of the main features of WAQM among others is that it is evaluation tool independent for ranking and accessibility monitoring scenarios. This article proposes a method to attain evaluation tool independence for all foreseeable scenarios. After demonstrating that homepages have a more similar error profile than any other web page in a given web site, 15 homepages were measured with 10,000 different values of WAQM parameters using EvalAccess and LIFT, two automatic evaluation tools for accessibility. A similar procedure was followed with random pages and with several test files obtaining several tuples that minimise the difference between both tools. One thousand four hundred forty-nine web pages from 15 web sites were measured with these tuples and those values that minimised the difference between the tools were selected. Once the WAQM was tuned, the accessibility of 15 web sites was measured with two metrics for web sites, concluding that even if similar values can be produced, obtaining the same scores is undesirable since evaluation tools behave in a different way.

Methods for Evaluating Online, Resource-based Learning Environments for Teachers

Online delivery methods offer much promise for anytime, anywhere adult learning. As a method of outreach, and to provide support for just-in-time learning, teacher educators are increasingly deciding to design Web sites that are online, resource-based learning environments for teachers and preservice teachers. Automated evaluation tools and data collection methods can help such Web site designers develop and use online, resource-based learning environments to meet their goals of providing high quality learning opportunities for teachers. This article presents a three-pronged approach for evaluating issues of pedagogical design and user-centered functionality in online, resource-based Web sites. Checklist, Web site traffic analysis, and talk aloud protocols art each described and then illustrated through their application to a site aimed at educating teachers about technology integration. We discuss the relevance for developing effective learning environments when teacher educators apply this evaluation approach to resource-based learning environments.

Detecting Anomaly and Failure in Web Applications

Improving Web application quality will require automated evaluation tools. Many such tools are already available either as commercial products or research prototypes. The authors use their automated evaluation tools, ReWeb and TestWeb, for Web analysis and testing that improves Web pages and applications and to find some anomalies and failures in four case studies.

Statistical analysis of combined dose effects for experiments with two agents

Classical isobologram analysis offers a way for analysing combined drug effects in dose-response experiments statistically. The aim is to determine as to whether two agents or drugs can be considered synergistic or antagonistic in their effect. We describe a MATLAB-based software tool for automated isobologram analysis and computation of combination indices. Statistical issues like estimation together with respective confidence intervals are of key interest. Additional predictive values are computed to facilitate a more easy interpretation of obtained results. Analysis of an experimental and a real in vitro data set demonstrates the approach and the way of interpreting results. Results are summarized in two ways: tables and graphical displays containing classical isobolograms. Our package supplements the clinical software-equipment and is a tool for automatic evaluation of combined dose-response experiments in experimental oncology in the urologic clinic.