Traditional Evaluation Tools Research Articles

Autoencoder-enabled eye-tracking data analytics for objective assessment of user preference in humanoid robot appearance design

Humanoid robots have been extensively utilized in service industries to provide information and product delivery through direct interactions with users. As the design of humanoid robot appearance significantly impacts human-robot interactions, it is crucial to assess user preference towards it. Traditional evaluation tools, such as surveys, field observations, and interviews, are often time-consuming and subjective. Therefore, this study aims to develop a novel eye-tracking-based assessment tool to investigate user preference towards humanoid robot appearance design. We analyze the critical factors influencing user preference from two perspectives: the attributes of robot appearance and users' selective attention distribution. Accordingly, we propose an integrated machine learning method, combining an autoencoder neural network with a support vector machine to handle the collected visual data. This method, named ASVM, extracts several novel indicators from the eye-tracking data via an unsupervised autoencoder neural network and manual entropy analysis. The proposed ASVM achieves an accuracy of 91%, outperforming other classical machine learning methods, including decision tree, naive Bayes, and support vector machine. ASVM can objectively assess user preference towards humanoid robot appearance design with high time resolution. Furthermore, it can enhance humanoid robot design by revealing the visual attention distribution in assessing robot appearance.

ODs with a positive TPR conclusion, not subject to a conditional approval, and approved without requiring a PASS would be more likely to be reimbursed in Spain

BackgroundThe present study aims to assess clinical and regulatory variables that would influence pricing and reimbursement (P&R) decisions for Orphan Drugs (ODs) in Spain. ODs approved by the European Commission (EC) between 2006 and 2021 were classified according to their P&R status in Spain: approved, undergoing decision and rejected. A statistical analysis was carried out to assess the potential association between clinical and regulatory variables and P&R decision of ODs in Spain: therapeutic area, rarity of disease, existence of alternative therapies, availability of survival-related outcomes, safety profile, type of population, conditional approval status granted by the European Medicines Agency (EMA) and a positive Therapeutic Positioning Report (TPR) opinion.Results111 ODs have been approved by the EC and have obtained marketing authorisation in Spain between 2006 and 2021. Out of the 111 ODs, 57 (51.4%) were reimbursed, 24 (21.6%) were undergoing decision and 30 (27%) were rejected. According to the statistical analysis, ODs with a positive TPR conclusion (p-value < 0.01), not subject to a conditional approval by the EMA (p-value < 0.05) and approved without the obligation to conduct a post-authorisation safety study (PASS) (p-value < 0.05), were statistically significant, and therefore, would be more likely to obtain P&R approval in Spain.ConclusionsThis study shows that the TPR plays a key role in the P&R process in Spain and highlights that traditional evaluation tools, such us safety and efficacy, were the main drivers of P&R decisions for ODs. A positive conclusion of the TPR, non-conditional approval by the EMA and no obligation for a PASS seems to favourably affect P&R decisions in Spain.

A novel sensor-embedded holding device for monitoring upper extremity functions.

There are several causes that can lead to functional weakness in the hands or upper extremities (UE), such as stroke, trauma, or aging. Therefore, evaluation and monitoring of UE rehabilitation have become essential. However, most traditional evaluation tools (TETs) and assessments require clinicians to assist or are limited to specific clinical settings. Several novel assessments might apply to wearable devices, yet those devices will still need clinicians or caretakers to help with further tests. Thus, a novel UE assessment device that is user-friendly and requires minimal assistance would be needed. The cylindrical grasp is one of the common UE movements performed in daily life. Therefore, a cylindrical sensor-embedded holding device (SEHD) for training and monitoring was developed for a usability test within this research. The SEHD has 14 force sensors with an array designed to fit holding positions and a six-axis inertial measurement unit (IMU) to monitor grip strength, hand dexterity, acceleration, and angular velocity. Six young adults, six healthy elderly participants, and three stroke survivors had participated in this study to see if the SEHD could be used as a reference to TETs. During result analyses, where the correlation coefficient analyses were applied, forearm rotation smoothness and the Purdue Pegboard Test (PPT) showed a moderate negative correlation [r (16) = -0.724, p < 0.01], and the finger independence showed a moderate negative correlation with the PPT [r (10) = -0.615, p < 0.05]. There was also a highly positive correlation between the maximum pressing task and Jamar dynamometer in maximum grip strength [r (16) = 0.821, p < 0.01]. These outcomes suggest that the SEHD with simple movements could be applied as a reference for users to monitor their UE ability.

International Experience in Assessing the Effectiveness of Law Enforcement Agencies in Crime Prevention

The objective of the investigation was to analyze some international experience in assessing the effectiveness of law enforcement in crime prevention. Methodologically, the dialectic method, typical of materialistic philosophy, was combined with scientific techniques of cognition. It is concluded that the basis for assessing the effectiveness of law enforcement agencies in different countries is based on a set of quantitative and qualitative criteria; sometimes such criteria conflict with each other, as some are beneficial for bureaucratic reporting, while others reflect the public's interests. Public opinion, as one of the main criteria, is becoming increasingly important in assessing the effectiveness of law enforcement agencies in some countries (e.g., the Slovak Republic), and in the US, France, Japan, Austria, and Italy, it is a traditional evaluation tool. Ambiguous is the use of statistics to assess the effectiveness of law enforcement in certain countries and Finally, there are countries where criminal statistics are a priority to determine the effectiveness of law enforcement and in others not.

The unnoticed impact of long-term cost information on wind farms’ economic value in the USA. – A real option analysis

Renewable energies are a natural replacement for conventional or fossil fuel energy generation. One of these sources, wind energy, has exhibited significant cost reductions during the last decades. Even though this historical cost reduction trend is well known in the industry and in the academia, information on this cost trend and volatility has been scarcely incorporated in previous works, thus a natural concern arises when assessing their influence on the economic valuation of wind farms.Moreover, wind projects are modular in size and have a short time-to-market, providing a valuable managerial flexibility to defer investment under non-attractive market conditions, such as high development cost scenarios. Traditional evaluation tools often consider those scenarios, wrongly assuming that wind-developers would move forward with the project under such conditions.Accordingly, this paper analyzes the effect of incorporating both generation cost history and investment flexibility in the valuation of wind farms in the United States using Real Options methodology. Taking two decades of cost data and a reference wind farm, this “uncertainty-flexibility” effect is incorporated and valued, finding that the value of the wind project is 14.2% larger than the one neglecting such effect – where cost stochasticity and managerial flexibility are ignored. Therefore, traditional tools for project valuation, such as discounted cash flow and scenario analysis can no longer solely be used, as they do not properly account for the cost reduction trend, uncertainty, and managerial flexibility that a wind farm could face.

Using Ripple Effects Mapping to Evaluate a Community-Based Health Program: Perspectives of Program Implementers.

Traditional evaluation tools are oftentimes ill-suited for use in community settings where intervention outcomes of interest may occur at multiple levels and are influenced by interacting factors. Ripple effects mapping (REM) is a participatory technique engaging stakeholders to visually map project/program efforts and results to collect impact data. Using appreciative inquiry, the method helps participants and evaluators understand context for changes resulting from program experiences and is particularly well suited for community-based, participatory programs where impacts often occur beyond the individual level. This article describes the REM method and how it was applied to explore impact and refine program theory of the Extension Wellness Ambassador Program (EWAP), a community-based health-focused master volunteer program, from the perspective of program implementers (n = 10). Insights emerging from the REM session indicate EWAP promotes health behavior change, contributes to community development due to increased leadership capacity, and sustains and grows implementing organizations. The program theory shaping evaluation was refined to better capture impact beyond individual levels. Application of the REM method to a community health program demonstrates feasibility; health promotion practitioners should consider REM to understand program context and capture outcomes that typically evade measurement using traditional techniques.

Evaluating and Co-creating with Student Learning Developers

This is a research article supported by a video. To read the article, please click on the pdf. To play the video, click on the link at the bottom of this page.The Learning Development Centre (LDC) in the School of Health &amp; Life Sciences at Glasgow Caledonian University serves over 6000 students, of whom over 70% have off-campus placement requirements. Provision of high-quality online support materials is therefore a priority. The LDC team has developed a suite of contextualised online resources to assist students in their development of academic and digital literacies. The LDC Team undertook a project to evaluate and enhance the resources and the LDC website. This involved the use of traditional evaluation tools in addition to the more innovative use of the 'Think Aloud Method' (Ericsson and Simon, 1984), to assess interface usability and online community content. A key to this project was the Team’s partnership with students, as ‘Student Learning Developers’ who played a pivotal role in the evaluation and co-creation of resources. This has helped to embed a culture of student collaboration within the LDC. In addition to showcasing the community and sharing students’ experiences of participating in the project, this submission will include discussion about effective ways of creating, evaluating and promoting online learning development resources, together with the LDC team’s reflections on partnership working with students.

Improving evaluation at two medical schools.

Student evaluations of teaching can provide useful feedback for teachers and programme coordinators alike. We have designed a novel evaluation tool assessing teacher performance and student learning outcome. This tool was implemented at two German medical schools. In this article, we report student and teacher perceptions of the novel tool, and the implementation process. Focus group discussions as well as one-to-one interviews involving 22 teachers and 31 undergraduate medical students were conducted. Following adjustments to the feedback reports (e.g. the colour coding of results) at one medical school, 42 teachers were asked about their perceptions of the revised report and the personal benefit of the evaluation tool. Teachers appreciated the individual feedback provided by the evaluation tool and stated that they wanted to improve their teaching, based on the results; however, they missed most of the preparative communication. Students were unsure about the additional benefit of the instrument compared with traditional evaluation tools. Amajority was unwilling to complete evaluation forms in their spare time, and some felt that the new questionnaire was too long and that the evaluations occurred too often. They were particularly interested in feedback on how their comments have helped to further improve teaching. Student evaluations of teaching can provide useful feedback CONCLUSION: Despite evidence of the utility of the tool for individual teachers, implementation of changes to the process of evaluation appears to have been suboptimal, mainly owing to a perceived lack of communication. In order to motivate students to provide evaluation data, feedback loops including aims and consequences should be established.

Evaluation of urban underground space resources using digitalization technologies

Exploitation and utilization of urban underground space resources is gaining increasing attention in cities’ sustainable development. Due to the attributes of fragility and irreversibility, urban underground space resources can be extremely vulnerable to damage and it is often difficult in restoring their original state once damaged. Therefore, scientific and reasonable evaluation of urban underground space resources is of vital importance in urban planning and development. This paper first describes the content and methods for the evaluation of urban underground space resources, and introduces the digitalization technologies to handle the problems of strong subjectivity, low efficiency and rough precision encountered when using traditional evaluation tools. Then, the related concepts and main technologies for digitalization were demonstrated and an intelligent GIS-based engineering geology system was built. Based on the drilling information in Changzhou City, stratum standardization and 3D modeling of strata, underground structures and pipelines were conducted in the built intelligent system. Finally, with the help of spatial overlay analysis in the GIS-based platform and a combination of other digitalization technologies, the urban underground space resources of Changzhou City were evaluated. Maps of geological suitability were produced, which provide scientific guidance and reference for the exploitation and planning of urban underground space resources in Changzhou City. In addition, site selection for key projects was conducted based on the evaluation results using this platform.

Modelling experts' behavior with e-valUAM to measure computer science skills

In this work we present an evaluation method that focuses on experts' behavior instead of the traditional scores based just on the number of correct answers. The method presented here is especially suitable to measure the skills in Computer Science since this is a wide discipline very difficult to evaluate due to the many facts publicly available on the Internet. By using traditional evaluation tools, it is very difficult to measure the real knowledge of the users since they can correctly answer even without having acquired formal academic knowledge. To use this method, we have developed a test that can detect significant differences between standard users and experts in Computer Science. The test is applied by the e-valUAM application, which has been modified to store several parameters from the users' answers. By optimizing the parameters by a linear model, we have developed an equation that can be used to quantitatively compare the results of a single user with the results from the reference group of experts. This optimization is only possible because this group shows good stability and gives statistically different results compared to the other groups. The scores achieved with our method can be used to predict the formal knowledge of the users and modify their training when needed.

Piloting an outcome-based programme evaluation tool in undergraduate medical education.

Aims: Different approaches to performance-oriented allocation of resources according to teaching quality are currently being discussed within German medical schools. The implementation of these programmes is impeded by a lack of valid criteria to measure teaching quality. An assessment of teaching quality should include structural and procedural aspects but focus on learning outcome itself. The aim of this study was to implement a novel, outcome-based evaluation tool within the clinical phase of a medical curriculum and address differences between the novel tool and traditional evaluation methods. Methods: Student self-assessments before and after completion of a teaching module were used to compute performance gains for specific learning objectives. Mean performance gains in each module were compared to student expectations before the module and data derived from a traditional evaluation tool using overall course ratings at the end of the module.Results: A ranking of the 21 modules according to computed performance gains yielded entirely different results than module rankings based on overall course ratings. There was no significant correlation between performance gain and overall ratings. However, the latter were significantly correlated to student expectations before entering the module as well as structural and procedural parameters (Pearson’s r 0.7-0.9).Conclusion: Performance gain computed from comparative self-assessments adds an important new dimension to course evaluation in medical education. In contrast to overall course ratings, the novel tool is less heavily confounded by construct-irrelevant factors. Thus, it appears to be more appropriate than overall course ratings in determining teaching quality and developing algorithms to guide performance-oriented resource allocation in medical education.

An Empirical Analysis of What Counts as Attributes for Vocational Technical Educational Leaders in Ghana

Effective leaders are needed to manage vocational technical educational institutions in Ghana as government plans to raise the skill level of its workforce. Leadership development programmes need to be designed for the preparation and in-service training of vocational technical leaders. This study aimed at identifying what leadership attributes need to be developed in vocational technical institutional leaders and what learning activities need to be incorporated into training programs for these leaders. The study revealed that vocational technical leaders demonstrated managerial skills more often than they do demonstrate social skill and leadership skills. They also demonstrated attributes such as energetic, achieve- ment-oriented, accepting responsibility, insightfulness, networking, influencing, and intelligent with practical judgement more often than they do demonstrate attributes such team building, openness, motivating others, tolerance, conflict man- agement, and sensitivity. It is therefore recommended that the Leadership Attributes Inventory (LAI) be utilized concurrence with traditional evaluation tools for potential candidates for vocational technical institution leaders. Any curriculum designed for leadership development of vocational technical educational leaders should be measured against the attributes identified by the Leadership Attributes Inventory to ensure that they addressed the development of needed attributes.

As Accessible as a Book on a Library Shelf: The Imperative of Routine Simulation in Modern Health Care

As Accessible as a Book on a Library Shelf: The Imperative of Routine Simulation in Modern Health Care

Effects of NutriPlus Program on Child Nutritional Status and Guardian's Nutrition Knowledge and Attitudes

Description: DCOP is being developed as a part of the eXtension (http://www.extension.org), the virtual educational environment of the Cooperative Extension System. DCOP involves a multistate and multidisciplinary collaboration, including diabetes educators, public health and extension professionals, and clinicians. Development of the interactive Web site includes evidenced-based userfriendly information, instructional videos, fact sheets, frequently asked questions and answers for the consumers, and educational toolkits for the professionals. Evaluation: Development of the site requires rethinking of traditional evaluation tools that are not fully applicable to a virtual content. Logic model, process evaluation, design ofWeb pages with evaluation inmind, Google Analytics, and Instant Survey are some of the evaluation approaches. Conclusions and Implications: This presentation describes the development process of the DCOP and can help as a guide for educators when they develop virtual programs. Funding: USDA-eXtension.

Web analytics in library practice: Exploration of issues

AbstractThis study examines the use of web analytics in libraries to understand how this tool can be used to interpret users' behavior on the library's website. Evaluating library and information services is important for library management decision‐making regarding the quality of library services. Because such decisions‐making is time consuming and requires investment of many resources, libraries are constantly looking for efficient approaches that would provide insights into planning and changes to the library services. The study data consist of reports collected by Google analytics on the University of Missouri's (MU) library website and interactive group interviews with the library's usability committee. The poster presents the preliminary findings and discusses the issues of implementing web analytics in a library setting. The study findings would benefit libraries in understanding how web analytics can be used as an evaluation tool for libraries, especially as an addition to the traditional evaluation tools.

Successful CEOs of Private Equity Funded Ventures

There is wide agreement that the quality of top managerial leadership greatly influences the success of any business. The character and personality of the leader can drive business outcomes. Leadership quality is more important in new ventures than in established companies. Private equity investors need to find and use the most effective tools. This article argues that clinical psychological techniques can provide unique insights into the leadership quality of a prospective manager. We argue that private equity should learn how to become intelligent consumers of these methods. In particular, we believe that projective and semi-projective techniques provide the most valuable information. Using them in combination with more traditional evaluation tools should substantially increase the confidence that investors have selected the most capable executive, thereby enhancing private equity returns in portfolios of new venture investments.

Integrating Neurobehavioral Concepts into Early Intervention Eligibility Evaluation

Most screening and evaluation instruments used in early intervention are limited in their ability to describe the complex, behavioral repertoire that newborn infants bring to the world. As a result, a traditional eligibility evaluation may not capture the unique behavioral strengths and needs of each infant. The integration of qualitative neurobehavioral observation into the initial eligibility evaluation enriches both the process and the results for parents and interventionists. By combining traditional evaluation tools with neurobehavioral observation, early interventionists can better meet the needs of parents in the early weeks following homecoming while also complying with the intent of the federal and state legal mandates.

Lesson Plan Analysis As a Program Evaluation Tool

The use of teacher lesson plans as a tool to measure program effectiveness was explored as a part of an overall evaluation of a gifted program in grades 1 to 6. A Lesson Plan Evaluation Form (LPEF) was developed to provide systematic quantitative data about classroom functioning that are usually obtained only through attitudinal surveys. One hundred thirty-seven plans from 15 teachers were analyzed. The lesson plan analysis yielded useful information about program functioning in a cost-effective manner. Data from the LPEF were consistent with data from more traditional evaluation tools. The use of lesson plan analysis as a tool to measure program effectiveness should be further investigated.