Third-party Evaluation Research Articles

Third-party evaluators perceive AI as more compassionate than expert humans

Empathy connects us but strains under demanding settings. This study explored how third parties evaluated AI-generated empathetic responses versus human responses in terms of compassion, responsiveness, and overall preference across four preregistered experiments. Participants (N = 556) read empathy prompts describing valenced personal experiences and compared the AI responses to select non-expert or expert humans. Results revealed that AI responses were preferred and rated as more compassionate compared to select human responders (Study 1). This pattern of results remained when author identity was made transparent (Study 2), when AI was compared to expert crisis responders (Study 3), and when author identity was disclosed to all participants (Study 4). Third parties perceived AI as being more responsive—conveying understanding, validation, and care—which partially explained AI’s higher compassion ratings in Study 4. These findings suggest that AI has robust utility in contexts requiring empathetic interaction, with the potential to address the increasing need for empathy in supportive communication contexts.

Does physical exercise enhance physical appearance? An empirical study based on CFPS

BackgroundIn contemporary society, physical appearance significantly influences individuals' social interactions and career achievements. Although some studies suggest that physical exercise may positively impact physical appearance, most rely on subjective self-assessment data. In contrast, third-party evaluations are widely regarded as more objective and reliable. Thus, this study aims to explore the effects of physical exercise on physical appearance, as assessed by third-party evaluations, and to examine its heterogeneity.MethodsThis study utilized third-party physical appearance evaluation data from 25,460 respondents in the 2018 China Family Panel Studies (CFPS) and assessed the impact of physical exercise on physical appearance using multiple linear regression and instrumental variable analysis. Furthermore, the heterogeneity in the effects of physical exercise was investigated using subgroup regression and quantile regression analyses.ResultsThe findings indicate that physical exercise substantially enhances physical appearance, with its effects varying significantly across different populations. Specifically, women, urban residents, and those with lower appearance scores experience more pronounced enhancement from physical exercise compared to men, rural residents, and those with higher scores. Additionally, the effect of age on the impact of physical exercise on appearance exhibits an inverted U-shaped relationship. Middle-aged adults (40–59 years) experience the most significant improvements, while benefits are lower for adolescents (10–20 years), young adults (20–39 years), and older adults (60–80 years).ConclusionsThis study systematically reveals the positive impact of physical exercise on physical appearance, demonstrating that regular exercise can significantly enhance individual appearance scores. This finding contributes to a broader understanding of the multifaceted benefits of physical exercise.

The Challenges for EU User Testing Policies for Patient Information Leaflets.

Patient information leaflets (PILs) are essential tools in healthcare, providing crucial information about medication use. In the European Union, the European Medicines Agency (EMA) oversees the regulation and standardisation of PILs to ensure their readability and accessibility. However, challenges persist in ensuring these documents are comprehensible and user-friendly. This study employs a qualitative analytical approach, reviewing existing literature and regulatory documents to identify gaps in the EU user testing policies for PILs. It focuses on the diversity of participant samples, the independence of the testing process, and the robustness of user testing protocols. Findings indicate that current user testing practices often lack diversity and may be biased when pharmaceutical companies conduct their own tests. Additionally, there is a lack of user testing protocols for translated PILs, potentially compromising their accuracy and cultural relevance. To improve the efficacy of PILs, it is essential to include diverse and representative samples in user testing, mandate independent third-party evaluations, implement protocols for user testing on translated PILs, and ensure continuous updates to guidelines based on the latest best practices in health communication. These measures will enhance patient safety and understanding of medication information.

Correlation Analysis of ESG Ratings and Enterprise Performance

ESG (environment, society and governance) evaluation system, as a tool to comprehensively consider the sustainable development ability of enterprises, is increasingly receiving attention. In the context of the promotion of the concept of global sustainable development and the pursuit of high-quality economic development, many third-party evaluation institutions and index research organizations at home and abroad are actively building and improving the ESG rating framework. Through systematic review of domestic and foreign academic circles of ESG rating and enterprise performance correlation research results, this paper reveals an important discovery: ESG rating mechanism not only provides the mirror of self, make it can accurate positioning and optimization in environmental protection, social responsibility and corporate governance, also became the driving enterprises to the direction of more green, harmonious and efficient key force, and promote the long-term sustainable development of the enterprise.

What Gets Measured, Gets Managed: The Role of Sustainability Assurance in Green Transformation

Sustainability assurance (SA), an independent third-party evaluation aimed at enhancing the credibility of corporate sustainability disclosure, plays a vital role in fostering corporate green transformation. This paper systematically examines the impact of SA on corporate green transformation and the mechanisms through which it operates, using data on SA reports and green patents from A-share listed companies in China between 2010 and 2022. The findings reveal that SA significantly promotes corporate green transformation, particularly when conducted in accordance with standard ISAE 3000. Mechanism analysis indicates that SA facilitates green transformation by reducing the cost of credit and promoting risk-taking behaviors. Heterogeneity analysis further demonstrates that the positive effects of SA on green transformation are more pronounced in heavily polluting and competitive industries as well as in firms with weaker internal and external governance. This study not only underscores the beneficial impact and underlying mechanisms of SA on corporate green transformation but also contributes to the broader literature on the drivers of corporate green transformation.

Teaching on the Fly: A Needs Assessment for an Ambulatory Resident-as-Teacher Workshop.

Residents play an important role in medical education, yet often feel unprepared without formal training. Teaching in the ambulatory setting raises unique challenges such as the difficulty of educating in a limited amount of time. We designed a brief, focused intervention as an initial needs assessment for a residents-as-teachers program in an ambulatory setting to address these concerns. A 1-day, 2.5-hour workshop was designed focusing on microskills, providing feedback, and ways to address common barriers in ambulatory teaching. Pre- and post-intervention surveys were conducted with both residents and medical students to assess the effects of the workshop on resident teaching in the clinic. Although post-intervention surveys showed increased resident confidence and self-reported teaching behaviors, medical student surveys did not clearly demonstrate an increase in teaching behaviors. Didactic teaching on feedback and microskills with follow-on role playing were seen as the most helpful parts of the intervention. Self-assessment alone is an inadequate measure of effectiveness of our teaching intervention. While medical student data can help verify resident self-report, future iterations of our intervention should incorporate objective, third-party evaluation of teaching skill implementation.

The Influence of Dr. Rima Rudd's Organizational Health Literacy Scholarship in Maryland.

This article analyzes and reflects on Dr. Rima Rudd's organizational health literacy ideas and tools and their influence on the field generally and on four projects over 12 years in Maryland specifically. We present four organizational health literacy projects - two from oral health and two from COVID-19 vaccination - that used or were influenced by Dr. Rudd's the Health Literacy Environment of Hospitals and Health Centers. In the oral health projects, we describe the organizational assessments we conducted, the assessment results, and the actions organizations took in response. In a Frederick, Maryland, COVID-19 project, we worked with multiple organizations in a single city to train them in the organizational assessment process, and we report the activities and results of this training. In the Baltimore, Maryland COVID-19 project, we provided general information about organizational health literacy and trained key health professionals in local organizations. Our results confirm that Dr. Rudd's tools work mainly as intended because they help organizations or third-party evaluators identify health literacy barriers and create health literacy insights. Also, we observed that organizational health literacy tools can support organizations' interest in equity goals and increase their willingness to spend time on health literacy projects. Translating knowledge and skills to actions can require more time than organizations can commit or be more difficult than they can handle. In our projects, the four most positive examples were driven by a collaboration between our team and a change champion who had the power to institute new ideas and actions. While it can take time and money to gain traction, our Maryland work shows that organizational assessments are accessible, practical and tangible. We conclude that Dr. Rudd's influence extends beyond specific tools and is reflected in the field's acceptance of organizational and professional responsibility for health literacy as an equity and justice issue. [HLRP: Health Literacy Research and Practice. 2024;8(3):e151-e158.].

Acciones para establecer el esquema de certificación de Sistemas de Gestión de Organizaciones Educativas basado en NC-ISO 21001:2019 en Cuba

To demonstrate their quality, higher education institutions can be accredited by sectoral educational quality schemes or certify their management systems, following the approaches of the International Organization for Standardization. The objective of this work is to present the actions carried out in Cuba to establish the certification scheme, according to NC-ISO 21001:2019. To this end, a descriptive research was carried out, with a qualitative approach, supported by the analysis of scientific articles and other documents that substantiate the importance of the SGOE. The focus group technique was applied to know the experts' perceptions about the relevance of establishing this certification in the country. The necessary actions were planned to establish the scheme and it was designed with the University of Havana, as owner. The certification of Management Systems for Educational Organizations will allow institutions that provide higher-level and postgraduate training services to demonstrate the quality of their educational processes, through compliance with the NC-ISO 21001:2019 standard, by a third-party evaluation scheme, which includes educational organizations that are not governed by the sectoral accreditation system established in Cuba.

Evaluation of output factors of different radiotherapy planning systems using Exradin W2 plastic scintillator detector.

This study aims to evaluate the output factors (OPF) of different radiation therapy planning systems (TPSs) using a plastic scintillator detector (PSD). The validation results for determining a practical field size for clinical use were verified. The implemented validation system was an Exradin W2 PSD. The focus was to validate the OPFs of the small irradiation fields of two modeled radiation TPSs using RayStation version 10.0.1 and Monaco version 5.51.10. The linear accelerator used for irradiation was a TrueBeam with three energies: 4, 6, and 10MV. RayStation calculations showed that when the irradiation field size was reduced from 10 × 10 to 0.5 × 0.5cm2, the results were within 2.0% of the measured values for all energies. Similarly, the values calculated using Monaco were within approximately 2.0% of the measured values for irradiation field sizes between 10 × 10 and 1.5 × 1.5cm2 for all beam energies of interest. Thus, PSDs are effective validation tools for OPF calculations in TPS. A TPS modeled with the same source data has different minimum irradiation field sizes that can be calculated. These findings could aid in verification of equipment accuracy for treatment planning requiring highly accurate dose calculations and for third-party evaluation of OPF calculations for TPS.

Signaling outrage is a signal about the sender: moral perceptions of online flaming

Abstract Most flaming research addresses explanations for and the immediate effects of flaming on those engaging in and targeted by flaming. However, online interactions are increasingly visible, suggesting that understanding third-party evaluations of flaming is important. By integrating considerations in computer-mediated communication theorizing with the social-perceptual effects of online moral outrage, we explore how third-party observers evaluate flaming, also assessing beliefs about the signaling social function that flaming serves. In seven experiments (total N = 3,178), we manipulated the intentionality of triggering events and compared flaming to other types of online responses (less-toxic criticism; supportive), measuring reactions including moral regard, comment approval, and positive/negative engagement. Findings suggest that flaming may sometimes act as exculpable moral outrage when responding to egregious behaviors. However, contrary to participants’ beliefs, flaming does not reliably or persuasively influence perceptions of those whom it targets; rather, it mostly appears to send negative signals about the flamer.

A multimodal dialogue system for customer service based on user personality adaptation and dialogue strategies

ABSTRACT This paper presents a multimodal dialogue system with user personality adaptation and multiple nonverbal behavior generation for a conversational android robot that plays a customer service role in recommending travel plans. It is useful to estimate the user's personality to realize a dialogue system that adapts the dialogue strategy to individual users. To improve the user's impression of the conversation robot as a customer service agent, we must control the nonverbal behavior of the robot, such as its facial expressions and motions. Moreover, we need to design the dialogue strategy to provide information and make users enjoy the conversation by adding an ice-breaking step. Against this background, we develop a dialogue system prototype that adjusts the dialogue strategy based on the user's personality, as estimated by a pre-trained multimodal sensing model. Furthermore, we implement appropriate nonverbal behavior patterns for each situation, including the voice, motion, and facial expressions of the android robot, to provide it appropriate and polite service agent behaviors. A personality assessment game is introduced into the dialogue to prevent the user from becoming bored. The game plays an ice-breaking role in the dialogue. To let the user engage with the robot, we implement a complement to the user as a dialogue act. We evaluate the proposed multimodal dialogue system for customer service by means of participation in a dialogue robot competition. The fair evaluation results from 26 dialogue users who conversed with multiple systems show that the proposed dialogue system achieved the best score among 13 teams participating in the preliminary round of the competition, indicating that the user personality adaptation and various elements implemented in our dialogue system improve the dialogue experience of general users. The results of a third-party evaluation and multiple regression analysis show that natural response is the most important factor contributing to user impression.

Construction and practice of the teaching evaluation system for the undergraduate major of New Media Technology

This study explores the construction and practical implementation of a teaching evaluation system tailored for the undergraduate major of New Media Technology. Rooted in literature advocating for formative assessments and critical thinking skills, the research endeavors to fill this void with a bespoke 360-degree comprehensive evaluation mechanism. Drawing on principles of objectivity, comprehensiveness, adaptability, and developmental focus, the study proposes a system encompassing self-evaluation, peer evaluation, student evaluation, and third-party evaluation. Beyond its immediate application in New Media Technology, the study emphasizes the broader implications of its findings for emerging interdisciplinary majors, aiming to contribute to national strategies for scientific and technological innovation. The study unfolds in the context of the New Media Technology major at Qilu University of Technology, offering specific insights into curriculum features, talent cultivation objectives, and industry requisites. Following a detailed exploration of evaluation principles, content, and methods, the study presents the results of the newly implemented system, revealing nuanced strengths and areas for improvement. These findings culminate in a set of recommendations aimed at continuous system review, enhanced collaborative engagement, increased practical application in instructional design, strategic resource allocation, industry integration, and comprehensive student support services. The conclusion reflects on the study's contribution to refining teaching evaluation systems, fostering ongoing improvements, and nurturing the next generation of talents in the ever-evolving landscape of new media technology.

The centrality of reasoning in moral judgments: First- and third-party evaluations of cheating

The centrality of reasoning in moral judgments: First- and third-party evaluations of cheating

The Smart Trust framework for WBAN: An AI-driven approach for node trust assessment

The primary contribution of this research lies in its innovative use of artificial intelligence to automate the trust assessment process in WBANs, providing a dynamic solution to the challenge of maintaining data integrity and network reliability. The SmartTrust (SmTr) framework uses advanced machine learning techniques to accurately analyze historical and behavioral data of network nodes. Thus, computer trustworthiness scores allow one to effectively distinguish between trustworthy nodes and potentially malicious nodes. WBANs and their services are rapidly gaining popularity, but they pose unprecedented security challenges. These requirements are being met with WBAN as it evolves. In an increasingly complex, heterogeneous, and evolving mobile environment, completing these tasks can be difficult. A more secure and adaptable WBAN environment can be achieved by using trust management to meet WBAN security requirements. The reliability of a wireless sensor network is evaluated through behavioral evidence. Researchers use the results of node behavior almost directly or combine them with the results of third-party evaluation, instead of studying the original evidence of node behavior and ignoring the analysis of the history of node behavior, which leads to low confidence, rationality, and reliability. SmartTrust (SmTr) is a new approach based on artificial intelligence (AI) to improve trust and reliability over wireless body area networks (WBAN). As a modern healthcare system, this technology can be considered. Experimental results from implementing the SmTr framework demonstrate its effectiveness in improving network resilience against security threats, improving resource allocation, and thus increasing the quality and reliability of healthcare delivery.

노인장기요양서비스 평가체계의 변화에 관한 연구

The evaluation system for South Korea's long-term care services for the elderly was reorganized with a focus on services quality management through the long-term care facility evaluation system following quasi-marketizing reform in the late 2000s. However, unlike the old evaluation system, service quality is managed through the evaluation of providers who voluntarily entered the market, and this provider evaluation is conducted in connection with the benefit provision process by the insurer (National Health Insurance Services) as a characteristic of South Korea. With regard to the evaluation system, there is dissatisfaction with provider evaluation, and it is argued that a separate independent dedicated evaluation organization should be established together with the strengthening of the role of local governments.
 Therefore, taking notice of cases in Japan, which is introducing an evaluation system that is different from the one in South Korea, this study is intended to examine how the evaluation system for long-term care services for the elderly in Japan has been reorganized and changed in comparison with South Korea and find implications for South Korea. To that end, this study will analyze how the institutional devices for various evaluations such as the third-party evaluation system, the policy evaluation system, and the evaluation system for long-term care insurance business plans for the elderly have been reorganized to be pluralistic and multilayered in the institutional reform following quasi-marketization reform in Japan. Second, it will derive the fact that objective-based policy evaluation (=performance management) and participatory evaluation are being strengthened amid changes in policies regarding the evaluation system for long-term care services for the elderly in Japan. Third, it will capture the governance structure of the evaluation system by revealing the intergovernmental and inter-institutional linkage structure of Japan's evaluation system. Fourth, by identifying the functions carried out by the evaluation system through cases in Japan, this study will present policy implications that will enable the problems of South Korea's evaluation system to be overcome.

An evolutionary game for the behavior of third-party evaluators in pension public–private partnership incorporating public participation

This study analyzes the impact of public participation on the choice of third-party evaluators' behavior strategies during the service quality supervision process of China's pension public–private partnership project. An evolutionary game model between third-party evaluators and government regulators is developed, wherein the evolution rule of the two sides and public participation’s influence on their behavior under the two different conditions are analyzed, and a numerical simulation is carried out using MATLAB 2016a. It is found that third-party evaluators may choose the false evaluation strategy without public participation because of the inducement of rent-seeking or insufficient government punishment when the regulatory revenue of the government regulatory agencies is less than the regulatory cost. In contrast, in the case of public participation, the true evaluation strategy is chosen with an improvement in the level of public participation or an increase in reputation incentive. This suggests the construction and improvement of a third-party evaluation system, which shows that the construction of the service quality supervision system in China’s pension PPP project has a large operating space.

Beyond the usual suspects: using cross-sectoral partnerships to target and engage new citizen scientists

Citizen science offers enormous benefits to enhance public knowledge and understanding of science. Several opportunities to engage and share information with citizens are possible in citizen science projects. Recent evidence demonstrates however that individuals who participate in citizen science projects are demographically speaking not very diverse. For citizen science projects to successfully achieve their full potential in increasing public awareness and understanding of science, a wider social demographic needs to be engaged. We present a nationwide approach developed to achieve just that with respect to targeting and engaging residents in Denmark that did not previously have a prior connection to or interest in nature. Under the auspices of a campaign entitled Our Nature, our approach included the formation of a new, cross-sectoral partnership, and co-creating and implementing of a wide array of communication tactics and nature-based activities, including the development of a new citizen science project. Our cross-sectoral partnership allowed us to broaden the sectors of society that could be reached and develop cross-disciplinary activities to achieve goals for broad engagement. Extensive third-party evaluation revealed that 70% of the Danes interviewed across the country heard about Our Nature, and 70% of these gained more knowledge about Danish nature through the campaign. In addition to presenting our co-created projects and activities by working cross-sectorally and interdisciplinarily, we discuss the successes, challenges and limitations related to reaching our goal, based on evaluation results and our own experiences in citizen science and science communication. The citizen science project Denmark Explores that emerged from this campaign is used as a case study to demonstrate how our approach facilitated the broad engagement of citizens across the country––beyond the usual nature enthusiasts.

An updated tutorial on reproducible PyPI applications for advancing chemometrics and boosting learner motivation

ContextProgramming, particularly with the widely-used scripting language Python, is a powerful tool for expressing ideas in science and technology. PyPI, or the Python Package Index, is a management tool for Python packages. Despite the availability of many tutorials on PyPI, including on its official website, many users have encountered difficulties due to outdated information and issues with the twine upload library. ObjectivesThis paper demonstrates the use of a gas chromatography program to prototype the PyPI application, with the goal of maximizing its dissemination and validating its reproducibility in science and engineering worldwide. MethodsThe peer-reviewed Python program on gas chromatography is converted to the PyPI application. Third-party evaluations motivate learners to boost programming skills. The more the downloads from third-party evaluations, the greater the incentive for learners. ResultsThe gas chromatography program is successfully converted to agci PyPI application. Learners can be excited to check their progress on external downloads to motivate them to learn and program. This tutorial shows how to verify software reproducibility of the PyPI application via Code Ocean. ConclusionThis paper demonstrated the real gas chromatography program to the PyPI application and showed how to verify software reproducibility for maximizing dissemination skills and boosting the motivation of learners for advancing chemometrics research.

The Influence of Hotel Brand Image in Social Media Marketing on Consumer Purchase Behavior

In the post-epidemic era, many hotels want to win a high market share. And with the development of the times and the enhancement of consumers' brand awareness, the power of hotel brands is becoming more and more important. There are many ways for hotels to enhance their brand image. In addition to traditional ways, social media is currently a popular way. At present, more and more hotels are opening all over the country, and various brands emerge endlessly. In the frenziedly competitive hotel market, exploring the purchasing behavior of consumers and seeing how consumers choose is basically the brand image establishment process of a hotel. Through literature review, this article identified two attitudinal variables that have an important impact on consumer purchasing behavior, namely social media marketing and hotel brand image. Through understanding relevant cases and relevant data analysis, the study found that social media marketing is an important window for hotel brand display, and brand-related advertisements and third-party evaluations will affect consumers' purchasing behavior.

The Hedges Have Eyes: Third-party Evaluation of Social Networking in Organizations

The Hedges Have Eyes: Third-party Evaluation of Social Networking in Organizations