Transparent Evaluation Research Articles

Sustainability Competencies in Higher Education Research: An Analysis of Doctoral Theses in Portugal.

Educational research has been highlighting the importance of defining key competencies and learning outcomes related to education for sustainability as a reference for the transparent evaluation of students’ learning in this domain. Drawing on a reference framework that identifies five key competencies in sustainability (i.e., systems-thinking competency, anticipatory competency, normative competency, strategic competency, and interpersonal competency), the study reported in this paper aims to understand whether, how, and to what extent these competencies are present in doctoral theses in Higher Education published in Portugal in the past ten years. To address this objective, a qualitative study framed in an interpretative paradigm was conducted, and a literature review was used as a preferential research method to elicit meaning, gain understanding, and develop empirical knowledge. The retrieved documents were treated using deductive content analysis, which was performed using WebQDA software. Results of the analysis show that the competencies considered in the reference framework are present in research on education for sustainability carried out in recent years in Portugal, with a greater emphasis on strategic and anticipatory competencies. Findings suggest that it is important to continue to conduct research on these competencies to successfully integrate them into educational curricula and teacher education programs.

Estimating cetacean population trends from static acoustic monitoring data using Paired Year Ratio Assessment (PYRA).

The cetacean conservationist is often faced with evaluating population trends from abundance data that are either sparse or recorded at different times in different years. The presence of diel or seasonal patterns in the data together with unplanned gaps is often problematic. Such data are typical of those obtained from static acoustic monitoring. We present a simple and transparent non-parametric trend evaluation method, ‘Paired Year Ratio Assessment (PYRA)’ that uses only whole days of data wherever they are present in each of successive pairs of periods of 365 days. We provide a quantitative comparison of the performance of PYRA with traditional generalised additive models (GAMS) and nonparametric randomisation tests that require a greater level of skill and experience for both application and interpretation. We conclude that PYRA is a powerful tool, particularly in the context of identifying population trends which is often the main aim of conservation-targeted acoustic monitoring.

A Platform (Authorships.org) for the Objective Qualification and Order of Academic Authorship in Medical and Science Journals: Development and Evaluation Study Using the Design Science Research Methodology.

BackgroundThe qualification and order of authorship in scientific manuscripts are the main disputes in collaborative research work.ObjectiveThe aim of this project was to develop an open-access web-based platform for objective decision-making of authorship qualification and order in medical and science journals.MethodsThe design science process methodology was used to develop suitable software for authorship qualification and order. The first part of the software was designed to differentiate between qualification for authorship versus acknowledgment, using items of the recommendations of the International Committee of Medical Journal Editors. The second part addressed the order of authorship, using the analytical hierarchy process for objective multiple criteria decision-making and ranking. The platform was evaluated qualitatively (n=30) and quantitatively (n=18) using a dedicated questionnaire, by an international panel of medical and biomedical professionals and research collaborators worldwide.ResultsAuthorships.org represents an open-access software compatible with all major platforms and web browsers. Software usability and output were evaluated and presented for 3 existing clinical and biomedical research studies. All 18 international evaluators felt that the Authorships.org platform was easy to use or remained neutral. Moreover, 59% (n=10) were satisfied with the software output results while the rest were unsure, 59% (n=10) would definitely use it for future projects while 41% (n=7) would consider it, 94% (n=16) felt it may prove useful to eliminate disputes regarding authorship, 82% (n=14) felt that it should become mandatory for manuscript submission to journals, and 53% (n=9) raised concerns regarding the potential unethical use of the software as a tool.ConclusionsAuthorships.org allows transparent evaluation of authorship qualification and order in academic medical and science journals. Objectified proof of authorship contributions may become mandatory during manuscript submission in high-quality academic journals.

Aspects of research and evaluation of information transparency in supply chains

The article is devoted to the consideration of the need and multidimensional information transparency in supply chains as an element connecting cargo, means of working with cargo, transport and logistics infrastructure. Cost, logistics, statistical and process approaches to understanding information transparency in supply chains are presented. As part of the cost approach, information transparency is understood as the result of the investment activities of companies, therefore, examples of companies investing in the development of appropriate digital solutions are presented. The logistics approach determines the importance of information transparency, which consists in creating advantages in the supply chain, providing specific (depending on the type of cargo transported) results, as well as receiving answers to questions at each stage in the supply chain. The statistical approach formalizes (using the calculation methodology) the idea of ​ ​ information transparency of processes in the supply chain. The process approach allows you to understand how information transparency is created in the supply chain.

AccTEF: A Transparency and Accountability Evaluation Framework for Ontology-Based Systems

This paper proposes a new accountability and transparency evaluation framework (AccTEF) for ontology-based systems (OSysts). AccTEF is based on an analysis of the relation between a set of widely accepted data governance principles, i.e. findable, accessible, interoperable, reusable (FAIR) and accountability and transparency concepts. The evaluation of accountability and transparency of input ontologies and vocabularies of OSysts are addressed by analyzing the relation between vocabulary and ontology quality evaluation metrics, FAIR and accountability and transparency concepts. An ontology-based knowledge extraction pipeline is used as a use case in this study. Discovering the relation between FAIR and accountability and transparency helps in identifying and mitigating risks associated with deploying OSysts. This also allows providing design guidelines that help accountability and transparency to be embedded in OSysts. We found that FAIR can be used as a transparency indicator. We also found that the studied vocabulary and ontology quality evaluation metrics do not cover FAIR, accountability and transparency. Accordingly, we suggest these concepts should be considered as vocabulary and ontology quality evaluation aspects. To the best of our knowledge, it is the first time that the relation between FAIR and accountability and transparency concepts has been found and used for evaluation.

Sensitivity analysis in economic evaluation of payments for water and carbon ecosystem services

Payments for ecosystem services (PES) have become a widely accepted tool for mitigating ecosystem decline with hundreds of applications globally. Despite the extensive use of PES, challenges remain in evaluation of feasibility and impact. A key challenge relates to how the multiple, complex ecosystem and market processes determining outcomes are usually only understood with considerable uncertainty. These uncertainties arise throughout the processes to PES evaluation from the conceptual framing and modelling paradigm through to the input data and scenario assumptions. Understanding implications and risks arising requires rigorous and transparent uncertainty treatment and reporting. This article demonstrates best practice in transparent and rigorous PES evaluation with global sensitivity analysis (GSA). The approach systematically identifies uncertain parameters and applies Monte Carlo simulations with stochastic sampling from distributions of uncertain input parameters that are transformed to provide distributions of possible outcomes. The case study considered involves comparing the costs of forest regeneration with the stacked water and carbon value of ecosystem services derived from the vegetation. The results demonstrate the evaluation of multiple ecosystem service process and market uncertainties, quantifies their influence on the overall economic viability of the scheme and the relative impact of different uncertainties on scheme economic viability risks. The results and discussion outline how wider application of GSA could provide better pragmatic and informative assessment of the inevitable uncertainties within ecosystem service payment scheme evaluation.

A NEW OPEN BUDGET SYSTEM IN UZBEKISTAN TO ENSURE TRANSPARENCY IN THE BUDGET SYSTEM

The introduction of fiscal policy transparency practices in the contemporary local financial management system is one of the prerequisites for increasing the socio-economic development of regions. Creating a transparent budget environment contributes to a faster and more effective solution to the region’s current problems. The lack of a sufficient level of budgetary transparency is a consequence of poor public interest consideration, which impedes the realization of their own democratic aspirations and freedoms. This article is aimed to look through the importance of budget transparency and its good effects on the budgetary system. The public finance transparency assessment of Uzbekistan has shown that its population needs a higher level of information disclosure, use of the latest information channels, and the creation of convenient forms of involvement in the budget process. The transparency evaluation results in the areas of budgeting, public procurement, disposal of property of the territorial community, etc. have been investigated. Besides, directions for improving Uzbekistan’s local authorities information policy are proposed to increase its transparency. KEYWORDS: Budget system, budget transparency, local budget, local budget indicators, fiscal policy, budgetary system, open budget system.

Examining response engagement in online interactions between US government agencies and citizens

PurposeEmail communication is indispensable for US state agencies to respond to citizen requests and engage with constituents, contributing to building trust in local governments. While prior studies examine the responsiveness of elected officials, the quality of virtual interactions between government organizations and citizens is often overlooked. This study aims to investigate how US government agencies capitalize on the potential of online interactions with constituents to manage generic queries and introduce the response engagement index (REI) consisting of response time, reactive transparency and message interactivity to evaluate levels of communicative engagement.Design/methodology/approachThe authors conducted a field experiment encompassing emailing a request to 547 state agencies based in the five largest states and one small state. A total of 377 organizational responses were manually analyzed to reveal the usages of six communicative engagement strategies.FindingsThe results show the potential of online communication is underutilized as the average score of response engagement remains low. Human responses are less engaging than auto-reply messages and require a one-day waiting period, if not longer. Response types and gender of government communicators significantly differ in response time and engagement strategies. The findings identify divergent patterns of response engagement and provide practical implications for facilitating citizen engagement.Research limitations/implicationsThis research fills a critical gap by investigating the quality of online interactions between US government agencies and citizens. The authors develop a theory-grounded tool of response engagement to identify three features: response speed, reactive transparency and interactivity. The findings can improve the quality of email communication in state agencies, enhancing governance quality. The REI proposed here addresses what Pfau (2008) deemed problematic for communication scholarship: research is sparse on “functional issues” that examine the communication process. Pfau argued for research that provides knowledge of interest across disciplines so as to “cross-fertilize” ideas between political communication and public relations; this study sought to bridge that gap with a theoretical and practical tool for building public trust in governments.Practical implicationsTo support the evaluation of transparent and responsive governments, reliable and valid measurements are needed. The proposed REI provides practitioners with a theory-grounded tool to identify areas of engagement quality in government responses. The findings can be used to improve the quality of email communication in state agencies, enhancing governance quality.Social implicationsCitizens seek reciprocal dialogue through prompt, open and interactive communication. US state agencies should leverage the engagement features for increasing citizen trust – response time, reactive transparency and interactivity – when responding to public inquiries. Ultimately, trust in government agencies' interests in serving stakeholders cannot be strengthened without prompt and engaging responses to meet the public's needs.Originality/valueThis field experiment was one of the first to focus on US state agencies' responses to information requests. It introduces a new REI to assess communicative engagement in a government/citizen exchange.

Can I Trust This Location Estimate? Reproducibly Benchmarking the Methods of Dynamic Accuracy Estimation of Localization.

Despite the great attention that the research community has paid to the creation of novel indoor positioning methods, a rather limited volume of works has focused on the confidence that Indoor Positioning Systems (IPS) assign to the position estimates that they produce. The concept of estimating, dynamically, the accuracy of the position estimates provided by an IPS has been sporadically studied in the literature of the field. Recently, this concept has started being studied as well in the context of outdoor positioning systems of Internet of Things (IoT) based on Low-Power Wide-Area Networks (LPWANs). What is problematic is that the consistent comparison of the proposed methods is quasi nonexistent: new methods rarely use previous ones as baselines; often, a small number of evaluation metrics are reported while different metrics are reported among different relevant publications, the use of open data is rare, and the publication of open code is absent. In this work, we present an open-source, reproducible benchmarking framework for evaluating and consistently comparing various methods of Dynamic Accuracy Estimation (DAE). This work reviews the relevant literature, presenting in a consistent terminology commonalities and differences and discussing baselines and evaluation metrics. Moreover, it evaluates multiple methods of DAE using open data, open code, and a rich set of relevant evaluation metrics. This is the first work aiming to establish the state of the art of methods of DAE determination in IPS and in LPWAN positioning systems, through an open, transparent, holistic, reproducible, and consistent evaluation of the methods proposed in the relevant literature.

Internal Quality and Job Satisfaction in Health Care Services.

(1) Background: The main purpose of this study was to evaluate the health services’ internal quality level in Greek public hospitals and to investigate whether there is a relation between internal quality and health care professionals’ job satisfaction. (2) Material and Methods: A cross-sectional study was conducted in six public hospitals (four general and two specialized hospitals). The following tools were used to collect data: (a) the SERVQUAL questionnaire, which is designed to measure service quality through five dimensions, and (b) the Job Satisfaction Survey (JSS) questionnaire, which is designed to measure employees’ job satisfaction. Convenience sampling was used as a sampling technique. (3) Results: The level of internal quality service was found to be low as regards the dimensions of: reliability, assurance, responsiveness, and empathy, while the “tangibles” dimension was the only one which was assessed as having a satisfactory internal quality level. Τhe results also revealed a positive correlation between the job satisfaction subscales and the quality dimensions. Regarding demographic characteristics and their effect on employees’ perceptions of internal quality dimensions, the study found that the gender and the educational factor had no effect while younger employees have a more positive perception of the quality of responsiveness dimension in the health organizations where they work. In terms of profession, administrative staff had a lower perception of the quality of tangibles dimension than doctors and nurses. Concerning years of experience, the results indicated that employees with more previous working experience had a worse perception of the quality of reliability, responsiveness, and assurance dimensions. (4) Conclusions: According to the results, establishing a sense of trust and understanding between management and health professionals through effective communication, transparent evaluation, and reward is critical to developing, enhancing, and promoting an internal quality culture in a hospital setting.

Using OpenGovB Transparency Indicator to Evaluate National Open Government Data

Transparency evaluation in open government is a process of measuring the extent of transparency against a predefined set of indicators. In this paper, we address the existing initiatives regarding data and government transparency evaluation as two separate indicators and present the analysis of their advantages and drawbacks. Based on that analysis, we extend a part of the OpenGovB benchmark related to transparency in open government. What is unique about this benchmark is that it utilizes metadata of data published on the open government data portals to calculate the majority of indicators related to data transparency indicators. For the government transparency indicator evaluation, the benchmark utilizes some of the well-known transparency indicators. The article shows concrete results obtained from the application of the defined transparency evaluation model on 22 open data portals, thus demonstrating the possibilities of its application as well as the gains regarding generated results. The proposed model bridges the gap between available methodologies for evaluating transparency based on collaboration and participation and methodologies for evaluating transparency based on open data.

Evaluating the Level of Transparency and its Relationship to Performance Evaluation in a Sample of Iraqi Banks Listed on the Iraq Stock Exchange

The research aims to measure the degree of transparency in disclosure in a sample of Iraqi banks listed in the Iraqi Stock Exchange and to predict the future of these banks by relying on their future expectations and to measure the impact of the degree of transparency of Iraqi banks and its relationship to performance evaluation. The current study gained its importance considering that performance evaluation is the basic process that shows the capabilities of banks to continue their activities with failure or success. Therefore, providing financial or non-financial measures gives the bank a clear image about its products and services and the processes that lead to the production of those products and services, and the need to develop cost accounting and management accounting systems in the bank so that its outputs are not limited to mere financial measures, but extends the scope of those outputs to include the generation of non-financial measures Help to raise the efficiency, effectiveness and economic performance. One of the results of the research is that the Iraqi banks in the research sample vary in the level of their reporting on transparency according to the S&P scale. The highest level of transparency amounted to (52%) for the ethnic commercial bank and the lowest level was (28.75%) for the Bank of Baghdad. The highest degree of financial ratios for performance evaluation amounted to the Commercial Bank of Iraq and the lowest for Elaf Islamic Bank. The research also concluded that there is positive relationship between transparency and performance evaluation. One of the most important recommendations is to awareness of the investors about the importance of information and their right to obtain it, then holding the relevant authorities accountable for any failure in the event of concealing any necessary information.

Transparent soil test evaluation of vertical–horizontal mixed curtain during dewatering

Transparent soil test evaluation of vertical–horizontal mixed curtain during dewatering

Budget transparency as an important condition for strengthening the financial self-sufficiency of territorial communities

The article characterizes theoretical principles of the impact of budget transparency on ensuring the financial self-sufficiency of territorial communities determined by the ability of their authorities to finance in full the spending powers for solving social and cultural needs of the daily life of both individual residents and the community in general at the expense of the financial resources formed in local budgets. The content of a number of interrelated key characteristics (transparency, openness, timeliness, accountability, credibility) and types of budget transparency (political, economic, procedural, public, regulatory, institutional) is highlighted and revealed, taking into account the key areas of their manifestation. Attention is focused on the main problems of budgetary transparency of territorial communities: a decrease in trust and respect for the authorities on the part of citizens, a lack of open dialogue with representatives of the authorities, and a low level of readiness of society to participate in the decision-making process for solving local problems. The article justifies that ensuring the accountability of authorities and effective involvement of the community in the budget process requires the development and implementation of an effective communication strategy at the level of territorial communities, the creation of expert platforms, the spread of the practice of forming local budgets on a participatory basis, the development and launch of complex public finance management systems, and comprehensive evaluation of budget transparency, which would cover all local budgets. Particular attention is paid to highlighting the forms of citizen involvement in managing the finances of territorial communities in terms of the stages of the budget cycle and the possible effects of such participation. The article offers the following directions for strengthening the financial self-sufficiency of the budgets of territorial communities based on the increase in the level of budget transparency - increasing the loyalty and trust of economic agents in the activities of subnational governments, activating the financing of investment projects of local importance due to the increase in participation in the process of planning, formation, and use of local budget baskets, and optimization of the budgetary resources spending due to the growing interest of members of the territorial community in solving issues of local importance.

Design of Teaching Material Evaluation Incentive Mechanism based on Game Theory

As the carrier of knowledge and the basis of teaching, teaching materials affect the teaching quality of schools. However, the selection of teaching materials in colleges and universities is in a dilemma because of the variety of teaching materials and the uneven quality of teaching materials. Blockchain has gradually drawn attention from various fields, due to its secure storage, non-forgery and non-tamper features. The combination of blockchain and teaching material evaluation can provide fair and transparent grading and evaluation for teaching materials, which is of great reference value for colleges and universities to select teaching materials. This paper proposes a reputation incentive mechanism for teaching material evaluation blockchain based on game theory, analyzes the behavior of users in the scoring stage of teaching material evaluation, and gives the Nash equilibrium solution. Through the numerical simulation experiment, the effectiveness of the incentive mechanism and the reliability of the teaching materials’ evaluation results are proved.

Determinanty (nie)przejrzystości finansów samorządowych

Purpose – The aim of the article is to identify the determinants of transparency of local government finance and to indicate which of them positively and negatively affect its level, with particular emphasis on the situation in Poland. Research method – The article is theoretical in nature, based on a review of the literature on the subject in economics and finance, organisation and management, and sociology. The method of descriptive analysis and comparative analysis was used. Results – The role of local government reporting as a source of useful information is restricted. The flow of information about the condition of self-government finance and communal assets between the self-government authorities and the inhabitants is limited. Frequent changes in the legal regulations of self-government finance increase its opacity. Centralisation of public finance limits the financial independence of the local government and negatively affects the transparency of its finance. Originality/value/implications/recommendations – Studies on the evaluation of transparency of local government finance in Poland are rarely undertaken. Attention was paid to the supply-side and demand-side approaches to transparency in local government finance. Determinants of financial transparency of local government units were identified in their further environment (macro-environment) in the socio-cultural, political-legal, economic dimension, and in their closer environment (micro-environment) in the institutional dimension.

DeviceD: Audience-dancer interaction via social media posts and wearable for haptic feedback.

The performative installation DeviceD utilizes a network of systems toward facilitating interaction between dancer, digital media, and audience. Central to the work is a wearable haptic feedback system able to wirelessly deliver vibrotactile stimuli, with the latter initiated by the audience through posting on Twitter social media platform; the system in use searches for specific mentions, hashtags, and keywords, with positive results causing the system to trigger patterns of haptic biofeedback across the wearable's four actuator motors. The system acts as the intermediator between the audience's online actions and the dancer receiving physical stimuli; the dancer interprets these biofeedback signals according to Laban's Effort movement qualities, with the interpretation informing different states of habitual and conscious choreographic performance. In this article, the authors reflect on their collaborative process while developing DeviceD alongside a multidisciplinary team of technologists, detailing their experience of refining the technology and methodology behind the work while presenting it in three different settings. A literature review is used to situate the work among contemporary research on interaction over internet and haptics in performance practice; haptic feedback devices have been widely used within artistic work for the past 25years, with more recent practice and research outputs suggesting an increased interest for haptics in the field of dance research. The authors detail both technological and performative elements making up the work, and provide a transparent evaluation of the system, as means of providing a foundation for further research on wearable haptic devices.

Quality Evaluation of Official Development Assistance Provided to the Republic of Moldova

The purpose of this paper is not to assess the role or performance of international financial institutions in the Republic of Moldova, but rather to focus on a somehow insufficiently investigated topic and namely: quality evaluation of official development assistance provided by donors. Several factors were considered when determining the efficiency of official development assistance: consolidation of units within the Ministry of Finance, update of the electronic platform for foreign aid management, transparent evaluation of donors’ policies from the perspective of the two critical lines of research in this field: effectiveness and coherence. At the same time, special emphasis was placed on conceptualizing and researching the political conditionality imposed by some of the donors for reflecting local changes and if development partners achieve the objective of having efficient and qualitative official development and technical assistance based on the dynamics of political conditionality use. As a result of the conducted research, it is established that high political volatility and weak institutional capacity have hampered the efforts to implement the reforms supported by development partners, thus it is necessary to further increase the capacity of the program portfolio, adjust foreign assistance in the sociopolitical, institutional context and from the perspective of the current economic crisis.

Strategy Analysis for project portfolio evaluation in a technology consulting company by the hybrid method THOR

Regarding the scope of project portfolio management, the correct prioritization of actions leads to the achievement of the organizational strategic objectives. Based on the strategic planning of a technology consulting company, the paper presents an alternative model for project prioritization from the moment that the project selection decision is a vital process in technological associations in two ways. First, the software development project budget requires a large investment and must be in alignment with the strategic objectives of the company. Second, the multidimensionality of organizational returns on projects is naturally risky in terms of the expected outcome. This study considers the project analysis based on a multicriteria approach by the THOR method, providing to make a transparent evaluation, highlighting the most important variables concerning the problem, and making the identification of the most favorable solution.

Dynamics of cumulative advantage and threats to equity in open science: a scoping review.

Open Science holds the promise to make scientific endeavours more inclusive, participatory, understandable, accessible and re-usable for large audiences. However, making processes open will not per se drive wide reuse or participation unless also accompanied by the capacity (in terms of knowledge, skills, financial resources, technological readiness and motivation) to do so. These capacities vary considerably across regions, institutions and demographics. Those advantaged by such factors will remain potentially privileged, putting Open Science's agenda of inclusivity at risk of propagating conditions of ‘cumulative advantage’. With this paper, we systematically scope existing research addressing the question: ‘What evidence and discourse exists in the literature about the ways in which dynamics and structures of inequality could persist or be exacerbated in the transition to Open Science, across disciplines, regions and demographics?’ Aiming to synthesize findings, identify gaps in the literature and inform future research and policy, our results identify threats to equity associated with all aspects of Open Science, including Open Access, Open and FAIR Data, Open Methods, Open Evaluation, Citizen Science, as well as its interfaces with society, industry and policy. Key threats include: stratifications of publishing due to the exclusionary nature of the author-pays model of Open Access; potential widening of the digital divide due to the infrastructure-dependent, highly situated nature of open data practices; risks of diminishing qualitative methodologies as ‘reproducibility’ becomes synonymous with quality; new risks of bias and exclusion in means of transparent evaluation; and crucial asymmetries in the Open Science relationships with industry and the public, which privileges the former and fails to fully include the latter.