Holistic Assessment Framework Research Articles

ATLANTIS: A multidimensional maturity model for evaluating the adoption of the internet of things in industrial SMEs

The Internet of Things presents significant opportunities to small and medium-sized companies in the industrial sector. However, adopting and implementing the Internet of Things effectively remains a challenge because it depends not only on technical capabilities but also on organizational readiness and workforce adaptation, especially in the industrial landscape. This study presents ATLANTIS, a novel maturity model specifically designed to assess the adoption and implementation of the Internet of Things in small and medium-sized industrial enterprises. Addressing a critical gap identified in the literature, ATLANTIS adopts a holistic assessment framework incorporating organizational, human, and technological dimensions based on an established taxonomy. The model consists of seven key components and detailed rubrics, allowing for a granular assessment of an organization’s progress in technology adoption. Validation was carried out using the Delphi method, with the participation of 44 experts from different countries. The predominantly positive feedback regarding satisfaction, completeness, and applicability suggests that ATLANTIS effectively captures the relevant dimensions and challenges associated with adopting the Internet of Things in this organizational context. This research provides a practical, robust, and validated tool for assessing Internet of Things maturity within small and medium-sized enterprises in the industrial sector. It is a valuable resource for organizations to understand their IoT adoption level and identify areas for improvement. Future work can focus on the longitudinal application of ATLANTIS for tracking maturity progression over time, comparing different countries, and exploring the correlation between Internet of Things maturity and organizational performance outcomes.

The Impact of Key Performance Indicators on Faculty Research in Higher Education

This study investigates the effectiveness of key performance indicators (KPIs) in evaluating the research contributions of academic staff in Uzbekistan, aiming to provide a nuanced understanding of faculty performance assessment. Employing a mixed-methods approach, the research integrates quantitative bibliometric analysis of publication data from 500 faculty members with qualitative interviews from 30 academics. The quantitative analysis focuses on metrics such as total publications, citation counts, and h-index scores, while the qualitative component explores faculty perceptions regarding the influence of KPIs on their research activities and career development. The analysis reveals significant variability in research productivity, with faculty averaging 12.4 publications and 110.2 citations. Econometric modeling indicates a strong positive correlation between publication metrics and citation impact, suggesting that higher publication output enhances academic visibility. Qualitative insights highlight the pressures of a "publish-or-perish" culture, which adversely affects research quality and faculty morale, particularly in contexts with limited institutional support. This research contributes to the discourse on academic performance evaluation by emphasizing the limitations of traditional metrics and advocating for a more holistic assessment framework. It highlights the need for higher education institutions to consider both quantitative and qualitative measures to better reflect the complexities of academic contributions in Uzbekistan.

Developing a pragmatic consensus procedure supporting the ICH S1B(R1) weight of evidence carcinogenicity assessment.

The ICH S1B carcinogenicity global testing guideline has been recently revised with a novel addendum that describes a comprehensive integrated Weight of Evidence (WoE) approach to determine the need for a 2-year rat carcinogenicity study. In the present work, experts from different organizations have joined efforts to standardize as much as possible a procedural framework for the integration of evidence associated with the different ICH S1B(R1) WoE criteria. The framework uses a pragmatic consensus procedure for carcinogenicity hazard assessment to facilitate transparent, consistent, and documented decision-making and it discusses best-practices both for the organization of studies and presentation of data in a format suitable for regulatory review. First, it is acknowledged that the six WoE factors described in the addendum form an integrated network of evidence within a holistic assessment framework that is used synergistically to analyze and explain safety signals. Second, the proposed standardized procedure builds upon different considerations related to the primary sources of evidence, mechanistic analysis, alternative methodologies and novel investigative approaches, metabolites, and reliability of the data and other acquired information. Each of the six WoE factors is described highlighting how they can contribute evidence for the overall WoE assessment. A suggested reporting format to summarize the cross-integration of evidence from the different WoE factors is also presented. This work also notes that even if a 2-year rat study is ultimately required, creating a WoE assessment is valuable in understanding the specific factors and levels of human carcinogenic risk better than have been identified previously with the 2-year rat bioassay alone.

A conceptual framework for holistic assessment of decision support systems for sustainable livestock farming

The livestock sector has complex relationships with the three fundamental pillars of sustainability, i.e., environmental, economic, and social. Devising a livestock farming strategy by considering the different sustainability pillars is essential. Although several decision support systems (DSSs) are available for the livestock sector, these DSSs differ in the way they address sustainability. This work emphasizes the importance of a holistic approach to sustainable livestock management rather than only targeting individual sustainability dimensions. We, therefore, propose an initial assessment framework to evaluate the capacity of livestock DSSs in targeting the different sustainability pillars. In line with this, we present a conceptual basis for deriving assessment criteria and indicators. We then use the proposed assessment framework to assess existing openly available livestock DSSs. We observe that the main focus of the existing and openly available livestock-related DSSs is on the indicators from environmental pillars, and only a few of them accommodate economic aspects. No openly available DSS includes social and governance-related points. More importantly, none of these DSSs can handle data streams from Internet of Things (IoT) devices and, hence, they miss on the superiority that advanced modelling techniques can provide. With these observations, we draft an extensive set of guidelines for future livestock-related DSSs to holistically target sustainability.

Condition-Based Maintenance in Aviation: Challenges and Opportunities

Condition-Based Maintenance (CBM) is a policy that uses information about the health condition of systems and structures to identify optimal maintenance interventions over time, increasing the efficiency of maintenance operations. Despite CBM being a well-established concept in academic research, the practical uptake in aviation needs to catch up to expectations. This research aims to identify challenges, limitations, solution directions, and policy implications related to adopting CBM in aviation. We use a generalizable and holistic assessment framework to achieve this aim, following a process-oriented view of CBM development as an aircraft lifecycle management policy. Based on various inputs from industry and academia, we identified several major sets of challenges and suggested three primary solution categories. These address data quantity and quality, CBM implementation, and the integration of CBM with future technologies, highlighting future research and practice directions.

Index-based tools for livelihood security and resilience assessment (LiSeRA) in lower Mekong Basin

This paper presents a framework and toolkit for assessment of multi-hazard livelihood security and resilience in the Lower Mekong Basin (LMB) communities. The LMB is a subsidiary region of the Mekong region in South East Asia, and is frequently exposed to hydrometeorological hazards and anthropogenic stressors that expose and directly affect the livelihoods of more than sixty-five million people living in the region. The main purpose of the study is to support decision-making and risk management planning through integration of the concepts of livelihood security and resilience into a holistic framework, and subsequently developing an index-based toolkit for conducting assessments. Firstly, dimensions, sub-dimensions and indicators for measurement of livelihood security and resilience in the LMB were identified through comprehensive literature review and expert consultation. Then, several local workshops were conducted with various stakeholders (researchers, government officials, community people) in the LMB region to validate the indicators and generate weightages. The indicators were then arranged in a matriculated form, and the weightages were used to generate the algorithm for computing the quantitative outputs of livelihood security and resilience in study area. An Excel toolkit and a ‘R’ programming package were developed using the algorithm for visualization of the assessment outcomes. The proposed framework and toolkit are expected to assist researchers, government officials and development professionals in generating robust resilience assessment indices for risk informed decision-making and planning.Brief outline of the method•Livelihood security and resilience concepts were integrated to generate a holistic assessment framework and an indicator library.•Weightages for indicators were generated using the Analytical Hierarchical Process (AHP) through consultation with relevant stakeholders.•The indicator library was developed into an algorithm-based Excel and ‘R’ programming toolkit that provides quantitative assessment outputs.

Holistic Assessment Framework for General Education Program Review

ABSTRACT An assessment framework was developed and implemented for UCLA’s science general education (GE) program that combined overall program goals, individual course improvement, and faculty engagement. Specifically, the framework targeted (a) developing institutional goals for GE courses, (b) ensuring alignment of individual course learning outcomes with institutional goals, (c) exploring the student experience during GE courses, and (d) engaging faculty to understand the classroom experience. Using the methods detailed, information was gathered regarding student learning experiences, scientific skill/literacy level, and attitudes toward science after completing the GE course. Furthermore, this process also facilitated faculty involvement in the assessment and an improved awareness about course pedagogy. This framework can be applied to assess other general education programs (diversity, writing, etc.) or even individual departmental science programs.

Titan: Security Analysis of Large-Scale Hardware Obfuscation Using Graph Neural Networks

Hardware obfuscation is a prominent design-for-trust solution that thwarts intellectual property (IP) piracy and reverse-engineering of integrated circuits (ICs). Researchers have proposed several large-scale obfuscation techniques that achieve high output corruption—thus offering resilience against seminal attacks along with acceptable power, performance, and area overheads. However, the research community has primarily evaluated hardware obfuscation on relatively small scales of obfuscation (i.e., a fixed number of obfuscated components). Moreover, prior art caters toward specific schemes based either on gate obfuscation or interconnect obfuscation, i.e., two prominent types of hardware obfuscation. The former shortcoming suggests focusing on large-scale obfuscation schemes, and the latter suggests the need for a holistic assessment framework. In this work, we propose Titan, a holistic framework considering large-scale gate and interconnect obfuscation schemes. More specifically, we propose a graph neural network (GNN)-based attack framework that is trained to exploit structural and functional properties of any secured circuit to recover its obfuscated components. We evaluate Titan on various obfuscation schemes, considering selected ITC-99 benchmarks with up to 50% obfuscation scale, i.e., up to 21,326 obfuscated components. We observe a substantial information leakage through structural and functional properties of secured designs even for large-scale obfuscation. We quantify the information leakage in two ways: first, an average reduction of Hamming distance (HD, a well-established metric for attack evaluation) by 23.27 and 16.19 percentage points over the baseline of random guessing for gate and interconnect obfuscation, respectively; second, an average recovery of 63.40% and 77.94% of obfuscated components for gate and interconnect obfuscation, respectively. Importantly, these results are superior to six state-of-the-art attacks. We will open-source our framework and associated artifacts to enable reproducibility and foster future work.

Assessment of Wayfinding Performance in Complex Healthcare Facilities: A Conceptual Framework

Wayfinding is considered to be one of the most demanding challenges to be performed by hospitals’ users. Wayfinding has been an interest among researchers from different fields, such as architecture, interior design, cognitive psychology, and facilities management, to name a few. Previous scholars have highlighted the need for a holistic framework taking into consideration both user and environmental factors. A narrative review of the literature was carried out to understand the full extent of the issue and address the ever-increasing demand for a holistic assessment framework. This article attempts to address the underlying gap by proposing a comprehensive framework that takes into account both facets of the issue through a narrative review of the literature to some of the most prominent research attempts to address the problem of wayfinding in complex healthcare settings. Furthermore, the proposed framework can assist both researchers and practicing professionals by providing a comprehensive understanding of the issue of complex wayfinding as well as of the variables to be investigated in the assessment process.

Potential of harnessing operational flexibility from public transport hubs to improve reliability and economic performance of urban multi-energy systems: A holistic assessment framework

Growing penetration of renewable energy sources (RES) and emerging electrified loads (EEL) are bringing about increased difficulties for the power balancing and efficient operation of energy system, due to the impact of remarkable volatilities introduced. Public transport hub (PTH), as a new-style infrastructure of traffic service carriers, is regarded to offer a cogent solution to this problem, in terms of their potential operational flexibilities permitted by energy regulation, vehicular dispatch, and vehicle-to-grid (V2G) programs. As such, this paper carries out a comprehensive study to investigate the implication of harnessing PTH-enabled flexibility in a context of urban multi-energy system (UMES). The proposed methodology is established on a holistic reliability/economic analysis framework which is designed to indicate how UMES’s performances would vary with different utilization of PTH resources. In order to portray the real-time controllability of PTH during operation, a PTH model that takes into account the impacts of both energy- and service-related aspects has been developed, with particular focus on the interdependencies between the energy and transportation sector. The operational simulation of UMES in presence of PTHs is implemented by using a multi-modal-based optimization model, which captures the effects of PTH flexibility under both normal and contingency scenarios integratedly. By embedding the above formulation into a sequential Monte Carlo simulation-based assessment framework, the contribution of PTH to the reliability and economy of UMES can be determined. Numerical studies are conducted based on an illustrative electricity-gas-heat test case and the real PTH datasets in Beijing. The simulation results confirm the significance of PTH-enabled flexibility in improving the performances of UMES. Also, it is demonstrated that the reserving strategy adopted, the composition of vehicle model, and the travel demand profile of passengers are the noteworthy factors that influence the profitability of PTH exploitation.

A Comprehensive Assessment Framework for Evaluating Adaptive Security and Privacy Solutions for IoT e-Health Applications

There exist numerous adaptive security and privacy (S&P) solutions to manage potential threats at runtime. However, there is a lack of a comprehensive assessment framework that can holistically validate their effectiveness. Existing Adaptive S&P assessment efforts either focus on privacy or security in general, or are focused on specific adaptive S&P attributes, e.g. authentication, and, at certain times, disregards the architecture in which they should be comprehended. In this paper, we propose a holistic assessment framework for evaluating adaptive S&P solutions for IoT e-health. The framework utilizes a proposed classification of essential attributes necessary to be recognized, evaluated, and incorporated for the effectiveness of adaptive S&P solutions for the most common IoT architectures, fog-based and cloud/server-based architectures. As opposed to the existing related work, the classification comprehensively covers all the major classes of essential attributes, such as S&P objectives, contextual factors, adaptation action aptitude, and the system’s self-* properties. Using this classification, the framework assists to evaluate the existence of a given attribute with respect to the adaptation process and in the context of the architectural layers. Therefore, it stresses the importance of where an essential attribute should be realized in the adaptation phases and in the architecture for an adaptive S&P solution to be effective. We have also presented a comparison of the proposed assessment framework with existing related frameworks and have shown that it exhibits substantial completeness over the existing works to assess the feasibility of a given adaptive S&P solution.

Incorporating the life stages of fish into habitat assessment frameworks: A case study in the Baihetan Reservoir

Although it is widely accepted that the construction of dams may alter fish habitats, few studies have followed the life cycles of fish and combined the environmental conditions with the ecological behaviors and habit preferences of fish during reproductive processes to assess its effects of dam construction. In this study, we call for more sophisticated and holistic assessment framework, including effectiveness of technologies intended to mitigate environmental impacts in different life stages. An assessment framework that considers the swimming ability, perception ability of water flow and environmental preference of different fish species during migration, spawning and hatching was proposed. We used the Baihetan Reservoir as an example environment to assess the impoundment effect on the habitat of a tributary upstream of the reservoir. We observed shifts in the habitats of target fish in different life stages which is dominated by reservoir operation of the Baihetan Dam. Combined with the response of fish activities to impoundment, the selection of suitable positions for artificial breeding and release projects and the outlet of the fish transportation system were recommended measures to improve the migration possibilities. Our reassessment results also demonstrated the theoretical possibility and feasibility of joint improvements in spawning and hatching periods using instream structures. Our framework provides a complete set of “assessment-solution” processes for developers and managers to address the aquatic ecological degradation caused by resource development, and its use is strongly recommended for assessments or assessments of damming effects in other regions and on other fish species.

Holistic Communication Assessment for Young Children With Cleft Palate Using the International Classification of Functioning, Disability and Health:Children and Youth.

Purpose Children with a cleft palate (± cleft lip; CP±L) can have difficulties communicating and participating in daily life, yet speech-language pathologists typically focus on speech production during routine assessments. The International Classification of Functioning, Disability and Health: Children and Youth Version (ICF-CY; World Health Organization, 2007) provides a framework for holistic assessment. This tutorial describes holistic assessment of children with CP±L illustrated by data collected from a nonclinical sample of seven 2- to 3-year-old children, 13 parents, and 12 significant others (e.g., educators and grandparents). Method Data were collected during visits to participants' homes and early childhood education and care centers. Assessment tools applicable to domains of the ICF-CY were used to collect and analyze data. Child participants' Body Functions including speech, language, and cognitive development were assessed using screening and standardized assessments. Participants' Body Structures were assessed via oral motor examination, case history questionnaires, and observation. Participants' Activities and Participation as well as Environmental and Personal Factors were examined through case history questionnaires, interviews with significant others, parent report measures, and observations. Results Valuable insights can be gained from undertaking holistic speech-language pathology assessments with children with CP±L. Using multiple tools allowed for triangulation of data and privileging different viewpoints, to better understand the children and their contexts. Several children demonstrated speech error patterns outside of what are considered cleft speech characteristics, which underscores the importance of a broader assessment. Conclusion Speech-language pathologists can consider incorporating evaluation of all components and contextual factors of the ICF-CY when assessing and working with young children with CP±L to inform intervention and management practices.

Development of a Holistic Assessment Framework for Industrial Organizations

The evaluation and selection among the best production practices beyond the conventional linear models is, nowadays, concerned with those holistic approaches drawn toward environmental assessment in industry. Therefore, researchers need to develop an analysis that can evaluate the performance of industrial organization in the light of their environmental viewpoint. This study implemented a pilot co-integrated scheme based on an innovative in-house Holistic Assessment Performance Index for Environment (HAPI-E) industry tool while assimilating the principles of circular economy through the Eco-innovation Development and Implementation Tool (EDIT). For the latter, nine qualitative indicators were motivated and enriched the weighting criteria of the questionnaire. The decomposition of the complexity and preferences mapping was accompanied by a multi-criteria holistic hierarchical analysis methodology in order to synthesize a single index upon a need-driven scoring. This multi-criteria decision approach in industry can quantify the material and process flows, thus enhancing the existing knowledge of manipulating internal resources. The key-criteria were based on administrative, energy, water, emissions, and waste strategies. Subsequently, the HAPI-E industry tool was modeled on the food industry, being particularly focused on pasta-based industrial production. Then, the parameters of this tool were modeled, measured, and evaluated in terms of the environmental impact awareness. The magnitude of necessary improvements was unveiled, while future research orientations were discussed. The HAPI-E industry tool can be utilized as a precautionary methodology on sustainable assessment while incorporating multifaceted and quantification advantages.

Entrustable Professional Activities in General Surgery: Development and Implementation

Entrustable Professional Activities are a holistic assessment framework developed in the Netherlands in 2005, which have recently been adopted in undergraduate medical education in the United States. As part of an increased focus on competency-based assessment, the specialty of pediatrics has led the way in incorporating them into graduate medical education. We describe the development and initial pilot process of implementation of EPAs into the assessment of General Surgery trainees.

Dimensional accuracy and structural performance assessment of spatial structure components using 3D laser scanning

Abstract Spatial structures have been constructed around the world, and their components have become increasingly complicated. Quality inspection is primarily manually performed and is a labor intensive approach which is prone to error. Developments in 3D laser scanning offer great opportunities to improve the precision and efficiency of quality control operations. This paper presents a new approach for assessing the dimensional accuracy and structural performance of spatial structure elements using 3D laser scanning. The proposed approach establishes a holistic assessment framework by considering the evaluation parameters, optimized scanning strategy, and data processing. A data processing method is developed to automatically calculate and compare the difference between the reverse model and design model based on point cloud processing, reverse modeling, and finite element analysis. A new algorithm is proposed to automatically extract the point cloud boundaries, and it is capable of streamlining the generation of reverse models. The assessment criteria and error control for the framework are quantitatively described in a systematic manner. The proposed approach is tested and demonstrated through an actual project. The results show that using 3D laser scanning for the quality control of spatial structure elements is suitable and validate that applicability of the proposed approach compared to traditional measurement methods.

A FRAMEWORK FOR INTEGRATED SUSTAINABILITY ASSESSMENT OF WATER CYCLE SERVICES

A methodology and a framework for integrated sustainability assessment of water cycle services to be used for analysis of strategic options in integrated water management has been developed. The hypothesis was that a holistic assessment framework based on local data would enable decisionmaking and sustainable development on a local scale in integrated water management. The SUWAM assessment framework was developed using the water cycle services of a small town in Western Cape, South Africa, as case. The point of departure was the municipality's plan for sustainable development. The suitability of the SUWAM Framework was assessed by comparison with two other frameworks: TRUST, from the EU-project TRUST, and City Blueprint, developed in an EIPwater action group. The comparison demonstrated that a sustainability assessment framework must: (i) be tailored to the local conditions, and (ii) that the perspective of the user, often a decision maker, must be taken into account in the interpretation of results. The methodology used in the development of the SUWAM Framework fulfilled these two criteria and produced results that were recognised as relevant for the studied case by local stakeholders. The methodology is suited for comparison and further analysis of different mitigation and adaptation measures. Keywords: Integrated Water Management, Sustainability Assessment, Water Cycle Services

Assessment for Systems Learning: A Holistic Assessment Framework to Support Decision Making Across the Medical Education Continuum.

Viewing health care from a systems perspective-that is, "a collection of different things which, working together, produce a result not achievable by the things alone"-raises awareness of the complex interrelationships involved in meeting society's goals for accessible, cost-effective, high-quality health care. This perspective also emphasizes the far-reaching consequences of changes in one sector of a system on other components' performance. Medical education promotes this holistic view of health care in its curricula and competency requirements for graduation at the undergraduate and graduate training levels. But how completely does medical education apply a systems lens to itself?The continuum of medical training has undergone a series of changes that have moved it more closely to a systems organizational model. Competency assessment criteria have been expanded and more explicitly defined for learners at all levels of training. Outcomes data, in multiple domains, are monitored by external reviewers for program accreditation. However, translating increasing amounts of individual outcomes into actionable intelligence for decision making poses a formidable information management challenge.Assessment in systems is designed to impart a "big picture" of overall system performance through the synthesis, analysis, and interpretation of outcomes data to provide actionable information for continuous systems improvement, innovation, and long-term planning. A systems-based framework is presented for use across the medical education continuum to facilitate timely improvements in individual curriculum components, continuous improvement in overall program performance, and program decision making on changes required to better address society's health care needs.

Assessing the sustainability of crop production systems: Toward a common framework?

All three dimensions of sustainability —economic, social and environmental— must be integrated into a holistic assessment framework in the development of sustainable cropping systems (CS). Numerous sustainability assessment methods meet this requirement, but most of them handle only one type of production system (arable crops, fruit or vegetables). We propose here a common framework for sustainability assessment applicable to various types of crop production. The DEXiPM model, which was designed for the ex ante assessment of innovative arable CS was adapted to other production systems. Three groups of experts analyzed and modified this model, to develop suitable methods for assessing the sustainability of pomefruit orchards, field vegetable systems, and grapevine systems. We used the resulting models to formalize a sustainability assessment framework, in which a fixed core of hierarchically organized generic agricultural sustainability issues can be weighted according to stakeholder priorities, and a set of basic attributes can be estimated in a flexible manner, depending on the situation to be assessed and the data available. This common framework for sustainability assessment has several advantages. It can facilitate communication between stakeholders involved in the development of innovative production systems. It can also help researchers to identify gaps in knowledge and the means of bridging them. Its results can provide recommendations for policy makers, concerning actions likely to incite the adoption of innovative systems, for example. We also consider the general applicability and limitations of this framework.

Creating a Framework for Holistic Assessment of Aesthetics: A Response to Nilsson and Axelsson (2015) on Attributes of Aesthetic Quality of Textile Quality1.

Nilsson and Axelsson (2015) made an important contribution by linking recent scientific approaches from the field of empirical aesthetics with everyday demands of museum conservators of deciding which items to be preserved or not. The authors made an important effort in identifying the valuable candidates of variables - but focused on visual properties only and on quite high-expertise aspects of aesthetic quality based on very sophisticated evaluations. The present article responds to the target paper by developing the outline of a more holistic approach for future research as a kind of framework that should assist a multi-modal approach, mainly including tactile sense.