Empirical Validation Studies Research Articles

Theoretical and Methodological Foundations of Uncertainty Modeling in Real Estate Markets

Motivation: The need to improve the accuracy and reliability of market valuation and risk assessment in real estate markets, especially under conditions of uncertainty.Aim: To integrate theoretical foundations and methodological approaches for modeling aleatoric and epistemic uncertainties in real estate markets using credal networks and confidence boxes (c-boxes).Approach: This paper presents a comprehensive theoretical and methodological framework for uncertainty modeling in real estate markets, focusing on the application of credal networks and confidence boxes. It does not include empirical validation or practical case studies, instead providing a detailed conceptual and methodological discussion.Results: The proposed method demonstrates significant improvements in uncertainty quantification and market analysis accuracy in theoretical terms, offering valuable insights for investors, urban planners, and policymakers. However, empirical validation is suggested for future research to confirm practical applicability.

Do long-term acoustic-phonetic features and mel-frequency cepstral coefficients provide complementary speaker-specific information for forensic voice comparison?

A growing number of studies in forensic voice comparison have explored how elements of phonetic analysis and automatic speaker recognition systems may be integrated for optimal speaker discrimination performance. However, few studies have investigated the evidential value of long-term speech features using forensically-relevant speech data. This paper reports an empirical validation study that assesses the evidential strength of the following long-term features: fundamental frequency (F0), formant distributions, laryngeal voice quality, mel-frequency cepstral coefficients (MFCCs), and combinations thereof. Non-contemporaneous recordings with speech style mismatch from 75 male Australian English speakers were analyzed. Results show that 1) MFCCs outperform long-term acoustic phonetic features; 2) source and filter features do not provide considerably complementary speaker-specific information; and 3) the addition of long-term phonetic features to an MFCCs-based system does not lead to meaningful improvement in system performance. Implications for the complementarity of phonetic analysis and automatic speaker recognition systems are discussed.

Optimizing Supply Chain Resilience in the Era of Global Disruptions: A Comprehensive Framework for Effective Risk Management

In the contemporary global business environment, characterized by increasing interconnectivity and frequent disruptions, supply chain resilience has become paramount for organizational success. This thesis introduces a comprehensive framework aimed at optimizing supply chain resilience amidst such challenges. Drawing from extensive literature, theoretical models, and empirical research, the framework emphasizes proactive risk identification, robust mitigation strategies, and agile response mechanisms. It stresses collaboration, information sharing, and alignment among supply chain partners, leveraging technology for enhanced visibility and adaptability. Additionally, the framework underscores the significance of organizational culture, leadership, and employee empowerment in fostering resilience. Through empirical validation and case studies, practical insights are provided, addressing challenges like resource allocation and sustainability integration. Moreover, the thesis explores emerging trends such as climate change impacts and technological advancements, guiding organizations towards future-proofing their supply chains.

Empirical validation of building energy simulation model input parameter for multizone commercial building during the cooling season

This paper presents a critical advancement in Building Energy Modeling (BEM) through an empirical validation approach using a high-quality dataset from a multizone commercial office building in Oak Ridge, TN, USA. BEM is widely utilized in diverse construction applications, but its effectiveness relies on the accuracy of its predictions. The study focuses on empirical validation of input parameters in BEM, including building envelope data, infiltration modeling, and rooftop unit system performance curves. The validation of simulation input parameters leads to substantial improvements in the accuracy of simulation results. Notable both NMBE and cv (RMSE) values are reduced by 0.5 % for indoor air temperature and 17 % for indoor air relative humidity compared to the previous model. At the system level, both NMBE and cv (RMSE) values are reduced by 2 % for fan energy consumption and 4 % for cooling energy consumption, compared to the previous model. A literature review highlights a significant gap in empirical validation studies, which predominantly concentrate on either component-level or whole building validation. Furthermore, many studies employ simplified setups that may not faithfully represent the complexities of multizone commercial buildings. This paper distinguishes itself by emphasizing the critical importance of component-level input parameter validation. It underlines the need to validate data related to building envelope components and HVAC system performance curves, resulting in more accurate simulation outcomes. The utilization of actual multizone commercial building data enhances the study's practical relevance. In summary, this research underscores the pivotal role of input parameter validation in enhancing the accuracy and reliability of BEM.

In-situ empirical validation of common indoor climate parameters in an inhabited multizone dwelling

Combined building indoor climate and energy simulation models only recently gained vast popularity and their application has been moving from the research community to a broader audience. Yet, in-situ empirical validation of this new generation of complex multi-purpose dynamic simulation models has lagged behind. Using a dynamic multizone building indoor climate and energy simulation model in Modelica with the IDEAS library and buoyancy driven airflow components (validated with CONTAM), this research presents model validation results and lessons learned from an in-situ empirical validation study of common indoor climate parameters (i.e., indoor air temperature (), relative humidity (RH) and CO2 concentration (CO2)) for an inhabited and mechanically ventilated case study dwelling in The Netherlands. The simulation results show that the latest generation of building indoor climate and energy models in Modelica have great ability to accurately predict common indoor climate parameters in multizone inhabited dwellings (provided that user behavior info is available). Evaluation metrics for the three studied parameters show excellent calibration criteria (i.e., MAE between 0.60–0.78 °C (), 3.5–4.6% (RH) and 88–181 ppm (CO2)) and the accompanying graphs corroborate the findings. In the event that no motion sensor data is available, statistically generated occupancy profiles prove good representative alternatives on the condition that basic info is available about the number of inhabitants and the inhabitants’ lifestyle. In-situ monitoring for empirical model validation proves to be a real challenge full of (un)foreseen obstacles.

Empirical validation study and psychometric evaluation of the properties of the populist attitudes scale for the portuguese population

Background and objectiveRecent developments in Europe and Portugal provide a fertile ground for the rise of populism. Despite the growing interest in the topic, there is no reliable tool to gauge Portuguese citizens’ populist attitudes to date. The Populist Attitudes Scale (POP-AS), developed by Akkerman et al. [1], is one of the best-known instruments for measuring populist attitudes. However, no version for use in the Portuguese population is available. This paper describes the psychometric validation of the POP-AS for the Portuguese population.MethodsTrustworthy measures of validity suggested by Boateng et al. [2] to address the psychometric features of the POP-AS were approached. A robust psychometrical pipeline evaluated the reliability, construct validity, cross national/educational validity, and internal validity of the POP-AS.ResultsThe Portuguese version of the POP-AS exhibited sound internal consistency and demonstrated adequate properties of validity: a one-factor model was obtained, revealing evidence of construct validity; invariance was ensured for education and partially ensured for the country; All the items of the POP-AS revealed relatively good values of discrimination and contributed adequately to the total score of the scale, ensuring evidence of internal validity.ConclusionPsychometric analysis supports the POP-AS as a valid and reliable instrument for measuring populist attitudes among Portuguese citizens. A validation framework for measurement instruments in political science was proposed. Implications of the findings are discussed.

Machine Learning for Software Engineering: A Tertiary Study

Machine learning (ML) techniques increase the effectiveness of software engineering (SE) lifecycle activities. We systematically collected, quality-assessed, summarized, and categorized 83 reviews in ML for SE published between 2009 and 2022, covering 6,117 primary studies. The SE areas most tackled with ML are software quality and testing, while human-centered areas appear more challenging for ML. We propose a number of ML for SE research challenges and actions, including conducting further empirical validation and industrial studies on ML, reconsidering deficient SE methods, documenting and automating data collection and pipeline processes, reexamining how industrial practitioners distribute their proprietary data, and implementing incremental ML approaches.

Online Computerized Adaptive Tests of Children's Vocabulary Development in English and Mexican Spanish.

Measuring the growth of young children's vocabulary is important for researchers seeking to understand language learning as well as for clinicians aiming to identify early deficits. The MacArthur-Bates Communicative Development Inventories (CDIs) are parent report instruments that offer a reliable and valid method for measuring early productive and receptive vocabulary across a number of languages. CDI forms typically include hundreds of words, however, and so the burden of completion is significant. We address this limitation by building on previous work using item response theory (IRT) models to create computer adaptive test (CAT) versions of the CDIs. We created CDI-CATs for both comprehension and production vocabulary, for both American English and Mexican Spanish. Using a data set of 7,633 English-speaking children ages 12-36 months and 1,692 Spanish-speaking children ages 12-30 months, across three CDI forms (Words & Gestures, Words & Sentences, and CDI-III), we found that a 2-parameter logistic IRT model fits well for a majority of the 680 pooled vocabulary items. We conducted CAT simulations on this data set, assessing simulated tests of varying length (25-400 items). Even very short CATs recovered participant abilities very well with little bias across ages. An empirical validation study with N = 204 children ages 15-36 months showed a correlation of r = .92 between language ability estimated from full CDI versus CDI-CAT forms. We provide our item bank along with fitted parameters and other details, offer recommendations for how to construct CDI-CATs in new languages, and suggest when this type of assessment may or may not be appropriate.

An empirical validation study of Spolsky's language policy model

ABSTRACT This study investigates Spolsky's conception of language policy as the amalgam of language beliefs, language practices and language management. To the best of our knowledge, no quantitative evidence has ever been produced for his theoretical model. The current paper addressed this void, albeit specifically in the domain of Early Childhood Care and Education – scarce in terms of language policy research, but vital for children's (multilingual) language development. An Exploratory Factor Analysis yielded three valid and reliable scales that capture language policy from the perspective of early childhood professionals (N = 305) associated with infant welfare clinics in the Flemish Community of Belgium. Furthermore, a Confirmatory Factor Analysis indicated that the three components are ‘independent’ as well as ‘interconnected,’ thereby quantitatively corroborating Spolsky's theory. We conclude that language policy is no unitary concept; therefore, if one intends to study language policy holistically, the three indispensable components are to be identified. As a corollary, one simply cannot investigate management without considering beliefs and practices, and vice versa.

Use of an Automated System to Evaluate Feature Dissimilarities in Handwriting Under a Two-Stage Evaluative Process*, †.

The two-stage evaluative process is an established framework utilized by forensic document examiners (FDEs) for reaching a conclusion about the source(s) of handwritten evidence. In the second, or discrimination, stage, the examiner attempts to estimate the rarity of observations in a relevant background population. Unfortunately, control samples from a relevant background population are often unavailable, leaving the FDE to reach this determination based on subjective experience. Automated handwriting feature recognition systems are capable of performing both feature comparison and discrimination, yet these systems have not been subjected to empirical validation studies. In the present study, we repurposed a commercially available automated system to generate empirical distributions for ranking feature dissimilarity scores among pairs of handwritten phrases. The blinded results of this automated process were used to survey an international cohort of 36 FDEs regarding their strength of support for same- and different-writer propositions. The survey served to cross-validate FDE decision-making under the two-stage approach. Results from the survey demonstrated a clear pattern of response consistent with ground truth. Predictive regression analyses indicated that the automated feature dissimilarity scores and the log of their cumulative distribution functions accounted for 72% of the variability in FDE opinions. This study demonstrated that feature dissimilarity scores acquired using automated processes and their distributions are closely aligned with FDE decision-making processes supporting the heuristic value of the two-stage evaluative framework.

Prediction of RECRUITment In randomized clinical Trials (RECRUIT-IT)\u2014rationale and design for an international collaborative study

BackgroundPoor recruitment of patients is the predominant reason for early termination of randomized clinical trials (RCTs). Systematic empirical investigations and validation studies of existing recruitment models, however, are lacking. We aim to provide evidence-based guidance on how to predict and monitor recruitment of patients into RCTs. Our specific objectives are the following: (1) to establish a large sample of RCTs (target n = 300) with individual patient recruitment data from a large variety of RCTs, (2) to investigate participant recruitment patterns and study site recruitment patterns and their association with the overall recruitment process, (3) to investigate the validity of a freely available recruitment model, and (4) to develop a user-friendly tool to assist trial investigators in the planning and monitoring of the recruitment process.MethodsEligible RCTs need to have completed the recruitment process, used a parallel group design, and investigated any healthcare intervention where participants had the free choice to participate. To establish the planned sample of RCTs, we will use our contacts to national and international RCT networks, clinical trial units, and individual trial investigators. From included RCTs, we will collect patient-level information (date of randomization), site-level information (date of trial site activation), and trial-level information (target sample size). We will examine recruitment patterns using recruitment trajectories and stratifications by RCT characteristics. We will investigate associations of early recruitment patterns with overall recruitment by correlation and multivariable regression. To examine the validity of a freely available Bayesian prediction model, we will compare model predictions to collected empirical data of included RCTs. Finally, we will user-test any promising tool using qualitative methods for further tool improvement.DiscussionThis research will contribute to a better understanding of participant recruitment to RCTs, which could enhance efficiency and reduce the waste of resources in clinical research with a comprehensive, concerted, international effort.

Simplifying the Assessment of Measurement Invariance over Multiple Background Variables: Using Regularized Moderated Nonlinear Factor Analysis to Detect Differential Item Functioning

Determining whether measures are equally valid for all individuals is a core component of psychometric analysis. Traditionally, the evaluation of measurement invariance (MI) involves comparing independent groups defined by a single categorical covariate (e.g., men and women) to determine if there are any items that display differential item functioning (DIF). More recently, Moderated Nonlinear Factor Analysis (MNLFA) has been advanced as an approach for evaluating MI/DIF simultaneously over multiple background variables, categorical and continuous. Unfortunately, conventional procedures for detecting DIF do not scale well to the more complex MNLFA. The current manuscript therefore proposes a regularization approach to MNLFA estimation that penalizes the likelihood for DIF parameters (i.e., rewarding sparse DIF). This procedure avoids the pitfalls of sequential inference tests, is automated for end users, and is shown to perform well in both a small-scale simulation and an empirical validation study.

How Valid Are Social Vulnerability Models?

Social vulnerability models are becoming increasingly important for hazard mitigation and recovery planning, but it remains unclear how well they explain disaster outcomes. Most studies using indicators and indexes employ them to either describe vulnerability patterns or compare newly devised measures to existing ones. The focus of this article is construct validation, in which we investigate the empirical validity of a range of models of social vulnerability using outcomes from Hurricane Sandy. Using spatial regression, relative measures of assistance applicants, affected renters, housing damage, and property loss were regressed on four social vulnerability models and their constituent pillars while controlling for flood exposure. The indexes best explained housing assistance applicants, whereas they poorly explained property loss. At the pillar level, themes related to access and functional needs, age, transportation, and housing were the most explanatory. Overall, social vulnerability models with weighted and profile configurations demonstrated higher construct validity than the prevailing social vulnerability indexes. The findings highlight the need to expand the number and breadth of empirical validation studies to better understand relationships among social vulnerability models and disaster outcomes.

Accuracy of metabolic rate estimates from heart rate under heat stress-an empirical validation study concerning ISO 8996.

The standard ISO 8996 provides methods for the determination of metabolic rate from measured oxygen consumption (MVO2), as well as simplified estimation algorithms based on heart rate (MHR). We quantified the accuracy of these methods by comparing MHR with MVO2 measured in 373 climatic chamber experiments under different workloads and widely varying heat stress conditions. While our results confirmed the 5% accuracy level for MVO2, MHR considerably overestimated MVO2 due to the rise in core temperature concomitantly increasing heart rate by approximately 30 bpm/°C resulting in an overall error of 43%. After individually correcting for this bias the accuracy was 10–15% as stipulated by the standard. Thus, methods correcting for the thermal component of heart rate, e.g. by introducing intermittent resting periods of sufficient length of at least five min when investigating heat stress at workplaces, should become a mandatory element in the ongoing revision of the relevant standards.

Can the choice of building performance simulation tool significantly alter the level of predicted indoor overheating risk in London flats?

The accurate prediction of building indoor overheating risk is critical in order to mitigate its possible consequences on occupant health and wellbeing. The Chartered Institution of Building Services Engineers issued Technical Memorandum 59 (TM59) with the aim of achieving consistency in the modelling processes followed for the prediction of overheating risk in new dwellings. However, as each tool’s prediction may depend on its inherent assumptions, an inter-model comparison procedure was used to assess whether the choice of building performance simulation tool influences the overheating assessment. The predictions of two popular tools, IES VE and EnergyPlus, were compared for nine variations of a naturally ventilated, purpose built, London flat archetype, modelled under the default algorithm options. EnergyPlus predicted a high overheating risk according to TM59 criteria in seven out of the nine model variants, contrary to the low risk of all the IES VE variants. Analysis of heat transfer processes revealed that wind-driven ventilation and surface convection algorithms were the main sources of the observed discrepancies. The choice of simulation tool could thus influence the overheating risk assessment in flats, while the observed discrepancies in the simulation of air and heat transfer could have implications on other modelling applications. Practical application:Technical Memorandum 59 issued by the Chartered Institution of Building Services Engineers may be widely adopted within the industry to assist the prediction of overheating risk in new dwellings. This work suggests that the choice of building performance simulation tool can greatly influence the predicted overheating risk. Furthermore, the differences identified in the modelling of heat transfer processes could also impact other modelling applications. Following these results, the need for detailed empirical validation studies of naturally ventilated homes has been highlighted.

Validación psicométrica del POSIT para el cribado del consumo de riesgo de alcohol y otras drogas entre adolescentes

Early detection of alcohol and drug abuse among adolescents is decisive not only for rapid referral and intervention in cases of risk, but also as an indicator for use in the evaluation of prevention programs and public policies to reduce consumption. One of the most widely-used screening instruments in the world is the Problem Oriented Screening Instrument for Teenagers (POSIT) (Rahdert, 1991), whose substance use and abuse subscale (POSITUAS) is a brief tool of enormous applied potential. However, there is still no empirical validation study that would ensure its good psychometric performance in Spain. The aim of this paper is to analyse the psychometric properties of POSITUAS among Spanish adolescents. For this purpose, 569 students aged between 12 and 18 years (M = 14.71; SD = 1.79) were personally interviewed. The study sample was selected through two-stage sampling. The results obtained, using the Adolescent Diagnostic Interview (Winters & Henly, 1993) as the gold criterion, allow us to inform that the Spanish version of the POSITUAS has excellent psychometric behaviour, both at the level of internal consistency (a = .82) as well as regards sensitivity (94.3%) and specificity (83.9%), with an area under the ROC curve of .953. Also, the realisation of a Confirmatory Factor Analysis allows for verifying the one-dimensional character of the scale. As a result, POSITUAS is made available to researchers and professionals in the field of addictive behaviours for use with a minimum of psychometric guarantees.

Empirical validation of the CRAFFT Abuse Screening Test in a Spanish sample.

The CRAFFT Substance Abuse Screening Instrument, developed by the Center for Adolescents Substance Abuse Research (CeASAR) (Knight et al., 1999), is a screening tool for high-risk alcohol and drug risk consumption designed for use with adolescents. Since its publication it has been the subject of translations and validations in different countries, populations and contexts that have demonstrated its enormous potential. However, there is still no empirical validation study that would ensure its good psychometric performance in Spain. The aim of this paper is to develop an adapted version of the CRAFFT in Spanish and to analyze its psychometric properties in a sample of Spanish adolescents. For this purpose an individual interview was conducted on 312 adolescents aged between 12 and 18 years of age (M = 15.01; SD = 1.83) from the Galician community. The interview included a part of the Adolescent Diagnostic Interview (ADI) and the Problem Oriented Screening Instrument for Teenagers (POSIT). The results obtained, similar to those found in other countries, allow us to report that the Spanish version of the CRAFFT has a good psychometric behaviorproperties. It was found to have a satisfactory internal consistency with a Cronbach’s alpha value of .74. In terms of sensitivity and specificity, values of 74.4% and 96.4% respectively, were obtained and the area under the ROC curve was .946. The Spanish version of the CRAFFT is made available to researchers and professionals in the field of addictive behaviors, so that it can be used with the necessary psychometric guarantees.

Cognitive, experiential, and marketing factors mediate the effect of brand personality on brand equity

Many researchers have investigated the effects of brand personality on brand equity; however, the antecedent factors contributing to the development of brand personality have been largely ignored. We aimed to bridge this research gap by integrating cognitive, experiential, and marketing factors as antecedents to explore the effect of brand personality on the relationship between these antecedents and brand equity. Participants were 292 cosmetics users in Taiwan, who were asked to identify the cosmetic brands they favored, then interviewed about their perception of a brand they had mentioned. Results indicated that cognitive, experiential, and marketing factors all had significant effects on brand equity, and that brand personality served as a key mediator increasing the influence of these antecedents on brand equity. Our findings offer a reference point for academics to conduct further empirical validation studies, and for professionals to develop appropriate branding strategies in the cosmetics marketplace.

Mapping practices of Christian education: Toward a framework of contextual differences around the globe

This article reports the development of a framework that structures differences in Christian educational practices worldwide. One of its purposes is to simplify the complexity of the contexts in which global partners cooperate. The framework also offers the theoretical basis for an instrument that nongovernmental organizations can use to determine how to choose the appropriate support for Christian education in a certain region. Based on a literature search and a validation study in Zambia, we find three overarching domains in which 24 dimensions are identified: the teaching and learning processes in classrooms; the educational administration, including stakeholders and their motives, aims and ideals to maintain Christian education; and four kinds of societal factors (historical and cultural, religious, legal and political, and socioeconomic). The framework will be the basis for empirical validation studies in different regions. An operational instrument for determining support needs is currently being developed.

Assessing the influence of feedback-inclusive rapid prototyping on understanding the semantics of parallel UML statecharts by novice modellers

ContextUML diagrams are the de facto standard for analysing, communicating and designing software systems, as well as automated code generation. However there is a certain degree of difficulty in understanding a system represented by means of UML diagrams. ObjectOur previous research demonstrates a significant improvement in understanding the structural aspects of a system represented as a UML class diagram when using a feedback-inclusive prototype of a model. This paper extends our previous work with an empirical validation study for the effectiveness of the feedback-inclusive rapid prototyping (FIRP) method, on the comprehension of system dynamics represented as multiple interacting UML statecharts. Because models often combine structural and behavioural views that are highly intertwined, we additionally evaluate the effectiveness of the proposed method with respect to comprehension of the between-view consistency. MethodThe FIRP environment was built following the principles of Design Science Research in Information Systems. This study targets the empirical validation of the effectiveness of the proposed technique using an experimental study method. Two experiments were conducted with the participation of 65 final-year master students in the context of different modelling courses from different study programs at KU Leuven using two two-group factorial experimental designs. The effectiveness of the FIRP method was measured by comparing students’ performance between the cycles with and without the use of the method, using the understandability (comprehension test results) as the dependent variable and the use of FIRP as the independent variable. Effects from unknown variables were neutralized by means of randomized group compositions. The effectiveness of FIRP was additionally assessed with respect to personal characteristics (age, gender, previous knowledge, self-efficacy) and user acceptance (perceived ease of use, perceived utility, preference, satisfaction). ResultsThe findings reveal a significant positive impact of the use of the prototyping technique on students’ comprehension of system dynamics represented as multiple interacting statecharts. ConclusionsThe findings provide empirical support for the advantage of the use of FIRP over manual inspection of interacting statecharts. The findings also suggest that the method is suitable for training system's analysis and modelling skills when UML statecharts are involved.