Local Validation Studies Research Articles

Comparative validation of global and climate-specific decomposition models at high latitudes

Reliable modelling of the solar potential of urban surfaces (i.e. roofs, facades and ground) in the built environment can contribute to boost the exploitation of solar energy. Empirical and quasi-physical models are combined into model chains for this scope. Among these models, the decomposition models are used to separate the direct from the diffuse solar irradiance. Such variables are rarely measured, thus making necessary their computation. Studies in literature pointed out that Yang4 is the best-performing decomposition model globally. However, in geographically limited applications, quasi-universal decomposition models like Yang4 and Engerer4 can be outperformed by local models (i.e. models parametrized with climate-specific data) such as Skartveit3 and Starke3. This makes necessary to perform local validation studies to verify the findings from worldwide validation studies. In this study, the four best-performing decomposition models are implemented in Python and experimentally validated against 1-min solar irradiance data (i.e. direct and diffuse irradiance) of Trondheim (Lat. 63°26′N, Norway). More than 1-year long data are considered. The study confirms that the Yang4 model performed the best for high-latitude applications, but Starke3 can achieve a level of performance close to Yang4.

Effect of COVID-19 on sexual function and activities among reproductive-age women in Ibadan, South-West Nigeria.

Since the outbreak of COVID-19 disease, the environment, families, individuals, and their ways of living have been affected. Social distancing was one of the strategies for the prevention of SARS-CoV-2 infection. It led to limited social interactions for fear of contracting the infection, which ultimately affected relationships, among which is sexual health. To determine the risk of female sexual dysfunction and the effect of the COVID-19 pandemic on sexual function and activities among women of reproductive age in Ibadan, South-West Nigeria. This cross-sectional study of 218 reproductive-age women evaluated the sexual function before, during, and after the COVID-19 disease pandemic and lockdown. Pretested semistructured self-administered questionnaires with the Female Sexual Function Index (FSFI) were used for data collection. Information collected included sociodemographic and gynecologic characteristics and COVID-19 experiences, as well as sexual history and function before, during, and after the COVID-19 pandemic lockdown. The level of significance was set at P < .05. Respondents were aged 20 to 50years (95%) with a mean ± SD age of 34.82 ± 8.52 years; the majority were married (75.58%); one-fifth (21.9%) tested positive for COVID-19 infection; and participants who tested positive for COVID-19 infections were mostly health care workers. An absence of sexual activity was reported in 9.18% of participants during lockdown, 7.73% before lockdown, and 4.18% after lockdown. The risk of female sexual dysfunction was prevalent among participants, especially those who tested positive for COVID-19 infection. The prevalence was worse during the pandemic lockdown (53.57%) than before (48.39%) or after (51.61%), and a similar pattern was seen among those who tested negative. There was no statistically significant difference in risk of developing sexual dysfunction during the COVID-19 pandemic between those who tested positive and negative to COVID-19. The arousal and desire domains contributed the highest proportion of low FSFI scores. Nationwide lockdowns, which may be a method of control for pandemics, may result in psychosocial complications such as female sexual dysfunction. Most respondents had tertiary education and were able to respond to self-administered questionnaires, yet the risk of recall bias remains a concern whereas the pandemic met the world unprepared and baseline FSFI prior the pandemic was not available for participants. There are no local validation studies for the FSFI in Nigeria, which could have aided interpretation of results. A low FSFI score is prevalent in Ibadan, South-West Nigeria, with a higher incidence reported during the COVID-19 pandemic lockdown.

Validation of decomposition models for solar irradiance at high latitudes: A preliminary study

The accurate estimation of buildings’ solar potential contributes to boost the exploitation of solar energy at high latitudes. The decomposition of global irradiation into the direct and diffuse fractions is a fundamental step of the solar irradiance model chain. Diffuse and direct irradiation are, in fact, rarely measured. Previous works recommended Yang4 as the decomposition model with the best overall performance. However, in geographically limited applications, quasi-universal decomposition models such as Yang4 and Engerer4 can be outperformed by local models (i.e., models parametrized with climate-specific data) such as Skartveit3 and Starke3. This makes necessary to perform local validation studies to verify the findings from worldwide validation studies. In this study, the four decomposition models are implemented in Python and experimentally validated against one-minute solar irradiance data (i.e., direct and diffuse irradiance) of Trondheim (Lat. 63°26’ N, Norway). Two months representative of clear sky (August) and overcast (October) conditions are considered. The study confirms that the Yang4 model performed the best for high-latitude application: the nMBE ranged from -0.54% (August) to 0.65% (October), the nRMSE from 17.18% (August) to 22.29% (October), and the R2 from 0.96 (August) to 0.97 (October). However, Skartveit3 combines a level of performance close to Yang4 with the lower number of input parameters.

Neuron-specific enolase and long-term neurological outcome after OHCA – A validation study

AimsTo investigate what NSE levels predict long-term neurological prognosis at 24, 48 and 72 hours after ROSC in a cohort of out-of-hospital cardiac arrest and to validate previously suggested NSE cut-offs, including the latest ERC guidelines (2021). MethodsPatients admitted to intensive care units in four hospitals in Southern Sweden between 2014–2018 were included. Blood samples were handled by a single local laboratory. The primary outcome was neurological outcome according to the Cerebral Performance Category (CPC) scale at 2–6 months after cardiac arrest. Results368 patients were included for analysis. A ≤2% false positive rate for the prediction of poor neurological outcome was achieved with an NSE cut-off value of >101 μg/L at 48 hours and >80 μg/L at 72 hours. The cut-off suggested by the recent ERC guidelines of >60 μg/L at 48 and/or 72 hours generated a false positive rate of 4.3% (95 %CI 0.9–7.4%). ConclusionA local validation study of the ability of serum levels of neuron-specific enolase to predict long-term poor neurological outcome after out-of-hospital cardiac arrest generated higher cut-offs than suggested by previous publications.

Clinical Validation Is the Key to Adopting AI in Clinical Practice.

Clinical Validation Is the Key to Adopting AI in Clinical Practice.

The local technical validation of new plasma tube with a mechanical separator

Abstract Background Unsuitable samples are common problem for laboratories. The blood collection tubes need to be validated or verified prior to their being used in the routine laboratory for reducing this situation. Objective We aim to compare the technical qualifications of routinely used BD Vacutainer® Serum Separator Tubes™ II Advance Plus with BD Vacutainer® Barricor™ LH Plasma Tubes for local technical validation. Materials and methods Apparently healthy 150 voluntary subjects were enrolled in the study. Samples were collected in two separated tubes by a single phlebotomist. Twelve quality indicators were used to compare these two different types of tubes for local technical validation. Differences (%) between them were calculated with the formula proposed by EFLM. In case of any difference of less than 1% for each indicator, the evaluated tube was considered as non-inferior. Results Indicators, such as tubes with physical defects, that fail to create vacuum, not properly fitting into the blood collection device, under filling (10%), cracked tubes, tubes exterior surface contaminated with blood, hemolysed specimens, including fibrin strand/mass in the sample, red blood cell adhesion, poor/incomplete barrier formation were found non-inferior in Barricor™ tubes. White particulate matter (WPM) was observed in 24.6% of Barricor™. Therefore, the last indicator (tubes including gel/foreign material/WPM in sample after centrifugation) was found inferior for Barricor™. Conclusion Technical local validation studies should be encouraged in terms of quality management. It was thought that WPM would not cause any interference in a properly filled tube. In addition to, Barricor™ was also found to be technically acceptable when evaluated through using all other indicators.

Autoestima, Optimismo y Resiliencia en Niños en Situación de Pobreza

La pobreza es un importante problema económico y social, que está asociado a diversos factores de riesgo para el desarrollo físico y psicológico, especialmente de los niños, ya que incrementa la probabilidad de estar en desigualdad. Existe evidencia empírica que indica que tanto la autoestima como el optimismo se asocian de manera positiva significativa con la resiliencia en muestras de adultos en pobreza extrema. Sin embargo, esto no se ha investigado en el caso de los niños y las niñas. Por ello, se plantea la pregunta de investigación: ¿la autoestima y el optimismo influyen en la resiliencia en una muestra de niños/as en situación de pobreza? En este estudio, la resiliencia es entendida en términos de factor protector interno (PFI), factor protector externo (PFE) y empatía (FE). Asimismo, se operacionalizó la autoestima en términos de: yo (YO), familia (FAM), fracaso (FRA), trabajo intelectual (TI), éxito (ÉXITO) y afectivo-emocional (AEM); y el optimismo, como una característica disposicional (OD) y situacional (OS). Este estudio es de corte transversal, correlacional y no experimental, en el que fueron evaluados 188 participantes, el 38.3% son niños (n = 72) y el resto niñas (61.7%, n = 116) entre 9 y 12 años de edad (M =10.59, DT= .69) todos estudiantes de educación básica, que asisten a una escuela pública de una comunidad rural en situación de pobreza. Se aplicaron tres escalas: 1) escala de autoestima, 2) escala de optimismo y 3) escala de resiliencia, todos son instrumentos de autoreporte, tipo Likert. Se llevó a cabo la validación local de las escalas a través de análisis factoriales exploratorios y confirmatorios. Asimismo, se planteó un modelo de ecuaciones estructurales para evaluar la asociación de los principales constructos. Los análisis revelaron que la variable autoestima fue constituida por cuatro de las seis dimensiones, el optimismo por dos dimensiones y resiliencia por tres dimensiones. Los índices de bondad de ajuste muestran un modelo estructural aceptable. Tal y como se había planteado, el modelo estructural indica que tanto la variable autoestima como el optimismo, ejercen un efecto directo sobre la resiliencia. Sin embargo, la variable optimismo mostró un mayor efecto sobre la resiliencia que la autoestima. Concluimos que se requiere evidencia empírica adicional para sustentar la robustez del modelo. Para ello, sugerimos una muestra más grande e incorporar variables familiares y contextuales. De esta manera, se puede obtener evidencia respecto a los factores de protección en el modelo. Además, estos hallazgos son útiles para incorporarlos en programas de intervención dirigidos a niños que viven en situación de pobreza, con la finalidad de generar acciones en pro de su bienestar.

Validación de un Inventario de Comportamientos Agresivos en el Tránsito

Anualmente fallecen más de 20 mil personas en accidentes viales. (Cervantes-Trejo, 2009). Asimismo, otras 700 mil sufren daños que requieren de hospitalización, así como 40 mil personas sufren lesiones que terminan en algún tipo de discapacidad (Cervantes-Trejo, 2009). Sin embargo, estos números rápidamente van en aumento año tras año (Cervantes-Trejo, 2009). Una de las causas principales de la accidentalidad es la agresividad vial. Para este estudio, la agresividad vial se define en términos de poca consideración, conducción de riesgo y manifestación de frustración/enfado. Para ello, este estudio estableció como objetivo principal el construir el Inventario de Comportamientos Agresivos en el Tránsito que midiera el constructo. Se planteó la siguiente pregunta de investigación: ¿Es válido y fiable el inventario generado? Esta investigación fue una validación local en Cuernavaca, Morelos, con 2454 participantes. Se hicieron tres estudios. El primero encontró que hubo más ítems generados en manifestación de frustración/enfado que para poca consideración y conducción de riesgo. El segundo estudio encontró una alta validez de constructo y aceptable consistencia interna. El tercer estudio encontró una alta validez de constructo y buena consistencia interna. Sin embargo, no se evaluaron la validez de contenido, convergente ni divergente. Se concluye que el Inventario de Comportamientos Agresivos es un instrumento válido pero requiere de estudios adicionales para evaluar sus propiedades psicométricas en una muestra representativa.

Local Validation Study for NutricheQ Tool in Lebanon

Background: Globally, there is a trend towards consumption of energy-dense, nutrient-poor foods leading to dietary imbalances in toddlers. Nutrient deficiencies are associated with the risk of obesity and other health risks. NutricheQ is an 18-item questionnaire that can quickly identify these risks based on dietary patterns in healthy toddlers. Objectives: This study aimed to validate the NutricheQ tool, describe the dietary intake of toddlers and classify them into risk groups. Methods: This was a national, multi-center, observational, validation study conducted at 21 medical centers in Lebanon. Data was collected from NutricheQ administered to the parents of toddlers during a single outpatient visit, and food diary filled by parents during a 4-day follow-up. The validation process of NutricheQ was done using Pearson’s and Spearman’s correlation, ANOVA and ANCOVA. The accuracy of NutricheQ in the Identifying categories of nutritional risk was evaluated using the receiver operating characteristic (ROC) curves. All analyses were performed using IBM-SPSS version 24 and P value of ≤ 0.05 was considered to be statistically significant. Results: Of the 487 toddlers enrolled, 467 had full data on NutricheQ and food diary data was available for 247 only. Majority of toddlers in Lebanon (80% - 88%) were categorized in the moderate or low-risk groups of dietary intake indicating good eating habits. There was a significant positive association between the total NutricheQ score and both, age (r = 0.11, P = 0.021) and BMI Z-score (r = 0.11, P = 0.021). ROC fitting showed a very low discriminatory ability of the scores to correctly classify toddlers into the high-risk group based on their BMI Z-score. Significant positive correlation was obtained between the NutricheQ total score and exchangeable fat and meat with high fat, in the food diaries (R = 0.170, P = 0.045), indicating risk. Also, there was a negative association between the total NutricheQ score and soluble and insoluble fiber (Rs = 0.181, P = 0.004). Conclusions: Results from this study showed that NutricheQ has reasonable validity in the Lebanese context to assess the dietary quality of toddlers in Lebanon and in identifying those at increased nutritional risk.

Assessing Exposure to Household Air Pollution: A Systematic Review and Pooled Analysis of Carbon Monoxide as a Surrogate Measure of Particulate Matter.

Background:Household air pollution from solid fuel burning is a leading contributor to disease burden globally. Fine particulate matter () is thought to be responsible for many of these health impacts. A co-pollutant, carbon monoxide (CO) has been widely used as a surrogate measure of in studies of household air pollution.Objective:The goal was to evaluate the validity of exposure to CO as a surrogate of exposure to in studies of household air pollution and the consistency of the relationship across different study settings and conditions.Methods:We conducted a systematic review of studies with exposure and/or cooking area and CO measurements and assembled 2,048 and CO measurements from a subset of studies (18 cooking area studies and 9 personal exposure studies) retained in the systematic review. We conducted pooled multivariate analyses of associations, evaluating fuels, urbanicity, season, study, and CO methods as covariates and effect modifiers.Results:We retained 61 of 70 studies for review, representing 27 countries. Reported correlations (r) were lower for personal exposure (range: 0.22–0.97; ) than for cooking areas (range: 0.10–0.96; ). In the pooled analyses of personal exposure and cooking area concentrations, the variation in ln(CO) explained 13% and 48% of the variation in ln(), respectively.Conclusions:Our results suggest that exposure to CO is not a consistently valid surrogate measure of exposure to . Studies measuring CO exposure as a surrogate measure of PM exposure should conduct local validation studies for different stove/fuel types and seasons. https://doi.org/10.1289/EHP767

Validity Stability Across Entering College Cohorts: Exploring the temporal generalizability of local validity estimates

Local validity studies rely on the assumption that validity estimates from one incumbent sample approximate validity for future applicant pools. We test this assumption using SAT scores and high school grades as predictors of first year college grade point average across multiple college applicant pools for over 100 schools. We present evidence for substantial absolute and rank order consistency in validity estimates. However, this consistency is far less than perfect, resulting in potentially meaningful utility differences over time. In addition, observed fluctuations are not fully explained by sampling error alone.

Should Employers Rely on Local Validation Studies or Validity Generalization (VG) to Support the Use of Employment Tests in Title VII Situations?

Since the landmark U.S. Supreme Court case Griggs v. Duke Power (1971), employers have been subject to challenge by plaintiffs or government enforcement agencies when they use employment tests that have adverse impact. In such situations the 1991 Civil Rights Act requires employers to demonstrate that the test is “…job related for the position in question and consistent with business necessity.” Employers typically rally such a defense by addressing the Uniform Guidelines on Employee Selection Procedures—the federal treatise framed in 1978 for the express purpose of Title VII enforcement—along with relevant professional standards (Joint Standards, 1999; SIOP Principles, 2003). To address such requirements, employers are faced with several viable validation alternatives, ranging from conducting a local criterion-related validation study to relying on validity evidence from other studies for similar positions, employers, and tests (i.e., Validity Generalization). The strengths and limitations of both of these strategies in a Title VII context are reviewed, and employers are ultimately encouraged to select the local validation strategy whenever technically feasible for a variety of reasons discussed.

Choosing the best method for local validity estimation: Relative accuracy of meta-analysis versus a local study versus Bayes-analysis.

This study assessed the relative accuracy of 3 techniques--local validity studies, meta-analysis, and Bayesian analysis--for estimating test validity, incremental validity, and adverse impact in the local selection context. Bayes-analysis involves combining a local study with nonlocal (meta-analytic) validity data. Using tests of cognitive ability and personality (conscientiousness) as predictors, an empirically driven selection scenario illustrates conditions in which each of the 3 estimation techniques performs best. General recommendations are offered for how to estimate local parameters, based on true population variability and the number of studies in the meta-analytic prior. Benefits of empirical Bayesian analysis for personnel selection are demonstrated, and equations are derived to help guide the choice of a local validity technique (i.e., meta-analysis vs. local study vs. Bayes-analysis).

Combining predictors to achieve optimal trade-offs between selection quality and adverse impact.

The authors propose a procedure to determine (a) predictor composites that result in a Pareto-optimal trade-off between the often competing goals in personnel selection of quality and adverse impact and (b) the relative importance of the quality and impact objectives that correspond to each of these trade-offs. They also investigated whether the obtained Pareto-optimal composites continue to perform well under variability of the selection parameters that characterize the intended selection decision. The results of this investigation indicate that this is indeed the case. The authors suggest that the procedure be used as one of a number of potential strategies for addressing the quality-adverse impact problem in settings where estimates of the selection parameters (e.g., validity estimates, predictor intercorrelations, subgroup mean differences on the predictors and criteria) are available from either a local validation study or meta-analytic research.

Implications of empirical Bayes meta-analysis for test validation.

Empirical Bayes meta-analysis provides a useful framework for examining test validation. The fixed-effects case in which rho has a single value corresponds to the inference that the situational specificity hypothesis can be rejected in a validity generalization study. A Bayesian analysis of such a case provides a simple and powerful test of rho = 0; such a test has practical implications for significance testing in test validation. The random-effects case in which sigma2rho > 0 provides an explicit method with which to assess the relative importance of local validity studies and previous meta-analyses. Simulated data are used to illustrate both cases. Results of published meta-analyses are used to show that local validation becomes increasingly important as sigma2rho increases. The meaning of the term validity generalization is explored, and the problem of what can be inferred about test transportability in the random-effects case is described.

Demographic variables in medical school admission

Admission committees report that demographic variables, although accurate, reliable, and easily obtained from applicants to medical schools, are only moderately important in their decision making. This may be because the committees are concerned about the validity and legality of using such data as admissions criteria. This essay discusses the research on the validity of demographic variables and the recommendations for their legal use in selecting students for medical school. The relationships of age, gender, size of hometown, parental education, parental occupation, parental income, and marital status to medical school outcomes of preclinical performances, clinical performance, attrition, specialty choice, and practice location are summarized. [Race or ethnic group is discussed in a separate essay in this issue.] Although the authors focus on the predictive value of demographic variables, these variables play a more important role as the moderator variables for other predictors of medical school outcomes. The full value of using demographic variables derives from data obtained in local validity studies. To comply with the equal-protection and due-process requirements, admission officers must assure that the use of demographic information in the admission process is explicit in bulletins given to applicants and is uniformly applied in the evaluation of all applicants.

Automated MMPI interpretation based on a modification of Gilberstadt's codebook.

Several limitations to commercially available automated MMPI reports prompted the psychology staff at Topeka VA Hospital to develop its own automated interpretive system. The computer programs of the system require further modification, and local validity studies of the automated interpretations are needed. However, it is hoped that this project will encourage others to consider the development of a local automated interpretive system as an alternative to commercial interpretive services.