Digital Assessment Tool Research Articles

Implementation and Impact of an Adapted Digital Perioperative Competency-Building Tool to Enhance Teaching, Learning And Feedback in Postgraduate Competency-Based Medical Education

The purpose of this educational intervention was to introduce, iteratively adapt, and implement a digital formative assessment tool in a surgical speciality. The study also evaluated the intervention's impact on perioperative teaching, learning, feedback, and surgical competency. A participatory action research model with a mixed methods approach. This study was performed over 10 months in an institutional hospital in South Africa with a general surgery department. Twelve supervising surgical trainers/faculty and 12 surgical trainees/residents consented to participate in the intervention. The first 4 months of the intervention focused on relationship building, a multi-stakeholder contextual needs assessment and training sessions to support a shared mindset and shift in the teaching and learning culture. The final adapted perioperative competency-building tool comprised a 23-item assessment with four open-text answers (Table 1). Over the following 6-month period, 48 workplace-based competency-building perioperative evaluations were completed. Most trainees took less than 5 minutes to self-assess (67%) before most trainers (67%) took less than 5 minutes to give oral feedback to the trainee after the perioperative supervised learning encounter. On average, the digital tool took 6 minutes to complete during the bidirectional perioperative teaching and learning encounter with no negative impact on the operational flow. All trainers and trainees reported the training and implementation of the digital tool to be beneficial to teaching, learning, feedback, and the development of surgical competency. Analysis of the completed tools revealed several trainees showing evidence of progression in surgical competency for index procedures within the speciality. The focus groups and interviews also showed a change in the teaching and learning culture: more positively framed, frequent, structured, and specific feedback, improved accountability, and trainee-trainer perioperative readiness for teaching. Highlighted changes included the usefulness of trainee self-assessment before perioperative trainer feedback and the tool's value in improving competency to Kirkpatrick Level 4. Implementing an adapted digital Workplace-Based Assessment (WBA) tool using a participatory action research model has proven successful in enhancing the effectiveness of supervised perioperative teaching and learning encounters. This approach has improved teaching and feedback practices, facilitated the development of surgical competency, and ultimately impacted the overall culture to Kirkpatrick level 4. Importantly, it has positively influenced the trainee-trainer relationship dynamic. Based on these positive outcomes, we recommend using this effective method and our relationship-centred framework for implementing formative competency-building tools in future studies. By doing so, larger-scale and successful implementation of Competency-Based Medical Education (CBME) could be achieved in various contexts. This approach can potentially enhance teaching and learning encounters, promote competency development, and improve the overall educational experience for surgical trainees and trainers.

Navigating the digital world: development of an evidence-based digital literacy program and assessment tool for youth

The rapid expansion of digital connectivity has provided youth with wide-ranging access to digital platforms for communication, entertainment, and education. In light of this profound shift, there have been growing concerns about online safety, data privacy, and cybersecurity. A critical factor influencing the ability of youth to responsibly navigate digital platforms is digital literacy. While digital literacy programs have been implemented in various regions worldwide, significant disparities remain not only in overall digital literacy levels, but also the assessment of digital literacy initiatives. To address these challenges, an environmental scan and literature review were conducted to identify existing digital literacy programs in Canada developed specifically for youth, as well as digital literacy assessment tools, respectively. The search encompassed peer-reviewed articles, organizational curricula, and assessment measures indexed in various databases and organization websites. The environmental scan identified 15 programs targeting key components of digital literacy such as data safety, cyberbullying, and digital media. The literature review identified 12 digital literacy assessment tools. Based on the findings, data were synthesized from shortlisted programs and assessment tools to inform the development of both a new digital literacy program and assessment tool to complement the youth-focused program. The new program focuses on four key components: (1) digital fluency, (2) digital privacy and safety, (3) ethics and empathy, and (4) consumer awareness. A 15-item assessment tool was also developed consisting of 4–5 questions specific to each program component. Given the growing importance of digital competencies, a youth-focused program and assessment tool are crucial for understanding and addressing digital literacy among this vulnerable cohort. This program's adaptability allows for customization across sociodemographic target groups, including culturally diverse and geographically remote communities—an aspect that has the potential to enhance digital literacy across settings. Implementing digital literacy programs can better prepare youth for an increasingly digital world, while minimizing potential risks associated with technology use.

Understanding the limitations of puberty measures to co-design a novel digital assessment tool

Understanding the limitations of puberty measures to co-design a novel digital assessment tool

Changes in Digital Speech Measures in Asymptomatic Carriers of Pathogenic Variants Associated With Frontotemporal Degeneration.

Clinical trials developing therapeutics for frontotemporal degeneration (FTD) focus on pathogenic variant carriers at preclinical stages. Objective, quantitative clinical assessment tools are needed to track stability and delayed disease onset. Natural speech can serve as an accessible, cost-effective assessment tool. We aimed to identify early changes in the natural speech of FTD pathogenic variant carriers before they become symptomatic. In this cohort study, speech samples of picture descriptions were collected longitudinally from healthy participants in observational studies at the University of Pennsylvania and Columbia University between 2007 and 2020. Participants were asymptomatic but at risk for familial FTD. Status as "carrier" or "noncarrier" was based on screening for known pathogenic variants in the participant's family. Thirty previously validated digital speech measures derived from automatic speech processing pipelines were selected a priori based on previous studies in patients with FTD and compared between asymptomatic carriers and noncarriers cross-sectionally and longitudinally. A total of 105 participants, all asymptomatic, included 41 carriers: 12 men [30%], mean age 43 ± 13 years; education, 16 ± 2 years; MMSE 29 ± 1; and 64 noncarriers: 27 men [42%]; mean age, 48 ± 14 years; education, 15 ± 3 years; MMSE 29 ± 1. We identified 4 speech measures that differed between carriers and noncarriers at baseline: mean speech segment duration (mean difference -0.28 seconds, 95% CI -0.55 to -0.02, p = 0.04); word frequency (mean difference 0.07, 95% CI 0.008-0.14, p = 0.03); word ambiguity (mean difference 0.02, 95% CI 0.0008-0.05, p = 0.04); and interjection count per 100 words (mean difference 0.33, 95% CI 0.07-0.59, p = 0.01). Three speech measures deteriorated over time in carriers only: particle count per 100 words per month (β = -0.02, 95% CI -0.03 to -0.004, p = 0.009); total narrative production time in seconds per month (β = -0.24, 95% CI -0.37 to -0.12, p < 0.001); and total number of words per month (β = -0.48, 95% CI -0.78 to -0.19, p = 0.002) including in 3 carriers who later converted to symptomatic disease. Using automatic processing pipelines, we identified early changes in the natural speech of FTD pathogenic variant carriers in the presymptomatic stage. These findings highlight the potential utility of natural speech as a digital clinical outcome assessment tool in FTD, where objective and quantifiable measures for abnormal behavior and language are lacking.

Start to Move: Measuring the Feasibility of a Teacher-Led Digital Fundamental Movement Skills Assessment Tool

Purpose: This study evaluated the feasibility of the “Start to Move” (S2M) digital assessment of children’s fundamental movement skills being implemented by primary school teachers within PE lessons. Methods: Nine primary school teachers in the United Kingdom trialed S2M weekly over a 6-week period. Posttrial surveys and interviews were used to ascertain responses. Results: Feasibility was measured using seven dimensions of Bowen et al. framework; acceptability, demand, implementation, practicality, adaptation, integration, and expansion. Acceptance and demand of S2M was high with participants feeling that its contents aligned to the PE curriculum. Participants were able to implement S2M with PE lessons without assistance and stated that they would continue to use it within their teaching. They felt S2M would enhance their teaching and would recommend it to other teachers. Discussion/Conclusion: S2M is feasible for primary teachers to implement within PE lessons and has the potential to heighten the use of assessment for learning within PE in primary schools.

What Can We Learn from a Co-Creation Journey for a Quick Scan Digital Transformation Maturity Assessment Tool for Development NGOs?

What Can We Learn from a Co-Creation Journey for a Quick Scan Digital Transformation Maturity Assessment Tool for Development NGOs?

Associations between Digital Literacy and Executive Functions in College Undergraduates

Digital literacy can be defined as a set of skills for searching, accessing, navigating, integrating, and evaluating the reliability of information on the internet, mainly in written form. The present study aimed to examine the contribution of executive function skills (inhibition, cognitive flexibility and working memory) to performance on a digital literacy assessment tool in college undergraduates. Eighty-five Argentinean university undergraduates (56 female, mean age 20.34 ± 4.6 years) completed a digital literacy test and a computerized executive functions battery. After controlling for vocabulary scores and sociodemographic factors, cognitive flexibility was the main predictor of digital literacy scores. In particular, it was associated with information search and integration task performances. Visuospatial working memory predicted information search scores and inhibition correlated positively with reliability evaluation tasks. Our results indicated: 1) a general involvement of cognitive flexibility in digital literacy, possibly reflecting the need to shift between multiple documents, reading or navigation strategies; 2) a specific link between visuospatial working memory and information search, which might be indicating a spatial processing load related to hyperlink navigation; and 3) a possible involvement of inhibition in the evaluation of web documents, which may contribute to the suppression of irrelevant information.

Stakeholder-Informed Solutions To Address Barriers for Alcohol Screening and Brief Intervention in Thai Hypertension Care.

Premature deaths from NCDs disproportionately affect people in low- and middle-income countries. Since alcohol use is one of the most common causes of reversible hypertension, interventions targeting alcohol use may be a feasible and effective low-cost approach to synergistically reduce the prevalence of harmful drinking and high blood pressure. This study sought to identify key factors in successfully implementing alcohol use screening and brief intervention in hypertension care in Thailand. For this purpose, we surveyed participants (NRound 1 = 91, NRound 2 = 27) from three different groups of Thai stakeholders (policy- and decisionmakers, primary healthcare practitioners, and patients diagnosed with hypertension) in a two-round stakeholder elicitation. In round 1, we identified limited resources, lack of clear guidelines for lifestyle intervention, stigmatization, and inconsistent monitoring of patients' alcohol use as important barriers. In round 2, we sought to elicit solutions for the barriers identified in round 1. While stakeholders emphasized the need for adaptability to existing realities in Thai primary healthcare such as a high workload and limited digitization, they favorably evaluated a digital alcohol assessment tool with integrated, tailored advice for brief intervention as a potential scalable solution. Findings suggest that as one possible route to reduce the NCD burden caused by hypertension in Thailand, primary healthcare services may be enhanced by digital tools that support resource-effective, intuitive, and seamless delivery of alcohol screening and brief intervention.

Challenges in reaching patients with severe mental illness for trials in general practice—a convergent mixed methods study based on the SOFIA pilot trial

BackgroundPatients with severe mental illness (SMI) die prematurely due to undetected and inadequate treatment of somatic illnesses. The SOFIA pilot study was initiated to mend this gap in health inequity. However, reaching patients with SMI for intervention research has previously proven difficult. This study aimed to investigate the recruitment of patients with SMI for the SOFIA pilot study in 2021.MethodsWe used a mixed-method convergent design. The qualitative material comprised 20 interviews with general practitioners (GPs) and staff, during patient recruitment. The quantitative data consisted of process data on baseline characteristics, GPs reported reasons for excluding a patient, reported reasons for patients declining participation, and registered data from a Danish population of patients with SMI. We used thematic analysis in the qualitative analysis and descriptive statistics for the quantitative analysis. Pillar integration was used for integrating the material.ResultsOur findings show that selection bias occurred in the pilot study. We describe four main themes based on the integrated analysis that highlights selection issues: (1) poor data quality and inconsistency in defining severity definitions troubled identification and verification, (2) protecting the patient and maintaining practice efficiency, (3) being familiar with the patient was important for a successful recruitment, and (4) in hindsight, the GPs questioned whether the target population was reached. ConclusionsIn the light of theories of professions and street-level bureaucracy, we find that the main drivers of the patient selection bias occurring in the SOFIA pilot study were that 1) GPs and staff mended eligibility criteria to protect certain patients and/or to minimize workload and maintain efficiency in the practice 2) the data from the GP record systems and the digital assessment tool to assist recruitment was not optimal. Interventions targeting this patient group should carefully consider the recruitment strategy with a particular focus on professionals’ discretionary practices and information technology pitfalls.Trial registrationThe pilot trial protocol was registered on the 5th of November 2020. The registration number is NCT04618250.

S318 Implementation of a Digital Cancer Risk Assessment Tool and Genetic Testing Program in Gastroenterology Practices Throughout the US Identifies Inherited Cancer Susceptibility Syndromes in 15.6% of Patients Tested

S318 Implementation of a Digital Cancer Risk Assessment Tool and Genetic Testing Program in Gastroenterology Practices Throughout the US Identifies Inherited Cancer Susceptibility Syndromes in 15.6% of Patients Tested

A Novel, Expert-Endorsed, Neurocognitive Digital Assessment Tool for Addictive Disorders: Development and Validation Study.

Many people with harmful addictive behaviors may not meet formal diagnostic thresholds for a disorder. A dimensional approach, by contrast, including clinical and community samples, is potentially key to early detection, prevention, and intervention. Importantly, while neurocognitive dysfunction underpins addictive behaviors, established assessment tools for neurocognitive assessment are lengthy and unengaging, difficult to administer at scale, and not suited to clinical or community needs. The BrainPark Assessment of Cognition (BrainPAC) Project sought to develop and validate an engaging and user-friendly digital assessment tool purpose-built to comprehensively assess the main consensus-driven constructs underpinning addictive behaviors. The purpose of this study was to psychometrically validate a gamified battery of consensus-based neurocognitive tasks against standard laboratory paradigms, ascertain test-retest reliability, and determine their sensitivity to addictive behaviors (eg, alcohol use) and other risk factors (eg, trait impulsivity). Gold standard laboratory paradigms were selected to measure key neurocognitive constructs (Balloon Analogue Risk Task [BART], Stop Signal Task [SST], Delay Discounting Task [DDT], Value-Modulated Attentional Capture [VMAC] Task, and Sequential Decision-Making Task [SDT]), as endorsed by an international panel of addiction experts; namely, response selection and inhibition, reward valuation, action selection, reward learning, expectancy and reward prediction error, habit, and compulsivity. Working with game developers, BrainPAC tasks were developed and validated in 3 successive cohorts (total N=600) and a separate test-retest cohort (N=50) via Mechanical Turk using a cross-sectional design. BrainPAC tasks were significantly correlated with the original laboratory paradigms on most metrics (r=0.18-0.63, P<.05). With the exception of the DDT k function and VMAC total points, all other task metrics across the 5 tasks did not differ between the gamified and nongamified versions (P>.05). Out of 5 tasks, 4 demonstrated adequate to excellent test-retest reliability (intraclass correlation coefficient 0.72-0.91, P<.001; except SDT). Gamified metrics were significantly associated with addictive behaviors on behavioral inventories, though largely independent of trait-based scales known to predict addiction risk. A purpose-built battery of digitally gamified tasks is sufficiently valid for the scalable assessment of key neurocognitive processes underpinning addictive behaviors. This validation provides evidence that a novel approach, purported to enhance task engagement, in the assessment of addiction-related neurocognition is feasible and empirically defensible. These findings have significant implications for risk detection and the successful deployment of next-generation assessment tools for substance use or misuse and other mental disorders characterized by neurocognitive anomalies related to motivation and self-regulation. Future development and validation of the BrainPAC tool should consider further enhancing convergence with established measures as well as collecting population-representative data to use clinically as normative comparisons.

The Essential Network (TEN): engagement and mental health insights from a digital mental health assessment tool for Australian health professionals during COVID-19

BackgroundHealth professionals are at risk of poor mental health outcomes due to the COVID-19 pandemic. The Essential Network (TEN) is a blended care mental health support service for Australian health professionals, funded by the Australian Federal Department of Health. TEN comprises both digital and face-to-face components. We examined completed digital mental health assessments (Digital Mental Health Check-Up) to understand usage of TEN and the demographics and mental health of users.MethodsA total of 9889 completed assessments from a community sample of help-seeking health professionals who engaged with the Digital Mental Health Check-Up between May 2020 and December 2021 were examined. Users had the option to complete the Distress Questionnaire (DQ-5), Patient Health Questionnaire (PHQ-9), Generalised Anxiety Disorder (GAD-7), Oldenburg Burnout Inventory (OLBI-16), Work and Social Adjustment Scale (WSAS), Post-traumatic Stress Disorder Checklist (PCL-5), as well as provide demographic information on their gender, age, and profession.ResultsUsers were mostly women (85.7%) aged between 25 and 54 (73.7%). Nursing was the most reported profession (32.6%), followed by allied health (27.5%) and doctors (21.9%). Notably, 8.1% of users reported being in medical administration roles. Mental health measures were poor across all measures and professions and worse than would be expected from the general population. Disengagement (92.5%) from burnout and impaired social and occupational functioning (75%) were also notably high. Use of the DQ-5 as a screening tool in the Digital Mental Health Check-Up was confirmed through correlation with clinically significance scoring on all other measures and increased likelihood of opting to complete other optional measures following clinically significant scoring on the DQ-5.ConclusionsThe present study indicates good usage of the TEN digital components among Australian health professionals, with use across all professional categories. While self-selection bias is inherent in the sample, mental health measures were notably poor – particularly for burnout. Such findings highlight the continued need to provide mental health support to health professionals both during and after the eventual resolution of the COVID-19 pandemic, as well as to develop better support for health professionals experiencing burnout.

Opportunities for the Implementation of a Digital Mental Health Assessment Tool in the United Kingdom: Exploratory Survey Study.

Every year, one-fourth of the people in the United Kingdom experience diagnosable mental health concerns, yet only a proportion receive a timely diagnosis and treatment. With novel developments in digital technologies, the potential to increase access to mental health assessments and triage is promising. This study aimed to investigate the current state of mental health provision in the United Kingdom and understand the utility of, and interest in, digital mental health technologies. A web-based survey was generated using Qualtrics XM. Participants were recruited via social media. Data were explored using descriptive statistics. The majority of the respondents (555/618, 89.8%) had discussed their mental health with a general practitioner. More than three-fourths (503/618, 81.4%) of the respondents had been diagnosed with a mental health disorder, with the most common diagnoses being depression and generalized anxiety disorder. Diagnostic waiting times from first contact with a health care professional varied by diagnosis. Neurodevelopmental disorders (30/56, 54%), bipolar disorder (25/52, 48%), and personality disorders (48/101, 47.5%) had the longest waiting times, with almost half (103/209, 49.3%) of these diagnoses taking >6 months. Participants stated that waiting times resulted in symptoms worsening (262/353, 74.2%), lower quality of life (166/353, 47%), and the necessity to seek emergency care (109/353, 30.9%). Of the 618 participants, 386 (62.5%) stated that they felt that their mental health symptoms were not always taken seriously by their health care provider and 297 (48.1%) were not given any psychoeducational information. The majority of the respondents (416/595, 77.5%) did not have the chance to discuss mental health support and treatment options. Critically, 16.1% (96/595) did not find any treatment or support provided at all helpful, with 63% (48/76) having discontinued treatment with no effective alternatives. Furthermore, 88.3% (545/617) of the respondents) had sought help on the web regarding mental health symptoms, and 44.4% (272/612) had used a web application or smartphone app for their mental health. Psychoeducation (364/596, 61.1%), referral to a health care professional (332/596, 55.7%), and symptom monitoring (314/596, 52.7%) were the most desired app features. Only 6.8% (40/590) of the participants said that they would not be interested in using a mental health assessment app. Respondents were the most interested to receive an overall severity score of their mental health symptoms (441/546, 80.8%) and an indication of whether they should seek mental health support (454/546, 83.2%). Key gaps in current UK mental health care provision are highlighted. Assessment and treatment waiting times together with a lack of information regarding symptoms and treatment options translated into poor care experiences. The participants' responses provide proof-of-concept support for the development of a digital mental health assessment app and valuable recommendations regarding desirable app features.

Comparing the web-based and traditional self-reported 24-hour dietary recall data in the PakNutriStudy

BackgroundThe flaws in dietary assessment methods can generate misleading information and thus may impact on the interventions planned based on that information. Context specific digitalization of dietary assessment tools is a potential way forward to reduce biases and resources involved in data handling. MethodsTwo versions of Twenty-Four Hour Recall (24HR) (traditional [24HR Ver-01] and digital [24HR Ver-02]) were tested for data agreement and feasibility by gathering cross sectional paired data on both the versions from 102 participants (18–25 years age). The web based 24HR was setup using the system of Intake24 (New Castle University) with incorporation of South Asian food data base for beverages. ResultsThe data sets obtained from 24HR Ver-01 and 24HR Ver-02 on beverage consumption (food items as well as portion sizes) were compared for agreement. The highest percentage of agreement of food item reporting between 24HR Ver-01 and 24HR Ver-01 was during the lunch time. The average kappa value (κ =0.375833) for all the meals indicated a fair agreement betweenVer-01 and 24HR Ver-02 The correlation of portion sizes reported using 24HR Ver-01 and 24 HR Ver-02 was statisticallysignificant for morning snack, lunch and dinner (r = 0.465; r = 0.324; r = 0.407 respectively). According to Bland Altman plot, least agreement between the two versions was found in the portion sizes reported for morning snacks. Data collectors found 24 HR Ver-02 easier in terms of data processing but it was regarded time taking and less convenient by the participants. ConclusionThe Intake 24 (digital version of 24HR) can be a preferred tool of data collection as the data collected through it may reach fairly good levels of accuracy. Future directions for research like conducting a follow up study with cross over design, expanding the study using food items other than beverages, and testing the digital dietary assessment tool against an objective gold standard of dietary intake can be helpful in reaching more conclusive evidence.

EXPLORING DIGITAL ASSESSMENT LITERACY IN MALAYSIAN ESL TERTIARY INSTITUTION INSTRUCTORS’ INNOVATIVE PEDAGOGY

This conceptual paper aims to study the digital assessment literacy of ESL instructors in Malaysia. In education, assessment is one of the most crucial components in the teaching and learning process. Assessment allows the instructors to gather information of the students to determine the students’ knowledge and to make improvements based on the data collected. With the digitalisation in education, it has propelled the relevancy of digital assessment. Despite the increasing use of digital assessments in educational settings, many educators lack the necessary digital assessment literacy skills to effectively create, administer, and interpret these assessments. Therefore, this study aims to explore the digital assessment literacy of ESL instructors. This study will use semi-constructed interview and analysis of documents. The type of sampling employed in this study is purposive sampling towards 15 ESL instructors in Malaysian Universities. The data will be analysed using thematic analysis. The expected result from this study is that ESL instructors will have knowledge of digital assessment but have areas of improvement in doing digital assessment. The researchers acknowledge that knowing how to use digital assessment tool is not sufficient, and teachers need to remain aware of how these tools are successful or unsuccessful. It is hoped that the outcome of this research will help the teachers to be knowledgeable in how they understand, conduct, and interpret digital assessments

Digital vs. physical ear-nose-and-throat specialist assessment screening for complicated hearing loss and serious ear disorders in hearing-impaired adults prior to hearing aid treatment: a randomized controlled trial.

This study introduces a digital assessment tool for asynchronous and remote ear-nose-and-throat (ENT) specialist assessment screening for complicated hearing loss and serious ear disorders in hearing-impaired adults prior to hearing aid (HA) treatment. The +60 population will nearly double from 12% to 22% between 2015 and 2050 increasing the incidence of age-induced hearing impairment and the need for hearing rehabilitation. If un-diagnosed, age-related hearing loss negatively affects quality of life by accelerating social distancing and early retirement as well as increasing risk of anxiety, depression, and dementia. Therefore, innovative measures are essential to provide timely diagnostics and treatment. A total of 751 hearing-impaired adults without previous HA usage or experience were randomly assigned to digital or physical ENT specialist assessment screening prior to HA treatment initiation in 20 public and private hearing rehabilitation and ENT specialist clinics in the North Denmark Region. A total of 501 test group participants were assigned to digital assessment screening and 250 control group participants to physical assessment screening prior to HA treatment. In all, 658 (88%) participants completed the trial and were eligible for analysis. Digital screening sensitivity (0.85, 95% confidence interval (CI) 0.71-0.94) was significantly higher than physical screening sensitivity (0.2, 95% CI: 0.03-0.56). Screening specificity was high for both assessment methods. In a setting where hearing-impaired adults were assessed for HA treatment, digital ENT specialist assessment screening did not compromise patient safety or increase the risk of misdiagnosis in patients with complicated hearing loss and/or serious ear disorders when compared to physical ENT specialist assessment screening. https://clinicaltrials.gov/ct2/show/NCT05154539, identifier: NCT05154539.

Development and Evaluation of a Digital HIV Risk Assessment Tool incorporated within an App-Based Self-Testing Program.

Low risk perception is an important barrier to the utilisation of HIV services. In this context, offering an online platform for people to assess their risk of HIV and inform their decision to test can be impactful in increasing testing uptake. Using secondary data from the HIVSmart! quasi-randomised trial, we aimed to identify predictors of HIV, develop a risk staging model for South African township populations, and validate it in combination with the HIVSmart! digital self-testing program. Townships in Cape Town, South Africa. Using Bayesian predictive projection, we identified predictors of HIV and constructed a risk assessment model that we validated in external data. Our analyses included 3095 participants from the HIVSmart! trial. We identified a model of five predictors (being unmarried, HIV testing history, having had sex with a partner living with HIV, dwelling situation, and education) that performed best during external validation (AUC-ROC, 89% CrI: 0.71, 0.68 - 0.72). The sensitivity of our HIV risk staging model was 91.0% (89.1% - 92.7%) and the specificity was 13.2% (8.5% - 19.8%); but increased when combined with a digital HIV self-testing program, the specificity was 91.6% (95.9% - 96.4%) and sensitivity remained similar at 90.9% (89.1% - 92.6%). This is the first validated digital HIV risk assessment tool developed for South African township populations and the first study to evaluate the added value of a risk assessment tool with an App-based HIV self-testing program. Study findings are relevant for application of digital programs to improve utilization of HIV testing services.

OA23 Improving the biopsychosocial assessment and management of patients with fibromyalgia: developing a digital pre-clinic assessment tool

Abstract Background/Aims Fibromyalgia syndrome (FMS) is a common and complex condition with multiple symptoms. Delivering biopsychosocial assessment and tailored management, following EULAR recommendations, is difficult due to constraints on clinic appointments. A previous audit found that only 8% of our patients with FMS were referred to a specialist rehabilitation programme and medication was the primary management recommendation in 65%. We used quality improvement (QI) methods to improve this. Methods A QI project was carried out with three plan, do, study, act (PDSA) cycles over 5 years (2018-2022) with the aim of improving the recording of a quantitative and qualitative assessment of pain, function, and psychosocial impact of symptoms (minimum of 80% of encounters). The proportion of consultations that documented advice on physical activity/sleep/psychological interventions, recommended medication changes, provision of written information and referral to a pain management programme was calculated. Following each cycle, a sample of consultations conducted over a 3-month period were evaluated: Cycle 1 - ‘Moving 4ward’ booklet. An assessment booklet completed by patients and clinicians at the appointment including validated screening tools, information regarding previous treatments, summary of the medical assessment, information about FMS and treatment options leading to a shared action plan. Cycle 2 - Pilot of a digital pre-clinic assessment tool (digitalising the ‘Moving 4ward’ booklet). Link sent to patients using Microsoft Forms. Results accessed by the clinician via an Excel spreadsheet during the appointment. Cycle 3 - Complete digitalisation of the pre-clinic assessment tool. Automated interpretation of results and production of a report uploaded to patient notes using an inhouse rheumatology database, RhADIO. The report includes results of screening tools, patient history as well as targeted signposting and management advice to the clinician and patient. Results There was an improvement in the recording of quantitative and qualitative assessment of patients with FMS across all domains (pain, function, psychological) following each cycle. In the final cycle, 100% of patients diagnosed with FMS had quantitative measures recorded on their notes and greater than 80% had qualitative assessments recorded for pain and function. The interventions resulted in an improvement across each of the recorded management approaches in adherence to EULAR recommendations: 95% of patients were referred to a multidisciplinary pain rehabilitation service in 2022 compared to 8% at baseline. More patients were given advice on physical activity and sleep and provided written information regarding their diagnosis. Conclusion The project team has been able to improve the assessment and management of patients with FMS based on EULAR recommendations. Appointments are more targeted and tailored to the needs of patients based on information provided via the digital pre-clinic tool. Future work will consider how this digital tool may support all clinicians across the integrated care system in primary care, intermediate services and secondary care. Disclosure A. Saif: None. A. Wall: None. Y. Kang: None. J. Nicholas: None. R. Luqmani: None. A. Soni: None.

Comparison of tristimulus spectrocolorimeter measurements and human visual assessment of South Carolina field moist Munsell soil colors

AbstractSoil color has historically been estimated visually, but inaccuracies with this method have been documented, including disagreement in evaluators, color blindness, and differences in physical color books due to weathering. As new technologies become available to soil scientists, a reliable digital method to determine soil color in the field could offer a solution to these inaccuracies. This research aimed to assess if visual estimates of soil matrix color from 111 soil horizons differed from digital measurements of Munsell soil color across three South Carolina regions. Visual estimates of soil color were conducted using Munsell color books. The XRite Capsure gave digital measurements in Munsell notation specific for soils. The absolute difference between hues was similar among assessment methods. Visual estimates of color value and chroma were statistically greater than digital measurements. However, color values and chromas were less than one chip different, suggesting no practical difference between the two assessment methods. Utilizing a digital soil color assessment tool can assist with training and validate human estimates.

Psychometric evaluation of the electronic faces thermometer scale for pain assessment in children 8-17 years old: A study protocol.

It is often a challenge for a child to communicate their pain, and their possibilities to do so should be strengthened in healthcare settings. Digital self-assessment provides a potential solution for person-centered care in pain management and promotes child participation when a child is ill. A child's perception of pain assessment differs when it is assessed using digital or analog formats. As we move into the digital era, there is an urgent need to validate digital pain assessment tools, including the newly developed electronic Faces Thermometer Scale (eFTS). This study protocol describes three studies with the overall aim to evaluate psychometric properties of the eFTS for assessing pain in children 8-17 years of age. A multi-site project design combining quantitative and qualitative methods will be used for three observational studies. Study 1: 100 Swedish-speaking children will report the level of anticipated pain from vignettes describing painful situations in four levels of pain and a think-aloud method will be used for data collection. Data will be analyzed with phenomenography as well as descriptive and comparative statistics. Study 2: 600 children aged 8-17 years at pediatric and dental settings in Sweden, Denmark, Iceland, and USA will be included. Children will assess their pain intensity due to medical or dental procedures, surgery, or acute pain using three different pain Scales for each time point; the eFTS, the Faces Pain Scale Revised, and the Coloured Analogue Scale. Descriptive and comparative statistics will be used, with subanalysis taking cultural context into consideration. Study 3: A subgroup of 20 children out of these 600 children will be purposely included in an interview to describe experiences of grading their own pain using the eFTS. Qualitative data will be analyzed with content analysis. Our pilot studies showed high level of adherence to the study procedure and rendered only a small revision of background questionnaires. Preliminary analysis indicated that the instruments are adequate to be used by children and that the analysis plan is feasible. A digital pain assessment tool contributes to an increase in pain assessment in pediatric care. The Medical Research Council framework for complex interventions in healthcare supports a thorough development of a new scale. By evaluating psychometric properties in several settings by both qualitative and quantitative methods, the eFTS will become a well-validated tool to strengthen the child's voice within healthcare.