Support Data Use Research Articles

On the use of RWD in support of regulatory submission in drug development

ABSTRACT For the approval of a drug product, the United States Food and Drug Administration requires substantial evidence (SE) regarding effectiveness and safety of the test drug to be provided. In recent years, the use of real-world data in support of regulatory submission of pharmaceutical development has received much attention, and real-world evidence (RWE) is treated as complementary to SE by evaluating the real-world performance of the test treatment. In this article, we start by summarizing current regulatory perspectives on drug evaluation and some potential challenges in using RWE. To test for superiority in co-primary endpoints, a two-stage hybrid RCT/RWS adaptive design that combines randomized control trial for providing SE and real-world study for generating RWE is proposed. We use superiority in effectiveness and non-inferiority in safety as an example to illustrate how to implement this design. Numerical studies have shown that the proposed design has merits in reducing the required sample size compared with traditional co-primary endpoint tests while maintaining statistical power and controlling type I error inflation. The proposed design can be implemented in drug development considering co-primary endpoints, especially for oncology and rare disease drug development.

Introducing Elinor for monitoring the governance and management of area-based conservation.

Monitoring the governance and management effectiveness of area-based conservation has long been recognized as an important foundation for achieving national and global biodiversity goals and enabling adaptive management. However, there are still many barriers that prevent conservation actors, including those affected by governance and management systems from implementing conservation activities and programs and from gathering and using data on governance and management to inform decision-making across spatial scales and through time. We explored current and past efforts to assess governance and management effectiveness and barriers actors face in using the resulting data and insights to inform conservation decision-making. To help overcome these barriers, we developed Elinor, a free and open-source monitoring tool that builds on the work of Nobel Prize winner Elinor Ostrom to facilitate the gathering, storing, sharing, analyzing, and use of data on environmental governance and management across spatial scales and for areas under different governance and management types. We consider the process of codesigning and piloting Elinor with conservation scientists and practitioners and the main components of the assessment and online data system. We also consider how Elinor complements existing approaches by addressing governance and management in a single assessment at a high level for different types of area-based conservation, providing flexible options for data collection, and integrating a data system with an assessment that can support data use and sharing across different spatial scales, including global monitoring of the Global Biodiversity Framework. Although challenges will continue, the process of developing Elinor and the tool itself offer tangible solutions to barriers that prevent the systematic collection and use of governance and management data. With broader uptake, Elinor can play a valuable role in enabling more effective, inclusive, and durable area-based conservation.

Strengthening country-led water and sanitation services monitoring and data use for decision-making: lessons from WaterAid experience in four countries

Abstract Water, sanitation, and hygiene (WASH) are fundamental human rights, of critical importance to health, education, wellbeing, and economic prosperity. To fulfil these human rights and drive progress towards universal and sustainable access to WASH services, government service-level monitoring processes and data use are vital for effective decision-making and accountability. Despite increasing sector efforts to improve WASH data access, there is limited evidence of this translating into effective data use to inform effective planning for equitable access and budgeting and of the factors affecting this. Four case studies where WaterAid has worked with national government and sector stakeholders to strengthen WASH monitoring processes in Uganda, Cambodia, Papua New Guinea (PNG), and Myanmar were analysed through an analytical framework to understand the impact of different factors and related system-strengthening activities towards outcomes of increased data coordination, timely and relevant data availability and data use to inform decision-making in WASH service delivery. The analysis highlighted that strengthening activities aiming at improving indicators, data collection and analysis, and the type of data collection and visualisation technology have a direct impact on improving WASH sector coordination and timely data availability. However, to ensure strengthening activities support data use for decision-making, they need to be developed from within and adapt to the on-going wider political economy systems evolution, including formal processes such as decentralisation and evolving informal political drivers.

Analysis of a Web-Based Dashboard to Support the Use of National Audit Data in Quality Improvement: Realist Evaluation.

BackgroundDashboards can support data-driven quality improvements in health care. They visualize data in ways intended to ease cognitive load and support data comprehension, but how they are best integrated into working practices needs further investigation.ObjectiveThis paper reports the findings of a realist evaluation of a web-based quality dashboard (QualDash) developed to support the use of national audit data in quality improvement.MethodsQualDash was co-designed with data users and installed in 8 clinical services (3 pediatric intensive care units and 5 cardiology services) across 5 health care organizations (sites A-E) in England between July and December 2019. Champions were identified to support adoption. Data to evaluate QualDash were collected between July 2019 and August 2021 and consisted of 148.5 hours of observations including hospital wards and clinical governance meetings, log files that captured the extent of use of QualDash over 12 months, and a questionnaire designed to assess the dashboard’s perceived usefulness and ease of use. Guided by the principles of realist evaluation, data were analyzed to understand how, why, and in what circumstances QualDash supported the use of national audit data in quality improvement.ResultsThe observations revealed that variation across sites in the amount and type of resources available to support data use, alongside staff interactions with QualDash, shaped its use and impact. Sites resourced with skilled audit support staff and established reporting systems (sites A and C) continued to use existing processes to report data. A number of constraints influenced use of QualDash in these sites including that some dashboard metrics were not configured in line with user expectations and staff were not fully aware how QualDash could be used to facilitate their work. In less well-resourced services, QualDash automated parts of their reporting process, streamlining the work of audit support staff (site B), and, in some cases, highlighted issues with data completeness that the service worked to address (site E). Questionnaire responses received from 23 participants indicated that QualDash was perceived as useful and easy to use despite its variable use in practice.ConclusionsWeb-based dashboards have the potential to support data-driven improvement, providing access to visualizations that can help users address key questions about care quality. Findings from this study point to ways in which dashboard design might be improved to optimize use and impact in different contexts; this includes using data meaningful to stakeholders in the co-design process and actively engaging staff knowledgeable about current data use and routines in the scrutiny of the dashboard metrics and functions. In addition, consideration should be given to the processes of data collection and upload that underpin the quality of the data visualized and consequently its potential to stimulate quality improvement.International Registered Report Identifier (IRRID)RR2-10.1136/bmjopen-2019-033208

Job Analyses of Earth Science Data Librarians and Data Managers

AbstractThis study’s purpose is to capture the skills of Earth science data managers and librarians through interviews with current job holders. Job analysis interviews were conducted of 14 participants—six librarians and eight data managers—to assess the types and frequencies of job tasks. Participants identified tasks related to communication, including collaboration, teaching, and project management activities. Data-specific tasks included data discovery, processing, and curation, which require an understanding of the data, technology, and information infrastructures to support data use, reuse, and preservation. Most respondents had formal science education and six had a master’s degree in Library and Information Sciences. Most of the knowledge, skills, and abilities for these workers were acquired through on-the-job experience, but future professionals in these careers may benefit from tailored education informed through job analyses.

Data use processes in rural schools: management structures undermining leadership opportunities and instructional change

ABSTRACT Data use has become a priority in educational systems throughout the world under the belief that rational instructional decisions can be tailored to individual learner needs. Despite increasing expectations for school principals to be instructional leaders, there is little evidence that they – or other system or school leaders – are responsible for anything more than ensuring structures are in place for teachers to work with data. In this study, we analyze interview and observational data collected over the period of 1 academic year in four elementary schools in one rural school district in the United States. We consider results through a conceptual framing of collective leadership to understand how leaders across district, school, and classroom levels do or do not support data use in the school system. Among our findings, data use is espoused and portrayed but generally unsupported. Data team meetings and structures are embedded in school cultures, but they are mostly managed and routine, prioritizing expediency and process over instructional adaptation or response. As a result, we conclude that the establishment of data team meetings and related structures is critical but insufficient to improve instruction and increase student learning.

Herpetological Assemblages in Tropical Forests of the Taguibo Watershed, Butuan City, Eastern Mindanao, Philippines

Tropical watershed ecosystems support heterogeneous habitats and diverse non-human species assemblages, together providing ecosystem services to humans. Amphibians and reptiles are recognized as sensitive indicators of ecosystem “health,” related to beneficial services (provisional, regulating, cultural, structural, functional) human societies receive from terrestrial watersheds. The Taguibo Watershed supplies fresh drinking water to Butuan City in the Caraga Region of northeast Mindanao Island. However, very little is known about the herpetofauna of the area. Here, we synthesize biodiversity data from historical (1971, 1979) and recent (2013, 2017) herpetological surveys from the region. We utilize specimen-associated occurrence records and natural history information to produce a species inventory, analyze their habitat utilization, and characterize diversity metrics to describe herpetological communities of the watershed – resulting in 44 species (27 new records). A number of historically-documented species persist, having partitioned riparian and terrestrial habitat types in dipterocarp and secondary-growth forests of Taguibo. Reptiles exhibit little overlap in the use of microhabitats – in contrast to amphibians, which exhibit either unique or frequently shared microhabitat substrates. In terrestrial microhabitats (not immediately associated with water), many newly-recorded reptiles and amphibians (particularly, of the genus Platymantis) partition space predictably – either occupying a single microhabitat or, in one species (a pit viper, Tropidolaemus subannulatus), two microhabitats. We anticipate that our initial characterization of Taguibo’s herpetofauna may serve as a baseline to promote further research and facilitate conservation initiatives. We emphasize the importance of primary sources – field-based surveys and re-surveys – and open-access biodiversity data served via online platforms that provide live, transparent access to original, unaltered data. Anthropogenic threats involving economic-driven activities present a need for field-based research in support of watershed management. Periodic, survey, and re-survey studies – continuously updating earlier work – are the most reliable, repeatable, and publicly-transparent use of biodiversity survey data in support of societal benefits and ecosystem services.

A design rationale to support data use in successful teacher practice: some considerations for educational technology design

Despite the growing ubiquity of educational technologies in K-12 classrooms, many scholars have pointed out that impact of these technologies for all students has been uneven (Gomez et al., 2010). Some attribute this to a lack of cumulativity in educational research, in general. In this paper, we present design rationale as a mechanism to support designers building upon others' designs and assumptions of designs. Reporting on design-based research study to support teachers' data-informed instruction, we highlight our own process connecting our decisions and outcomes to both the conditions on the ground as well as the current research literature.

A design rationale to support data use in successful teacher practice: some considerations for educational technology design

A design rationale to support data use in successful teacher practice: some considerations for educational technology design

Navigating Institutional Culture in Ensuring the Ethical Use of Data in Support of Student Success and Institutional Effectiveness

AbstractIn this chapter, we explore the contexts within which institutional researchers must try to ensure the ethical use of data. We also identify challenges institutional research professionals can face when navigating college culture while trying to ensure the ethical use of data. Finally, we identify potential solutions for improving institutional data culture.

How school leaders can build effective data teams: Five building blocks for a new wave of data-informed decision making

Data-informed decision making is considered important for school improvement. Working in data teams is a promising strategy for implementing data use in schools. Data teams consist of teachers and school leaders, who collaboratively analyze data to solve educational problems at their school. Studies show that school leaders can enable and hinder data use in such teams. This study aims at exploring what types of leadership behaviors are applied to support data use in data teams. The results of this study point to five key building blocks for school leaders wanting to build effective data teams in their school: (1) establishing a vision, norms, and goals (e.g., discussing vision, norms, and goals with teachers); (2) providing individualized support (e.g., providing emotional support); (3) intellectual stimulation (e.g., sharing knowledge and providing autonomy); (4) creating a climate for data use (e.g., creating a safe climate focused on improvement rather than accountability, and engaging in data discussions with teachers); and (5) networking to connect different parts of the school organization (e.g., brokering knowledge and creating a network that is committed to data use). Not only formal school leaders, but also teachers, can display these types of behavior. Finally, it is important to stress here that all these building blocks are needed to create sustainable data use practices. These building blocks can be used in a new wave of data-informed decision making in schools, in which teachers and school leaders collaboratively use a multitude of different data sources to improve education.

Supporting Collaboratively Constructed Independence

Independence is a central concern for people in the care of many chronic conditions. It is often viewed as a goal that can be facilitated with the use of patient data. It is also viewed, especially from the medical side, as something an individual achieves. However, in the lived experience, independence is often a set of collaborative practices. In this paper, we unpack findings from an interview-based study of Spinal Cord Injury (SCI) self-care. We found that independence - both functional and in the form of effecting agency - must be co-constructed by the choices and activities of the care network, including the person with disability, caregivers, and clinicians. This collaboratively shaped independence also affects potential collection and use of data in support of self-care. We describe how collaboratively shaped independence informs requirements and constraints for the design of sensor-based networks for self-care in long-term chronic disability.

Practical considerations for transitioning early childhood interventions to scale: lessons from the Saving Brains portfolio.

Small pilot studies of young children have frequently shown promise, but very few have been successfully scaled to the regional or national levels. How can we ensure that these promising approaches move from a suite of pilots to full-scale implementation that can deliver sustainable impact for hundreds of millions of children? To elucidate concrete lessons learned and suggestions on accelerating the transition to impact at scale, we reviewed the Saving Brains portfolio to better understand three points: (1) the extent to which useful signals of impact could be extracted from data at the seed phase, (2) the ways in which innovators (project leaders) were approaching human resource challenges critical for scaling, and (3) the multisector diversity of the portfolio and the way innovators entered partnerships. The findings suggest key considerations for transitioning early childhood development interventions to scale and sustainability: strong entrepreneurial leadership, rigorous measurement and active use of data in support of adaptive learning, and champions acting at subnational levels. Together, these can enable flexible, iterative learning that can make the scaling process an opportunity to increase the level of benefit each child receives from an intervention.

Baseline Practices for the Application of Genomic Data Supporting Regulatory Food Safety

The application of new data streams generated from next-generation sequencing (NGS) has been demonstrated for food microbiology, pathogen identification, and illness outbreak detection. The establishment of best practices for data integrity, reproducibility, and traceability will ensure reliable, auditable, and transparent processes underlying food microbiology risk management decisions. We outline general principles to guide the use of NGS data in support of microbiological food safety. Regulatory authorities across intra- and international jurisdictions can leverage this effort to promote the reliability, consistency, and transparency of processes used in the derivation of genomic information for regulatory food safety purposes, and to facilitate interactions and the transfer of information in the interest of public health.

Unpacking instructional alignment: The influence of teachers’ use of assessment data on instruction

The use of assessment data to inform instruction is an important component of a comprehensive standards-based assessment programme. Examining teachers’ data use for instruction can reveal the extent to which instruction is aligned with established content standards and assessment. This paper describes results of a qualitative study of teachers’ data use in a mid-Atlantic metropolitan area in the United States. Focus group interviews with 60 upper elementary and middle school teachers from 45 schools were conducted. Findings indicate teachers aligned instruction and assessments with the state curriculum with the goal of improving student performance. While teachers found day-to-day informal assessments essential to shaping instruction, periodic formal assessments helped them monitor student progress and remediation efforts. Teachers described challenges associated with the misalignment of periodic assessments with instructional content, the breadth of content and higher cognitive demand expected in the newer state curriculum and the lack of infrastructure to support data use.

Measuring what we manage – the importance of hydrological data to water resources management

Abstract. Water resources cannot be managed, unless we know where they are, in what quantity and quality, and how variable they are likely to be in the foreseeable future. Data from hydrological networks are used by public and private sectors for a variety of different applications. This paper discusses the value proposition behind the collection, analysis and use of hydrological data in support of these applications. The need for hydrological data and the requirements for the data are outlined, and information is provided on topics such as status of networks and data access and sharing. This paper outlines elements of the contribution by the World Meteorological Organization (WMO) to hydrological data collection and covers aspects related to quality management in the collection of hydrological data, especially regarding streamflow gauging, network design and capacity building for services delivery. It should be noted that the applications which make use of hydrological data may also be significantly impacted by climate change.

Moving data off the shelf and into action: an intervention to improve data-informed decision making in Côte d'Ivoire

Background Improving a health system requires data, but too often they are unused or under-used by decision makers. Without interventions to improve the use of data in decision making, health systems cannot meet the needs of the populations they serve. In 2008, in Côte d'Ivoire, data were largely unused in health decision-making processes. Objective To implement and evaluate an intervention to improve the use of data in decision making in Cote d'Ivoire. Design From 2008 to 2012, Cote d'Ivoire sought to improve the use of national health data through an intervention that broadens participation in and builds links between data collection and decision-making processes; identifies information needs; improves data quality; builds capacity to analyze, synthesize, and interpret data; and develops policies to support data use. To assess the results, a Performance of Routine Information System Management Assessment was conducted before and after the intervention using a combination of purposeful and random sampling. In 2008, the sample consisted of the central level, 12 districts, and 119 facilities, and in 2012, the sample consisted of the central level, 20 districts, and 190 health facilities. To assess data use, we developed dichotomous indicators: discussions of analysis findings, decisions taken based on the analysis, and decisions referred to upper management for action. We aggregated the indicators to generate a composite, continuous index of data use. Results From 2008 to 2012, the district data-use score increased from 40 to 70%; the facility score remained the same – 38%. The central score is not reported, because of a methodological difference in the two assessments. Conclusions The intervention improved the use of data in decision making at the district level in Côte d'Ivoire. This study provides an example of, and guidance for, implementing a large-scale intervention to improve data-informed decision making.

How leaders can support teachers with data-driven decision making

As accountability systems have increased demands for evidence of student learning, the use of data in education has become more prevalent in many countries. Although school and administrative leaders are recognizing the need to provide support to teachers on how to interpret and respond to data, there is little theoretically sound research on data-driven decision making (DDDM) to guide their efforts. Drawing on sociocultural learning theory, extant empirical literature, and findings from a recent study, this paper develops a framework for understanding how to build teacher capacity to use data, specifically informing what practices administrators might employ, when in the DDDM process to employ these practices, and how these mechanisms may build teacher knowledge and skills. Given the global economic climate, administrators face difficult choices in how to invest scarce resources to support data use and once invested, how to ensure that teachers gain, and sustain, the needed capabilities once the supports are removed. The framework provided herein presents a set of concepts that may be useful in guiding these decisions. Implications for leadership practice, as well as suggestions to guide future research and theory development, are discussed.

Assessing the ability of health information systems in hospitals to support evidence-informed decisions in Kenya

BackgroundHospital management information systems (HMIS) is a key component of national health information systems (HIS), and actions required of hospital management to support information generation in Kenya are articulated in specific policy documents. We conducted an evaluation of core functions of data generation and reporting within hospitals in Kenya to facilitate interpretation of national reports and to provide guidance on key areas requiring improvement to support data use in decision making.DesignThe survey was a cross-sectional, cluster sample study conducted in 22 hospitals in Kenya. The statistical analysis was descriptive with adjustment for clustering.ResultsMost of the HMIS departments complied with formal guidance to develop departmental plans. However, only a few (3/22) had carried out a data quality audit in the 12 months prior to the survey. On average 3% (range 1–8%) of the total hospital income was allocated to the HMIS departments. About half of the records officer positions were filled and about half (13/22) of hospitals had implemented some form of electronic health record largely focused on improving patient billing and not linked to the district HIS. Completeness of manual patient registers varied, being 90% (95% CI 80.1–99.3%), 75.8% (95% CI 68.7–82.8%), and 58% (95% CI 50.4–65.1%) in maternal child health clinic, maternity, and pediatric wards, respectively. Vital events notification rates were low with 25.7, 42.6, and 71.3% of neonatal deaths, infant deaths, and live births recorded, respectively. Routine hospital reports suggested slight over-reporting of live births and under-reporting of fresh stillbirths and neonatal deaths.ConclusionsStudy findings indicate that the HMIS does not deliver quality data. Significant constraints exist in data quality assurance, supervisory support, data infrastructure in respect to information and communications technology application, human resources, financial resources, and integration.

School-level organizational routines for learning: supporting data use

Purpose– The purpose of this paper is to use an extreme case to identify and describe the nature of routines that might support processes and outcomes of data use, drawing from a framework developed by Coburn and Turner (2012a).Design/methodology/approach– The author conducted a four-month case study (Stake, 1995) of an elementary school in a large urban school district that had implemented balanced score cards. The author identified a school that had strong qualities to support data use, including leadership and information systems.Findings– Two school-level organizational routines facilitated teachers’ data use: collaborative teams and processes of inquiry. These routines stored knowledge about the types of data teachers ought to notice, and to a lesser extent, how they ought to interpret data and construct implications for practice. These routines also provided opportunities for single and double-loop learning (Argyris and Schön, 1996) and might contribute to improvements in student learning. This case provides an example of how a school negotiated external performance management pressures, and maintained their professional autonomy, focussing on internally initiated assessments.Originality/value– Relatively little research has described what organizational routines support data use among practitioners. In addition to describing two routines, this case also demonstrated the need to frame these routines as organizational routines for learning. To further develop these routines, the author drew on the notion of the knowledge-creating company (Nonaka and Takeuchi, 1995) to explain how the school used their organizational routines to share tacit knowledge (socialization), and to convert tacit knowledge to explicit knowledge (externalization), which supported instructional innovations.