Research Funding Decisions Research Articles

Practical Incorporation of Stakeholder-Informed Ethics into Research Funding Decisions

Abstract: Research funding has been criticized as biased against novel initiatives and lacking diversity, which leads to further disparities. Patient and stakeholder engagement could support research that goes beyond traditional paradigms and increases diversity. However, best practices to engage stakeholders in research, including funding decisions, continue to be developed. We report on the implementation of stakeholder input in two federally funded initiatives, one that seeks to advance research reducing disparities, and the other seeks to advance deprescribing research. Overall, the review process includes stakeholders as decision makers and supports their efforts through group discussion and other activities. Reconciling stakeholder input that may differ from scientific peer review is a challenge within the decision for funding. Lessons learned include balancing stakeholder and scientific assessments and including guidance on stakeholder engagement to grant awardees.

Choosing Wisely: Applying Value-Based Economic Principles to Population Science Research Investment.

Scientific research requires a substantial investment of time, effort, and money by researchers and funders. The funding that would be needed for all meritorious proposals far exceeds available resources. Major funding organizations use a multistep process for allocating research dollars that follows and extends beyond scientific peer review with considerations including mission priority, budget, and potential duplication of past or ongoing research activities. At the level of programmatic review, the process tends to be less proscribed than scientific review, but considerations relate to and are akin to basic value-driven economic principles. We propose a framework that encompasses the elements of programmatic review and provide examples of how the economic principles of opportunity costs, diminishing marginal productivity, sunk costs, economic optimization, return on investment, and option value apply to both research planning and funding decisions. Examples use cancer control population science research, as the nature of observational and interventional research involves large population studies (large sample size, recruitment, and often long-duration follow-up costs) which demand a high level of resource utilization; the same principles can be applied throughout medical and population health research. Awareness of the aspects of programmatic review and context to focus discussion regarding funding decisions may help guide research planning, decision-making, and increase transparency of the overall review process.

Responding To: In Defense of Merit in Science

This commentary responds to a perspective article by a group of 27 co-authors, most affiliated to universities in the USA, with a sprinkling of international others: two each from France and Germany; one each from the UK, Australia and New Zealand; and three listing dual-country affiliations to USA as well as Israel, Germany and Australia. The perspective article claims that science is in serious danger from the replacement of merit by identity as the basis for assessments in science education, hiring of scientists, and research funding decisions. Although stopping just short of saying this ‘is’ happening, its purpose is to issue a warning that the future of science is under threat, if such trends were to continue. Given its relevance to science and science education in Aotearoa New Zealand, it is important for us as Māori and non-Māori to understand the claims and arguments it makes. Below, I list and discuss its key ideas, summarise its scholarly flaws, and comment on its meta-level significance in context.

Evaluating the dependability of reference-driven citation forecasts amid the COVID-19 pandemic: A bibliometric analysis across diverse journals.

The journal impact factor significantly influences research publishing and funding decisions. With the surge in research due to COVID-19, this study investigates whether references remain reliable citation predictors during this period. Four multidisciplinary journals (PLoS One, Medicine [Baltimore], J. Formos. Med. Assoc., and Eur. J. Med. Res.) were analyzed using the Web of Science database for 2020 to 2022 publications. The study employed descriptive, predictive, and diagnostic analytics, with tools such as 4-quadrant radar plots, univariate regressions, and country-based collaborative maps via the follower-leading cluster algorithm. Six countries dominated the top 20 affiliations: China, Japan, South Korea, Taiwan, Germany, and Brazil. References remained strong citation indicators during the COVID-19 period, except for Eur. J. Med. Res. due to its smaller sample size (n = 492) than other counterparts (i.e., 41,181, 12,793, and 1464). Three journals showed higher network density coefficients, suggesting a potential foundation for reference-based citation predictions. Despite variations among journals, references effectively predict article citations during the COVID-19 era, underlining the importance of network density. Future studies should delve deeper into the correlation between network density and citation prediction.

Sea Grant Research Funding: Advancing the Scientific Discourse by Addressing Local Research Priorities

Scientific research is often supported by public funds for the public good, and there is a trend toward using public funds to support actionable science that can be integrated into policy or practice. Funding agencies can set research priorities through the focus and timing of their research calls and funding decisions. There is a need to study funding models to better understand the relationship between agency priorities and scientific discourse. We present a case study of the National Sea Grant College Program’s research funding. Sea Grant is a federally funded network of 34 state- or territory-​based programs that fund research and perform education and outreach work on coastal environmental and natural resource issues. Individual Sea Grant programs set research priorities based on local needs, often focusing on actionable science. To assess the contributions of Sea Grant-funded research, we analyzed available citations and abstracts from Web of Science and examined patterns in article authors, journals, citations, keywords, and abstracts. The study shows that Sea Grant has significantly contributed to the scientific literature while being effective on cost-per-​publication and cost-per-citation bases. The results illustrated how targeted funding can address important local issues while substantially contributing to the broader scientific literature.

Piloting the use of RAiD in Research.fi

Research.fi is a national service for collecting, integrating and disseminating information on research conducted in Finland. This includes information on publications and related research outputs (including in the arts), datasets, funding decisions and research infrastructures. It has a specific section for researcher profiles, focusing on researcher descriptions, their research activities, merits, education - moving towards an affiliation-independent research CV. The plan is that within the Research.fi service, projects will become entities that bind different research objects together, even when funded from multiple sources and involving various researchers and organizations that may change over time. This concept of a project is aligned with the definition of a project in RAiD – the Research Activity Identifier. The RAiD service, developed originally by the Australian Research Data Commons, is being rolled out and adapted to the European context. RAiD provides a persistent and globally unique identifier that resolves in a browser to standardized information about a particular research project. Minting persistent identifiers (PIDs) for research projects facilitates the management of information about related participants, organizations, resources, and outputs. RAiD does so primarily by collecting other PIDs (where possible) representing project components, plus descriptive information about the project, storing it all in a metadata record associated with the identifier. This paper documents and discusses the implementation of RAiD in Europe and piloting its use in the national Finnish service context.

Piloting the use of PIDGraph data in Research.fi

Research.fi is a national service for collecting, integrating and disseminating information on research conducted in Finland. This includes information on publications and related research outputs (including in the arts), datasets, funding decisions and research infrastructures. Many other countries and regions also work on the development of CRISes. Sometimes features are developed before there is enough data to take full advantage of them. Sometimes you just wait for a critical mass of data to be available before starting as feature development. In this case it is a question of information on the various connections between different research objects. Research.fi has been somewhere in between, with 4 years since its initial release waiting for relationship data to come. You build it, and the data will come - has been a long wait. But PIDGraph to the rescue! This paper describes to piloting of PIDGraph scientific knowledge graph data in research.fi - to fill in the missing links. The PIDGraph is a collection of metadata about research objects, authors, publishers, institutions, and other research related items, all identified by persistent identifiers, and a series of systems to expose that metadata and make it available for analysis and integration with other systems.

Setting research priorities for English as an Additional Language

Abstract We report a priority setting partnership (PSP), which sought to establish the priorities for new research among research users (educators, parents, and learners) in the field of English as an Additional Language (EAL). A steering committee consisting of members of these research user groups was established to oversee the project. An online ‘uncertainty questionnaire’ was distributed to research users inviting them to identify areas of their practice about which they are uncertain. These ‘raw’ uncertainties were consolidated and converted into a list of research questions, which was then distributed to the same groups to rank in order of priority. The 25 highest ranked questions were then discussed and debated in a workshop, consisting of research user representatives, to collectively and democratically produce a final Top 10 list of questions. The Top 10 shows what EAL research users regard as priorities, which we hope will inform research agendas and funding decisions.

Discovering latent topics and trends in autonomous vehicle-related research: A structural topic modelling approach

Autonomous Vehicle (AV) technology is a disruptive transportation technology that promises to revolutionize how people travel. Due to their potential mobility benefits and associated impacts on public health and the environment, they have received massive attention from transportation experts. To date, there have been numerous academic research conducted regarding AVs, and many attempts have been made to summarize these research documents by manually reviewing them. Nevertheless, due to the vast nature of the existing literature, it is challenging to synthesize the themes in AV-related research and to explore their trends to inform future research direction. This study aims to utilize an advanced natural language processing technique to provide a comprehensive overview of the recent developments in AV-related research. The abstracts of 3292 articles published in transportation-based journals from January 2016 to October 2022 were collected and used for this study. A Structural Topic Model (STM) was employed to explore the dominant research themes hidden in the extant literature on AV research, examine their evolution over time, and determine the topics highly associated with developing and developed economies. Overall, the study highlighted that the least common themes in the literature are in the areas of the development of complex vehicle designs, safety, environmental benefits, and the impact of AVs on transportation infrastructure. Nevertheless, themes concerning AV safety and the environment are emerging as hot topics since late 2021. Topics related to investigating user perceptions, policies/approaches to increase user demand, and AV control/stability are highly studied. Besides, they predominantly originated from developed countries. The research themes, their associated trends, and the key policy suggestions drawn from the study are potentially useful for making well-informed research funding decisions, setting research priorities for countries, and guiding the future research endeavours of researchers.

A portfolio analysis of autism research funding in Aotearoa New Zealand 2007-2021.

We aimed to document the areas of autism research that have previously been funded in Aotearoa New Zealand. We searched for research grants awarded to autism research in Aotearoa New Zealand between 2007 and 2021. We compared the funding distribution in Aotearoa New Zealand to other countries. We asked people from the autistic community and broader autism community whether they were satisfied with this funding pattern, and whether it aligned with what is important to them and to autistic people. We found that the majority of funding for autism research was awarded to biology research (67%). Members of the autistic and autism communities were dissatisfied with the funding distribution, and expressed a lack of alignment with what is important to them. People from the community indicated that the funding distribution did not address the priorities of autistic people, and that it indicated a lack of engagement with autistic people. Autism research funding needs to reflect the priorities of the autistic and autism communities. Autistic people need to be included in autism research and related funding decisions.

PROTOCOL: Arts-based interventions for offenders in secure criminal justice settings to improve rehabilitation outcomes: An evidence and gap map.

This is the protocol for a Campbell review.The objective of this evidence and gap map is to presents the existing research on the impact of arts-based interventions in secure criminal justice settings (SCJS) that aim to improve desistance outcomes for offenders. It will indicate the quality of available evidence, highlighting the gaps and informing future research priorities. Importantly, it will also identify where the evidence could be systematically reviewed. This would clearly produce a more comprehensive understanding of the available knowledge and an opportunity to move forward in a more direct and focussed way, with the potential to influence research, intervention development, and inform funding decisions.

The Risk of Caution: Evidence from an Experiment

Innovation is important for firm performance and broader economic growth. However, breakthrough innovations necessarily require greater risk taking than more incremental approaches. To understand how managers respond to uncertainty when making research and development decisions, we conducted experiments with master’s degree students in a program focused on the intersection of business and technology. Study participants were asked to choose whether to fund hypothetical research projects using a process that mirrors real-world research and development funding decisions. The experiments provided financial rewards that disproportionately encouraged the choice of higher-risk projects. Despite these incentives, most participants chose lower-risk projects at the expense of projects more likely to generate a large payoff. Heterogeneity analysis and additional experimental treatments show that individual risk preferences predict greater tolerance of high-risk projects and suggest that more appropriate decision making can be learned. Thus, for firms seeking to fund breakthrough research and development, appropriate screening and training of employees may play important roles in increasing the likelihood of success. This paper was accepted by Gustavo Manso, finance. Funding: This work was supported by the National Science Foundation [Grant SBE-1561257]. Supplemental Material: The data files and online appendix are available at https://doi.org/10.1287/mnsc.2021.4292 .

The Future of Disability Research in Australia: Protocol for a Multiphase Research Agenda–Setting Study

BackgroundFor people with disabilities to live a good life, it is essential that funded research in health and social care addresses their interests, meets their needs, and fills gaps in our understanding of the impact that services, systems, and policies may have on them. Decisions about research funding should be based on an understanding of the research priorities of people with disabilities, their supporters and allies, disability researchers, service providers, and policy makers working in the field.ObjectiveThe aim of this protocol is to describe the research design and methods of a large-scale, disability research agenda–setting exercise conducted in 2021 in Australia.MethodsThe research agenda–setting exercise involves 3 integrated phases of work. In the first phase, a previous audit of disability research in Australia is updated to understand previous research and continuing gaps in the research. Building on this, the second phase involves consultation with stakeholders—people with disabilities and their supporters and family members, the disability workforce, and people working within services and connected sectors (eg, aging, employment, education, and housing), academia, and public policy. Data for the second phase will be gathered as follows: a national web-based survey; a consultation process undertaken through the government and nongovernment sector; and targeted consultation with Aboriginal and Torres Strait Islander people, children with disabilities and their families, people with cognitive disability, and people with complex communication needs. The third phase involves a web-based survey to develop a research agenda based on the outcomes of all phases.ResultsWe have started working on 2 parts of the research prioritization exercise. Through the research-mapping exercise we identified 1241 journal articles and book chapters (referred to as research papers) and 225 publicly available reports (referred to as research reports) produced over the 2018-2020 period. Data collection for the national survey has also been completed. We received 973 fully completed responses to the survey. Analysis of these data is currently underway.ConclusionsThis multi-method research agenda–setting study will be the first to provide an indication of the areas of health and social research that people across the Australian disability community consider should be prioritized in disability research funding decisions. Project results from all phases will be made publicly available through reports, open-access journal publications, and Easy Read documents.International Registered Report Identifier (IRRID)DERR1-10.2196/31126

Challenges and opportunities for health policy and systems research funding in the Western Pacific region.

Mixed methods. A desk review was conducted relating to HPSR funding, followed by in-depth interviews. Eight countries and areas were selected to represent characteristics of different health systems. Literature reviews included an analysis of available data relating to HPSR funding and national research and development (R&D) budgets, between 2010 and 2019 (inclusive). In-depth interviews were conducted with 23 stakeholders using an approved interview guide, to assess the attitudes of HPSR funding decision makers towards HPSR, determinants for HPSR and health research funding decisions, and proposals to strengthen HPSR funding and output. There are four main characteristics of HPSR funding in the WPRO: 1) a general absence of studies on HPSR funding and its determinants; 2) no universally accepted understanding of HPSR; 3) an absence of granular health research funding data in general and for HPSR in particular; and 4) HPSR funding is generally perceived to be minimal. In-depth interviews show that HPSR has different interpretations and emphases across WPRO countries, leading to a fragmented landscape where decision makers generally favour biomedical or clinical research. Participants indicate that political involvement increases overall research funding, especially if there is a clear connection between funders, producers and HPSR users. Suggestions from participants to strengthen HPSR include: appropriately using central agencies to generate demand and raise HPSR as a national priority; adopting interdisciplinary HPSR; and building HPSR capacity and organisational structures. HPSR in the Western Pacific region is generally not well funded, with biomedical and public health research often perceived as a higher priority. Although funding is a crucial component of the quality, quantity and relevance of HPSR outputs, HPSR practitioners and organisations must also generate demand for HPSR, build capacity for increasing the quantity and quality of HPSR outputs, and build pathways to translate HPSR outputs into real-world policies.

Scholarly Impact of Academic Ophthalmologists and Vision Scientists in Canada.

ObjectiveTo outline the current impact of Canadian ophthalmology and vision science research as measured by novel research metrics.DesignCross-sectional survey.ParticipantsAll Canadian ophthalmologists (n = 687) and vision scientists (n = 119) with an online bibliometric profile and academic appointment at a major ophthalmology training centre were included.MethodsFaculty lists of Canada’s 15 major academic ophthalmology departments were obtained. Faculty names, appointments, sex, and educational background were recorded. Elsevier’s Scopus database was used to calculate H-index, m-quotient, and total citations for each faculty member. Details around grant funding were obtained through the Canadian Institutes of Health Research (CIHR) Funding Decisions Database.ResultsAverage H-indices were 7.42 ± 7.98 for ophthalmologists and 23.78 ± 15.25 for vision scientists. Higher academic appointment was correlated with higher h-indices and m-quotients (p <0.0001 for both). Most academic departments had significantly more males than females (avg. 71% male, 29% female); however, more equal ratios were seen in faculties in Quebec. No significant differences in research impact were identified between male and female ophthalmologists when controlled for academic appointment and career stage (p > 0.05). In clinical ophthalmology research, the top three departments with the highest average H-indices were Western University, the University of Toronto, and Dalhousie University. The University of British Columbia, Université de Montréal, and McGill University received the most funding from the CIHR in the last 10 years.ConclusionThis study highlights the current scope of ophthalmology and vision science research in Canada. Important trends were identified in research productivity across academic rank, sex, and clinical subspecialty.

Selling science: optimizing the research funding evaluation and decision process

PurposeIn this contribution to EDI's professional insights, the authors develop practical and evidence-based recommendations that are developed for bias mitigation, discretion elimination and process optimization in panel evaluations and decisions in research funding. An analysis is made of how the expectation of “selling science” adds layers of complexity to the evaluation and decision process. The insights are relevant for optimization of similar processes, including publication, recruitment and selection, tenure and promotion.Design/methodology/approachThe recommendations are informed by experiences and evidence from commissioned projects with European research funding organizations. The authors distinguish between three aspects of the evaluation process: written applications, enacted performance and group dynamics. Vignettes are provided to set the stage for the analysis of how bias and (lack of) fit to an ideal image makes it easier for some than for others to be funded.FindingsIn research funding decisions, (over)selling science is expected but creates shifting standards for evaluation, resulting in a narrow band of acceptable behavior for applicants. In the authors' recommendations, research funding organizations, evaluators and panel chairs will find practical ideas and levers for process optimization, standardization and customization, in terms of awareness, accountability, biased language, criteria, structure and time.Originality/valueShowing how “selling science” in research funding adds to the cumulative disadvantage of bias, the authors offer design specifications for interventions to mitigate the negative effects of bias on evaluations and decisions, improve selection habits, eliminate discretion and create a more inclusive process.

The scope of carer effects and their inclusion in decision-making: a UK-based Delphi study

Background and objectiveHealth and social care may affect unpaid (family) carers’ health and wellbeing in addition to patients’ lives. It is recommended that such impacts (carer effects) are considered in decision-making. However, the scope of carer effects and range of decisions where carer effects should be considered is uncertain. This study aimed to identify: (i) how different categories of healthcare and social care were perceived to impact on unpaid carers; and (ii) whether there was consensus about when carer effects should be formally considered in decision-making contexts.MethodsA two round, online Delphi study was conducted with 65 UK-based participants (unpaid carers, care professionals, and researchers) with expertise in dementia, mental health, and stroke. Participants considered two broad forms of ‘interventions’ (patient treatment and replacement care) and two broad forms of ‘organisational change’ (staffing and changes in timing/location of care). Participants assessed the likely impacts of these on unpaid carers and whether impacts should be considered in decision-making.ResultsParticipants predicted interventions and organisational changes would impact on multiple domains of unpaid carers’ lives, with ‘emotional health’ the most likely outcome to be affected. Patient treatment and replacement care services (‘interventions’) were associated with positive impacts across all domains. Conversely, timing/location changes and staffing changes (‘organisational changes’) were perceived to have mixed and negative impacts. There was widespread support (80–81 %) for considering carer effects in research studies, funding decisions, and patient decision-making.ConclusionsThis study highlights a perception that carer effects are widespread and important to consider in economic evaluation and decision-making. It highlights the particular need to measure and value effects on carers’ emotional health and the need to use a societal perspective to avoid cost shifting to unpaid carers when introducing interventions and making organisational changes.

Participatory budgeting for research funding decisions

This paper argues that funding decisions about scientific research should be made using participatory budgeting public engagement methods, to support publicly-funded research translation into evidence-based policymaking, given evident public support. This would reflect international shifts toward public participation in governance and respond to the need for changes in research funding policies, evidenced in studies showing how research funding allocations are inefficient or inequitable. Involving consumers in decision making is an established practice in health research, while experiments involving the public in prioritising between research proposals have promising results. Furthering such methods and moving towards participatory budgeting for research funding would further the shift towards participatory governance and public engagement with science. However, for participatory budgeting to support evidence-based policy by evidencing public support, who is participating is an important consideration. Upstream research does not yet have clear consumers for applying stakeholder participation methods established in health research. Moreover, civic engagement differs from consumer engagement in promoting democratic ideals. Legitimacy for public funding and policy decisions depends on a diversity of public participants in decision-making processes, reflecting broader society rather than a consumer group.

Research priorities of women at risk for preterm birth: findings and a call to action

BackgroundTraditional hierarchical approaches to research give privilege to small groups with decision-making power, without direct input from those with lived experience of illness who bear the burden of disease. A Research Justice framework values the expertise of patients and communities as well as their power in creating knowledge and in decisions about what research is conducted. Preterm birth has persisted at epidemic levels in the United States for decades and disproportionately affects women of color, especially Black women. Women of color have not been included in setting the agenda regarding preterm birth research.MethodsWe used the Research Priorities of Affected Communities protocol to elicit and prioritize potential research questions and topics directly from women of color living in three communities that experience disproportionately high rates of preterm birth. Women participated in two focus group sessions, first describing their healthcare experiences and generating lists of uncertainties about their health and/or healthcare during pregnancy. Women then participated in consensus activities to achieve ‘top-priority’ research questions and topic lists. The priority research questions and topics produced by each group were examined within and across the three regions for similarities and differences.ResultsFifty-four women participated in seven groups (14 sessions) and generated 375 researchable questions, clustered within 22 topics and four overarching themes: Maternal Health and Care Before, During, and After Pregnancy; Newborn Health and Care of the Preterm Baby; Understanding Stress and Interventions to Prevent or Reduce Stress; and Interpersonal and Structural Health Inequities. The questions and topics represent a wide range of research domains, from basic science, translational, clinical, health and social care delivery to policy and economic research. There were many similarities and some unique differences in the questions, topics and priorities across the regions.ConclusionsThese findings can be used to design and fund research addressing unanswered questions that matter most to women at high risk for preterm birth. Investigators and funders are strongly encouraged to incorporate women at the front lines of the preterm birth epidemic in research design and funding decisions, and more broadly, to advance methods to deepen healthcare research partnerships with affected communities.

The Foundation Fighting Blindness Plays an Essential and Expansive Role in Driving Genetic Research for Inherited Retinal Diseases.

The Foundation Fighting Blindness leads a collaborative effort among patients and families, scientists, and the commercial sector to drive the development of preventions, treatments, and cures for inherited retinal diseases (IRDs). When the nonprofit was established in 1971, it sought the knowledge and insights of leaders in the retinal research field to guide its research funding decisions. While the Foundation’s early investments focused on gaining a better understanding of the genetic causes of IRDs, its portfolio of projects would come to include some of the most innovative approaches to saving and restoring vision, including gene replacement/augmentation therapies, gene editing, RNA modulation, optogenetics, and gene-based neuroprotection. In recent years, the Foundation invested in resources such as its patient registry, natural history studies, and genetic testing program to bolster clinical development and trials for emerging genetic therapies. Though the number of clinical trials for such therapies has surged over the last decade, the Foundation remains steadfast in its commitment to funding the initiatives that hold the most potential for eradicating the entire spectrum of IRDs.