Beyond Ticking Boxes: Holistic Assessment of Travel Award Programs Is Essential for Inclusivity

Abstract

One of the major activities any scientific professional society undertakes each year is organizing one or more conferences, at which society members can share work and network with others within their discipline. For any scientist, attending these regular meetings is an important part of establishing and maintaining a connection to the community of practice for a chosen field of study. Though there is typically an emphasis on providing participants with a venue to present research findings, interaction with others is a key motivation for attending a conference (1). In fact, even trainees who do not present work receive measurable benefits from attending meetings. There is no better opportunity to gain an appreciation for the broad range of scientific research and possible career paths available within a seemingly narrow field than by attending a large annual meeting (2). For trainees who do have research findings to share, conference attendance affords a unique opportunity to practice preparing and communicating a well-organized summary of scientific results, which is an essential skill for any working scientist, and central to the process of developing self-efficacy, research confidence, and a sense of belonging in science (3). Additionally, many other professional development opportunities offered by professional societies take place at these annual conferences and, thus, can only be accessed by attending these events.Of course, the benefits of attending a conference come with a cost—the registration fees and travel expenses involved. Professional societies commonly offer a number of travel awards to defray these costs for society members in need. Many professional societies use travel award programs to foster inclusion and facilitate the professional development of underrepresented minority (URM) scientists. All member societies that participate in the Alliance to Catalyze Change for Equity on STEM Success (ACCESS) do so to some degree. Members of this metaorganization recently came together to share different approaches (4). The practices of the Biophysical Society (BPS), one of the ACCESS member societies, may be of particular interest to The Biophysicist readership. Our society spends US$85,252 each year on travel awards. In addition, 2 paid staff members are dedicated to the administration of the program, and members of 5 committees volunteer time to judge applications. Although some other societies with larger membership rolls may spend more on travel awards each year, this is a substantial commitment for a society with about 7,500 members.The BPS travel award program is actually an amalgam of multiple programs sponsored by different committees within the society, each targeting a certain group of members. The Committee on Inclusion and Diversity (CID) supports applications from URM students and postdoctoral fellows, while the Committee for Professional Opportunities for Women funds applications from female postdoctoral fellows and midcareer scientists. The Education Committee supports applications from all students and postdoctoral fellows, while the International Relations Committee specifically funds applications from scientists working outside of the United States. In addition, the Membership Committee offers bridging funds travel grants to all regular members of the society, including independent investigators. This patchwork of awards collectively serves society members at all career levels and from URM groups. In fact, many society members are eligible to receive an award from more than one travel award program. Prior to 2019, candidates applied directly to each program, resulting in many duplicate applications that had to be evaluated by using volunteer labor. Each committee had a limited number of awards to distribute, and it was unclear whether the number was well matched to the needs of the targeted group. With respect to URM scientists, the group that ACCESS has a shared investment in supporting, the CID presented 18 awards each year. However, URM scientists could (and most certainly did) apply for travel awards from other committees. Because applicant demographic information was not collected by all of the committees administering awards, it was unclear just how many URM scientists were applying for and receiving awards beyond those offered by the CID. Moreover, it was not clear whether the CID travel award was attractive to URM scientists, given its specific labeling. As pointed out by Segarra et al. (4), there are multiple motivations beyond the monetary value of the travel award that drive people to apply. The receipt of a travel award is often viewed as a mark of distinction (5, 6) but can be accompanied by a stigma due to the widespread perception that programs that support URM are lifting up less competent individuals who could not achieve success without affirmative action (7, 8). Unfortunately, singling out URM scientists in the award process may actually exacerbate the feelings of isolation they experience and reduce the perceived value of the award.To address these concerns, the BPS moved to a single, centralized application for all travel awards. The new format streamlined the process for applicants and reduced the workload for volunteer judges by eliminating the need to assess duplicate applications. Under this new system, all applicants were asked demographic questions to determine eligibility for different travel award programs. Surprisingly, compared with the number of URM individuals who self-identified by applying for the CID award in 2018, the number of URM travel award applicants in 2019 nearly doubled. Meanwhile, the success rate for these applicants remained comparable, and, apparently, more URM scientists received awards under the centralized application system than did under a system with a certain number of awards set aside for this group (4). The change to a centralized application system for BPS travel awards was accompanied by a change from a system of several named awards to all recipients receiving an award with the same name: the BPS travel award. It seems likely that such an award is perceived as having more value than an award that highlights the recipient's minoritized status.This result shows that as a society, the BPS can indeed make strides to better support URM biophysicists. Moving forward, it is crucial to highlight why that is important. As many BPS members can attest, there are not many dedicated biophysics programs in universities, which can leave biophysicists at all career levels feeling isolated. According to the National Center for Education Statistics (9), there were 4,313 degree-granting postsecondary institutions in the United States during the 2017 to 2018 academic year. On the basis of self-reported data collected by the BPS, 141 U.S.-based colleges and universities have a biophysics program of some sort. Although these data may not include every program, these numbers still suggest that less than 5% of higher education institutions have an identifiable biophysics program. About half of these programs are at the graduate level only, so most undergraduates will have little, if any, exposure to the field before they make the decision to pursue graduate study. Furthermore, those who do learn about the field in time to choose to specialize in biophysics prior to beginning doctoral studies do not have many choices. This lack of community can have an outsized effect on URM individuals who choose to enter that field.Emerging evidence indicates that increased workforce diversity has benefits to the progress of science, technology, engineering, and math (STEM) disciplines. Recent findings indicate that scientists from underrepresented (UR) groups, both in terms of gender and race, are more likely than members of well-represented (WR) groups to make new connections between scientific concepts (10). This kind of innovation is the underpinning of the field of biophysics: we need scientists who can identify and exploit the links between what we know about the physics of inanimate objects and what we want to understand about the processes of life. However, the same dataset that demonstrates that “diversity breeds innovation” also reveals that scientists from UR groups are more likely than their counterparts from WR groups to see their novel contributions discounted (10). This kind of erasure may discourage URM scientists from persisting in biophysics, and we have no way of knowing the cost of the associated loss of talent.However, even if there were no evidence suggesting that welcoming and supporting URM scientists into the biophysics community might lead to an increase in innovation, all scientists deserve to have opportunities to pursue their interests. Supporting the participation of URM scientists means addressing the barriers that they often encounter. Marginalized status often correlates with a lack of resources; many URM scientists do not have the funds available to support attendance at conferences (11–13), resulting in missed opportunities to network with others. Given the scarcity of institutions that have a critical mass of biophysicists, URM biophysicists' networking needs may not be met by home institutions because they do not have a chance to meet others pursuing scientific questions that lie at the interfaces between “traditional” departments. Conferences offer opportunities to meet and interact with established members of the community of practice, as well as to access professional development opportunities and indicators of excellence, such as travel awards. As a result, conference attendance represents a critical building block to career success in any field and an important way to counteract the lack of resources that can undermine URM scientists. Although we have all experienced unprecedented social isolation as a result of widespread stay-at-home orders during the coronavirus disease 2019 pandemic, for members of marginalized groups in STEM, the impact is likely more severe than for those who have already developed a sense of belonging to a community within their discipline.If societies wish to use travel award programs to provide the kinds of support URM scientists truly need, it follows that the architects of these programs must formulate clear goals, collect data to assess whether these goals are being achieved, and make changes as necessary to maximize the attainment of the desired outcomes. Certainly, a first step is to determine how many URM scientists a program is reaching. The results of the recent changes to the BPS travel award program demonstrate that small adjustments can have large effects. However, if societies want to move beyond simply increasing the numbers and attempt to have an impact on the level of success for URM scientists, careful assessment of travel award programs is critical. A reasonable overarching goal for all society programs that target URM scientists would be to support their persistence in science in general and to make it more likely that these scientists choose to stay in the society's field. Given the known attrition rates of URM trainees from STEM fields, particularly at career transition points (14, 15), a more specific goal may be to use travel award programs to help URM scientists navigate these transitions more successfully. If societies want URM scientists to thrive, all programs targeting URM scientists should aim to increase the sense of belonging within the society and the scientific community.How can societies know if their travel award program is having the desired impact? The best way to know what is working in a program (and what is not) is to assess the program by using a mixture of different indicators. Societies should consider the inputs, the process, the outputs, and the outcomes of the programs. Analysis of all of these indicators will help societies know if their efforts are having the desired impact and may help reveal specific needs for additional support beyond monetary awards that will amplify the positive effects of travel awards. The ACCESS societies have come together to create a survey of travel award recipients to support societies in this effort (available as Supplemental Material). We have used the travel award assessment survey administered by the American Society for Cell Biology (supported by the National Institutes of Health grants T36-GM008622 and R25GM116707) as a starting point and have developed it further to create an instrument that includes questions that touch on the process and outputs of a travel award program, as well as the outcomes for travel award recipients. We have developed this survey to help societies move beyond the stage of ticking off boxes for numbers of participants in specific demographic groups. Rather, the ACCESS Common Travel Award Evaluation Survey probes a broad range of indicators for the goals identified previously: increasing the diversity of scientists reached by the travel award program; supporting the persistence of URM scientists in STEM in general (and in our field specifically); supporting URM scientists at critical career transition points; and ensuring that URM scientists feel that they have a place in our scientific societies.Input indicators probe what the society is investing in the program; they are not captured by surveying award recipients. Therefore, in addition to using the ACCESS Common Travel Award Evaluation Survey, we recommend that societies take time to consider inputs into their award program. Inputs that should be considered include the money spent by a society and the time invested by staff members and volunteers during program implementation. There is a limit to how many programs any society can run, so another important input to consider is focus. By choosing to focus on travel awards, other diversity initiatives may fall by the wayside. Societies should evaluate how all programming works together to accomplish broader goals. It may be that a well-designed travel award program is the missing piece that will complete a unified plan for increasing diversity. Alternatively, it may be that a society can achieve goals with less effort or expense. Put another way, program evaluation may reveal that efforts need to be refocused to maximize the impact on the society's goals. Evaluating program inputs is also fundamental for maintaining commitment to the undertaking. Stakeholders, such as donors and corporate underwriters, may be rightfully concerned about the return on investment. In addition, many travel award programs depend on volunteer effort to recruit applicants, vet awards, and plan and run associated events. Societies should consider where that effort is coming from because URM scientists often carry an excess burden when it comes to diversity initiatives and that burden is often not acknowledged (16). All academics need to be recognized for their volunteer work in service to the profession, especially those from UR groups.Process indicators are important for understanding how a travel award program is running. Even with careful and thoughtful planning, it is impossible to predict what will happen when a program is initiated. Process indicators let us know who is actually applying to the program. As the data from the BPS travel award discussed previously demonstrates, the best way to maximize diversity among applicants may not be immediately obvious. Process indicators can also help us understand what the application process is like from the applicant perspective. Finding out how applicants heard about the program can allow societies to better target advertising and outreach. Successful applicants needing multiple attempts before receiving an award can be an indication of whether the requirements are clear and attainable. The fraction of qualified applicants that receive awards is also an important indicator of whether the scale of the program matches the need within the society. Other process indicators can help societies understand whether the travel award program is actually meeting the needs of applicants. For instance, it is important to understand what might lead an applicant to decline an award. Were they unable to cover extra costs beyond the monetary value of the award? Did they feel that meeting attendance was not worth the expense? It is easy for established members of a scientific society to see the value of meeting attendance, but to attract and retain URM scientists to become active society members, it is crucial that there are meeting sessions and activities that are attractive to them and that these items are well publicized. For this reason, we recommend asking travel award participants about their knowledge of and participation in conference activities targeted toward this group. The data collected can be used to ensure that meeting planners make adjustments or expand programs in ways that are likely to further society goals.Output indicators show what the program has achieved in the short term regarding goals. These indicators include numbers that are simple to interpret but not always simple to collect accurately. For instance, if we want to know how many URM scientists are receiving travel awards or participating in our scientific society or both, we must ask questions about recipient demographics. These questions are inherently personal in nature, and they serve as reminders to URM respondents that they are indeed members of a minority group that may be expected to be less capable than members of the majority. This can have a negative impact on the responses given for the rest of the survey, a phenomenon known as stereotype threat. Although we do feel that it is important for societies to ask these difficult questions, we recommend placing questions about race, ethnicity, and gender at the end of the survey to avoid highlighting respondents' minoritized status. The outputs of a travel award can, and should, go beyond merely increasing the number of people from certain groups. In addition, we recommend that any survey of travel award recipients also includes questions that allow recipients to report the perceived benefits from the award, beyond the monetary value.Given the goals we have articulated, what we really should be attempting to find out is whether the receipt of travel awards has a positive impact on the experiences of URM scientists as they participate in our society's annual meeting and whether these kinds of experiences will make a difference in career trajectories and sustained society membership. These outcomes are difficult to measure; the effects occur over a longer period than any conference. However, we can start to assess these indicators by asking travel award recipients about what they are taking away from the conference and how they are doing in their careers. Although it may be difficult to ask participants whether they feel welcome in our scientific society and expect to receive an honest response, there are good alternative questions we can use to probe this important outcome. When we feel that we belong in a society or discipline, we are driven to participate fully and regularly. By asking travel awardees about current and planned participation in society activities, we can collect data that may help us gauge whether URM scientists do feel that they belong in our society.In conclusion, we are sharing the ACCESS Common Travel Award Evaluation Survey in its entirety with the goal of assisting any scientific society that wishes to assess whether the travel award program is helping them meet a goal of fully welcoming scientists from all backgrounds into the community and supporting them as they make a career in their discipline. Societies may choose to use some or all of the questions in the survey, depending on needs and the amount of time they feel comfortable asking participants to spend. Questions are labeled according to the type of indicator they are intended to probe. Process indicators will help tailor the program and ensure that it is reaching the right people. Some programs may already collect demographic data on travel awardees, but that type of data is not sufficient to determine if a program is successful. We, therefore, suggest including additional questions probing participants' goals in attending the conference and whether they were able to meet them. Finally, we recommend that societies use this survey (or a modified version) as a means to seek answers to the bigger questions that drive us to offer programs targeting URM scientists. Until we move beyond ticking boxes, we will never know whether our programs are making a difference in the effort to achieve full participation of all who are interested in the scientific endeavor.Supplemental ACCESS Common Travel Award Evaluation Survey is available at: https://doi.org/10.35459/tbp.2021.000195.s1.We thank the American Society for Cell Biology for sharing the travel award assessment survey, developed with the support of National Institutes of Health grants T36-GM008622 and R25GM116707. We also thank Patricia B. Campbell for many enlightening discussions on the topic of program evaluation. The ACCESS work is supported by the National Science Foundation (grant 1744098) to MRA and VAS.